For every release of LemonTree, we run a set of tests with different test-models to examine the change of both the memory consumption and the total diff/merge times.

This page shows the results of these tests.

As shown by the test results, our goal is to continuously both improve the performance and decrease the memory consumption of LemonTree.

Nevertheless, changes to the supported feature-set may impact both performance and memory consumption negatively. The Remarks section under the test results notes such changes.

Environment

Our test machine has the following setup:

Operating System: Windows 10 (64 Bit)
Processor: i7-6700HQ CPU @2.60 GHz
Memory: 16 GB

Test Case Description

The test results provided on this page are based on a fixed set of test models, which we diff with every released version of LemonTree.

Details on the used test models can be found here:

Test Case Description

Measurement Methods

To understand the measurement methods we have used to compile this data or conduct measurements on your own model for a comparison, please refer to this page:

Measurement Methods

By applying our measurement methods to your own models, you can calculate comparison values to approximate how LemonTree will perform for your models based on the data below.

Test results

LT Version	Impacted elements	Impacted diagrams	1) Loading models	2) Premerge calculation	3) Writing merge file	Max RAM usage
Small Model
2.6.3 (latest release)	23	6	7s	1s	1s	470 MB
2.6.2	23	6	6s	1s	1s	254 MB
2.6.1	23	6	7s	1s	1s	237 MB
2.6	23	6	4s	1s	1s	242 MB
2.5	23	6	8s	1s	1s	255 MB
2.4.1	23	6	19s	1s	1s	463 MB
2.4	23	6	19s	1s	1s	300 MB
2.3	²³	6	20s	1s	1s	297 MB
2.2	23	6	18s	1s	1s	292 MB
2.1.4	23	6	13s	1s	1s	263 MB
2.1.3	23	6	13s	1s	1s	265 MB
2.1.0	26	6	9s	1s	1s	257 MB
2.0	26	6	22s	1s	1s	257 MB
Medium Model
LT Version	Impacted elements	Impacted diagrams	1) Loading models	2) Premerge calculation	3) Writing merge file	Max RAM usage
2.6.3 (latest release)	1.105	399	2m24s	16s	5s	5,214 MB
2.6.2	1.105	399	1m48s	14s	5s	5.165 MB
2.6.1	1.105	399	2m17s	15s	9s	4.988 MB
2.6	1.105	399	1m51s	18s	7s	5.436 MB
2.5	1.105	399	1m52s	17s	6s	4.937 MB
2.4.1	1.093	389	2m10s	19s	15s	5.940 MB
2.4	1.093	389⁵	2m07s	19s	17s	5.908 MB
2.3	1.029	387	2m17s	13s	14s	5.928 MB
2.2	1.033	386	2m23s	13s	20s	6.184 MB
2.1.4	1.028	386	1m57s	32s	12s	5.595 MB
2.1.3	1.028	386	1m51s	36s	13s	5.630 MB
2.1	1.184	386	1m45s	23s	21s	5.980 MB
2.0	1.184	386	2m16s	20s	1m	5.454 MB
Large Model
	Impacted elements	Impacted diagrams	1) Loading models	2) Premerge calculation	3) Writing merge file	Max RAM usage
2.6.3 (latest release)	68.547	6.979	2m11s	7m57s	17m24s	11.447 MB
2.6.2	68.547	6.979	2m14s	7m41s	17m35s	11.135 MB
2.6.1	68.547⁹	6.979	2m18s	7m46s	22m42s	11.374 MB
2.6	68.546	6.979	2m35s	9m41s	24m02s	11.605 MB
2.5	68.546	6.979	2m21s	8m26s	19m10s	11.237 MB
2.4.1	68.542	6.979	2m30s	10m4s	27m57s	11.750 MB
2.4	68.542	6.979	2m52s	10m49s	24m14s	11.809 MB
2.3	68.549	6.979	3m13s¹	8m13s	29m14s⁴	10.548 MB
2.2.0	68.548	6.979	2m09s	7m55s	21m53s	10.486 MB
2.1.4	68.548	6.979	2m22s	17m58s	22m38s	10.011 MB
2.1.3	68.544	6.979	2m21s	18m2s	23m26s	9.888 MB
2.1	75.523	6.979	2m47s	14m19s	24m16s	9.406 MB
2.0			2m30s	15m	49m	9.525 MB
EAExample
	Impacted elements	Impacted diagrams	1) Loading models	2) Premerge calculation	3) Writing merge file	Max RAM usage
2.6.3 (latest release)	4.833	564	10s	16s	3m28s	750 MB
2.6.2	4.833	564	9s	12s	3m18s	770 MB
2.6.1	4.833	564	10s	13s	3m14s	775 MB
2.6	4.833	564	8s	12s	3m15s	750 MB
2.5	4.833	564	10s	17s	3m37s	747 MB
2.4.1	4.719	564	21s	14s	5m12s	774 MB
2.4	4.719	564	22s	16s	4m59s	826 MB
2.3	5.019	564	22s	16s	4m57s	841 MB
2.2.0	5.018	564	19s	14s	4m53s	805 MB
2.0.0			23s	10s	7m13s	764 MB
2.1.4	5.102	564	13s	32s	4m44s	789 MB
2.1.3	5.102	564	12s	36s	4m30s	782 MB
2.1.0	5.666	564	13s	28s	4m57s	738 MB
Tagged Value Model
	Impacted elements	Impacted diagrams	1) Loading models	2) Premerge calculation	3) Writing merge file	Max RAM usage
2.6.3 (latest release)	5.963	263	1m19s	50s	4m15s	4.558 MB
2.6.2	5.963	263	1m26s	27s	3m18s	4.313 MB
2.6.1	5.963	263	1m30s	26s	3m16s	4.257 MB
2.6	5.963	263	1m18s	24s	3m18s	4.256 MB
2.5	5.964	288	1m24s	32s	3m39s	3.839 MB
2.4.1	5.963	263	2m22s	29s	3m53s	5.091 MB
2.4	5.963	263	2m17	25s	4m12s	4.266 MB
2.3	5.961	263	1m47s	33s	4m10s	4.073 MB
2.2.0	5.961	263	1m46s	31s	3m57s	4.541 MB
2.1.4	5.961	263	1m19s	59s	3m54s	4.359 MB
2.1.3	5.960	263	1m18s	1m25s	3m57s	4.428 MB
2.1	6.107	263	1m52s	1m	2m39s	3.040 MB
2.0	6.113	263	1m52s	1m	5m54s	4.219 MB
TV Model (Triple Size)
	Impacted elements	Impacted diagrams	1) Loading models	2) Premerge calculation	3) Writing merge file	Max RAM usage
2.6.3 (latest release)	17.871	790	3m31s	2m28s	13m54s	10.839 MB
2.6.2	17.871	790	3m56s	1m15s	10m14s	9.156 MB
2.6.1	17.871	790	3m38s	1m18s	10m13s	9.701 MB⁹
2.6	17.871	790	3m29s	1m29s	10m29s	9.165 MB
2.5	17.871	790	3m44s	1m46s	10m57s	8.875 MB
2.4.1	17.871	790	6m35s	1m41s	13m20s	9.040 MB
2.4	17.871	790	6m46s	1m32s	14m41s	9.420 MB
2.3	17.869	790	4m16s	1m29s	12m23s	8.589 MB
2.2.0	17.869	790	4m40s	1m18s	11m56s	8.496 MB
2.1.4	17.869²	790	4m45s	2m58s	11m27s	8.685 MB
2.1.3	17.868¹	790	4m25s	4m23s	12m58s	9.290 MB
2.1	18.316	790	7m19s	3m21s	13m9s	8.000 MB

Remarks

1: Impacted Elements are reduced because we removed the Diagrams from the "Impacted Elements List".

2: LemonTree improved handling of stereotypes t_stereotype vs. t_xref: In some models stereotypes are stored redundant/inconsistent. That may lead to small differences in # of impacted elements.

3: LemonTree 2.2.2 also diffs Default Diagram

4: The test environment got a disc encryption since the last performance tests. We assume that's why the large models take longer on writing/loading. Older versions of LemonTree now also need longer with that models on writing.

5: LemonTree 2.4 diffs "CreatedDate" and "ModifiedDate". Also Stereotype handling is now different.

6: LemonTree 2.5 also diffs Scripts

7: LemonTree 2.5 detects more differences in Pos (0 vs "").

8: LemonTree 2.6 Version 2.6.0 has increased the performance of diffing and merging, but consumes more memory than version 2.5.0. This had a quite severe impact on the performance results
for our "Large Model" performance test, as it was approaching the cap of available memory for the test machine, which lead to page swapping.

9: LemonTree 2.6.1 processes all previously unhandled custom properties of elements

Page tree

Performance and Memory

Environment

Test Case Description

Test results

Remarks