For every release of LemonTree, we run a set of tests with different test-models to examine the change of both the memory consumption and the total diff/merge times.
This page shows the results of these tests.
As shown by the test results, our goal is to continuously both improve the performance and decrease the memory consumption of LemonTree.
Nevertheless, changes to the supported feature-set may impact both performance and memory consumption negatively. The Remarks section under the test results notes such changes.
Environment
Our test machine has the following setup:
- Operating System: Windows 10 (64 Bit)
- Processor: i7-6700HQ CPU @2.60 GHz
- Memory: 16 GB
Test Case Description
The test results provided on this page are based on a fixed set of test models, which we diff with every released version of LemonTree.
Details on the used test models can be found here:
Measurement Methods
To understand the measurement methods we have used to compile this data or conduct measurements on your own model for a comparison, please refer to this page:
By applying our measurement methods to your own models, you can calculate comparison values to approximate how LemonTree will perform for your models based on the data below.
Test results
Small Model | ||||||
LT Version | Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage |
---|---|---|---|---|---|---|
2.6.3 (latest release) | 23 | 6 | 7s | 1s | 1s | 470 MB |
2.6.2 | 23 | 6 | 6s | 1s | 1s | 254 MB |
2.6.1 | 23 | 6 | 7s | 1s | 1s | 237 MB |
2.6 | 23 | 6 | 4s | 1s | 1s | 242 MB |
2.5 | 23 | 6 | 8s | 1s | 1s | 255 MB |
2.4.1 | 23 | 6 | 19s | 1s | 1s | 463 MB |
2.4 | 23 | 6 | 19s | 1s | 1s | 300 MB |
2.3 | 23 | 6 | 20s | 1s | 1s | 297 MB |
2.2 | 23 | 6 | 18s | 1s | 1s | 292 MB |
2.1.4 | 23 | 6 | 13s | 1s | 1s | 263 MB |
2.1.3 | 23 | 6 | 13s | 1s | 1s | 265 MB |
2.1.0 | 26 | 6 | 9s | 1s | 1s | 257 MB |
2.0 | 26 | 6 | 22s | 1s | 1s | 257 MB |
Medium Model | ||||||
LT Version | Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage |
2.6.3 (latest release) | 1.105 | 399 | 2m24s | 16s | 5s | 5,214 MB |
2.6.2 | 1.105 | 399 | 1m48s | 14s | 5s | 5.165 MB |
2.6.1 | 1.105 | 399 | 2m17s | 15s | 9s | 4.988 MB |
2.6 | 1.105 | 399 | 1m51s | 18s | 7s | 5.436 MB |
2.5 | 1.105 | 399 | 1m52s | 17s | 6s | 4.937 MB |
2.4.1 | 1.093 | 389 | 2m10s | 19s | 15s | 5.940 MB |
2.4 | 1.093 | 3895 | 2m07s | 19s | 17s | 5.908 MB |
2.3 | 1.029 | 387 | 2m17s | 13s | 14s | 5.928 MB |
2.2 | 1.033 | 386 | 2m23s | 13s | 20s | 6.184 MB |
2.1.4 | 1.028 | 386 | 1m57s | 32s | 12s | 5.595 MB |
2.1.3 | 1.028 | 386 | 1m51s | 36s | 13s | 5.630 MB |
2.1 | 1.184 | 386 | 1m45s | 23s | 21s | 5.980 MB |
2.0 | 1.184 | 386 | 2m16s | 20s | 1m | 5.454 MB |
Large Model | ||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | |
2.6.3 (latest release) | 68.547 | 6.979 | 2m11s | 7m57s | 17m24s | 11.447 MB |
2.6.2 | 68.547 | 6.979 | 2m14s | 7m41s | 17m35s | 11.135 MB |
2.6.1 | 68.5479 | 6.979 | 2m18s | 7m46s | 22m42s | 11.374 MB |
2.6 | 68.546 | 6.979 | 2m35s | 9m41s | 24m02s | 11.605 MB |
2.5 | 68.546 | 6.979 | 2m21s | 8m26s | 19m10s | 11.237 MB |
2.4.1 | 68.542 | 6.979 | 2m30s | 10m4s | 27m57s | 11.750 MB |
2.4 | 68.542 | 6.979 | 2m52s | 10m49s | 24m14s | 11.809 MB |
2.3 | 68.549 | 6.979 | 3m13s1 | 8m13s | 29m14s4 | 10.548 MB |
2.2.0 | 68.548 | 6.979 | 2m09s | 7m55s | 21m53s | 10.486 MB |
2.1.4 | 68.548 | 6.979 | 2m22s | 17m58s | 22m38s | 10.011 MB |
2.1.3 | 68.544 | 6.979 | 2m21s | 18m2s | 23m26s | 9.888 MB |
2.1 | 75.523 | 6.979 | 2m47s | 14m19s | 24m16s | 9.406 MB |
2.0 | 2m30s | 15m | 49m | 9.525 MB | ||
EAExample | ||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | |
2.6.3 (latest release) | 4.833 | 564 | 10s | 16s | 3m28s | 750 MB |
2.6.2 | 4.833 | 564 | 9s | 12s | 3m18s | 770 MB |
2.6.1 | 4.833 | 564 | 10s | 13s | 3m14s | 775 MB |
2.6 | 4.833 | 564 | 8s | 12s | 3m15s | 750 MB |
2.5 | 4.833 | 564 | 10s | 17s | 3m37s | 747 MB |
2.4.1 | 4.719 | 564 | 21s | 14s | 5m12s | 774 MB |
2.4 | 4.719 | 564 | 22s | 16s | 4m59s | 826 MB |
2.3 | 5.019 | 564 | 22s | 16s | 4m57s | 841 MB |
2.2.0 | 5.018 | 564 | 19s | 14s | 4m53s | 805 MB |
2.0.0 | 23s | 10s | 7m13s | 764 MB | ||
2.1.4 | 5.102 | 564 | 13s | 32s | 4m44s | 789 MB |
2.1.3 | 5.102 | 564 | 12s | 36s | 4m30s | 782 MB |
2.1.0 | 5.666 | 564 | 13s | 28s | 4m57s | 738 MB |
Tagged Value Model | ||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | |
2.6.3 (latest release) | 5.963 | 263 | 1m19s | 50s | 4m15s | 4.558 MB |
2.6.2 | 5.963 | 263 | 1m26s | 27s | 3m18s | 4.313 MB |
2.6.1 | 5.963 | 263 | 1m30s | 26s | 3m16s | 4.257 MB |
2.6 | 5.963 | 263 | 1m18s | 24s | 3m18s | 4.256 MB |
2.5 | 5.964 | 288 | 1m24s | 32s | 3m39s | 3.839 MB |
2.4.1 | 5.963 | 263 | 2m22s | 29s | 3m53s | 5.091 MB |
2.4 | 5.963 | 263 | 2m17 | 25s | 4m12s | 4.266 MB |
2.3 | 5.961 | 263 | 1m47s | 33s | 4m10s | 4.073 MB |
2.2.0 | 5.961 | 263 | 1m46s | 31s | 3m57s | 4.541 MB |
2.1.4 | 5.961 | 263 | 1m19s | 59s | 3m54s | 4.359 MB |
2.1.3 | 5.960 | 263 | 1m18s | 1m25s | 3m57s | 4.428 MB |
2.1 | 6.107 | 263 | 1m52s | 1m | 2m39s | 3.040 MB |
2.0 | 6.113 | 263 | 1m52s | 1m | 5m54s | 4.219 MB |
TV Model (Triple Size) | ||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | |
2.6.3 (latest release) | 17.871 | 790 | 3m31s | 2m28s | 13m54s | 10.839 MB |
2.6.2 | 17.871 | 790 | 3m56s | 1m15s | 10m14s | 9.156 MB |
2.6.1 | 17.871 | 790 | 3m38s | 1m18s | 10m13s | 9.701 MB9 |
2.6 | 17.871 | 790 | 3m29s | 1m29s | 10m29s | 9.165 MB |
2.5 | 17.871 | 790 | 3m44s | 1m46s | 10m57s | 8.875 MB |
2.4.1 | 17.871 | 790 | 6m35s | 1m41s | 13m20s | 9.040 MB |
2.4 | 17.871 | 790 | 6m46s | 1m32s | 14m41s | 9.420 MB |
2.3 | 17.869 | 790 | 4m16s | 1m29s | 12m23s | 8.589 MB |
2.2.0 | 17.869 | 790 | 4m40s | 1m18s | 11m56s | 8.496 MB |
2.1.4 | 17.8692 | 790 | 4m45s | 2m58s | 11m27s | 8.685 MB |
2.1.3 | 17.8681 | 790 | 4m25s | 4m23s | 12m58s | 9.290 MB |
2.1 | 18.316 | 790 | 7m19s | 3m21s | 13m9s | 8.000 MB |
Remarks
1: Impacted Elements are reduced because we removed the Diagrams from the "Impacted Elements List".
2: LemonTree improved handling of stereotypes t_stereotype vs. t_xref: In some models stereotypes are stored redundant/inconsistent. That may lead to small differences in # of impacted elements.
3: LemonTree 2.2.2 also diffs Default Diagram
4: The test environment got a disc encryption since the last performance tests. We assume that's why the large models take longer on writing/loading. Older versions of LemonTree now also need longer with that models on writing.
5: LemonTree 2.4 diffs "CreatedDate" and "ModifiedDate". Also Stereotype handling is now different.
6: LemonTree 2.5 also diffs Scripts
7: LemonTree 2.5 detects more differences in Pos (0 vs "").
8: LemonTree 2.6 Version 2.6.0 has increased the performance of diffing and merging, but consumes more memory than version 2.5.0. This had a quite severe impact on the performance results
for our "Large Model" performance test, as it was approaching the cap of available memory for the test machine, which lead to page swapping.
9: LemonTree 2.6.1 processes all previously unhandled custom properties of elements