We run tests with different releases of LemonTree against different test-models to examine the improvement of the memory usage and performance.
This page gives an overview of the performance and memory usage and describes the test-models.
Performance and memory usage are continually improved in LemonTree.
Environment
Win10, 16GB RAM i7-6700HQ CPU @2.60 GHz, 64 BIT |
Test description
We have different test setups which run against the recent releases of LemonTree.
1: Small model, with few changes in 3-way comparison
These models are the example models which come with the LemonTree installation.
2: Medium model with several changes in 3-way comparison
A larger model which is diffed in a 3-way comparison. There are changes in A (respective to Base) and B (respective to Base). This test-case can be regarded as standard use-case for LemonTree, when used with models that are under version control (GIT, SVN).
3: Large model with a lot of changes in 2-way comparison
A really large model with a lot changes and even more elements to be checked by LemonTree. This is quite challenging for LemonTree. In a 2-way comparison Model A is also regarded as Base model.
4: EAExample (for EA 14), compared with empty model
When comparing an empty model with another model the diff results in a lot of new objects which leads to special challenges for LemonTree. In older versions of LemonTree (< 1.5) this was not successful within the test environment.
5: Tagged Value model with a lot of TaggedValues in 2-way comparison
TaggedValues needed a lot of RAM in older versions of LemonTree (<1.8).
Testmodels
Test Name | Version | File size | Total packages | Total diagrams | Total elements | Total connectors | Total TaggedValues* | Description |
---|---|---|---|---|---|---|---|---|
Small Model | Base | 3 MB | 6 | 17 | 202 | 152 | 242 | |
A | 3 MB | 6 | 17 | 205 | 154 | 242 | ||
B | 3 MB | 6 | 17 | 203 | 153 | 242 | ||
Medium Model | Base | 128 MB | 758 | 2.038 | 21.922 | 26.075 | 88.326 | |
A | 128 MB | 739 | 1.988 | 21.685 | 25.594 | 88.810 | ||
B | 133 MB | 739 | 1.987 | 21.670 | 25.568 | 88.331 | ||
Large Model | Base | 233 MB | 8.490 | 6.733 | 114.962 | 69.652 | 32.788 | same as A |
A | 233 MB | 8.490 | 6.733 | 114.962 | 69.652 | 32.788 | same as Base | |
B | 195 MB | 8.059 | 6.252 | 107.111 | 64.345 | 29.157 | ||
EaExample | Base | 1,4 MB | 1 | 0 | 0 | 0 | 0 | empty model |
A | 1,4 MB | 1 | 0 | 0 | 0 | 0 | empty model | |
B | 20 MB | 427 | 564 | 4.656 | 2.623 | 29.460 | ||
TV Model | Base | 142 MB | 338 | 909 | 46.380 | 22.819 | 316.458 | same as A |
A | 142 MB | 338 | 909 | 46.380 | 22.819 | 316.458 | same as Base | |
B | 152 MB | 347 | 982 | 50.810 | 27.289 | 336.759 |
Test results
Test results
Small Model | |||||||
Test name | LT Version | Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage |
---|---|---|---|---|---|---|---|
2.6 | 23 | 6 | 4s | 1s | 1s | 242 MB | |
2.5 | 23 | 6 | 8s | 1s | 1s | 255 MB | |
2.4.1 | 23 | 6 | 19s | 1s | 1s | 463 MB | |
2.4 | 23 | 6 | 19s | 1s | 1s | 300 MB | |
2.3 | 23 | 6 | 20s | 1s | 1s | 297 MB | |
2.2 | 23 | 6 | 18s | 1s | 1s | 292 MB | |
2.1.4 | 23 | 6 | 13s | 1s | 1s | 263 MB | |
2.1.3 | 23 | 6 | 13s | 1s | 1s | 265 MB | |
2.1.0 | 26 | 6 | 9s | 1s | 1s | 257 MB | |
2.0 | 26 | 6 | 22s | 1s | 1s | 257 MB | |
Medium Model | |||||||
LT Version | Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | |
2.6 | 1.105 | 399 | 1m51s | 18s | 7s | 5.436 MB | |
2.5 | 1.105 | 399 | 1m52s | 17s | 6s | 4.937 MB | |
2.4.1 | 1.093 | 389 | 2m10s | 19s | 15s | 5.940 MB | |
2.4 | 1.093 | 3895 | 2m07s | 19s | 17s | 5.908 MB | |
2.3 | 1.029 | 387 | 2m17s | 13s | 14s | 5.928 MB | |
2.2 | 1.033 | 386 | 2m23s | 13s | 20s | 6.184 MB | |
2.1.4 | 1.028 | 386 | 1m57s | 32s | 12s | 5.595 MB | |
2.1.3 | 1.028 | 386 | 1m51s | 36s | 13s | 5.630 MB | |
2.1 | 1.184 | 386 | 1m45s | 23s | 21s | 5.980 MB | |
2.0 | 1.184 | 386 | 2m16s | 20s | 1m | 5.454 MB | |
Large Model | |||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | ||
2.6 | 68.546 | 6.979 | 2m35s | 9m41s | 24m02s | 11.605 MB | |
2.5 | 68.546 | 6.979 | 2m21s | 8m26s | 19m10s | 11.237 MB | |
2.4.1 | 68.542 | 6.979 | 2m30s | 10m4s | 27m57s | 11.750 MB | |
2.4 | 68.542 | 6.979 | 2m52s | 10m49s | 24m14s | 11.809 MB | |
2.3 | 68.549 | 6.979 | 3m13s1 | 8m13s | 29m14s4 | 10.548 MB | |
2.2.0 | 68.548 | 6.979 | 2m09s | 7m55s | 21m53s | 10.486 MB | |
2.1.4 | 68.548 | 6.979 | 2m22s | 17m58s | 22m38s | 10.011 MB | |
2.1.3 | 68.544 | 6.979 | 2m21s | 18m2s | 23m26s | 9.888 MB | |
2.1 | 75.523 | 6.979 | 2m47s | 14m19s | 24m16s | 9.406 MB | |
2.0 | 2m30s | 15m | 49m | 9.525 MB | |||
EAExample | |||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | ||
2.6 | 4.833 | 564 | 8s | 12s | 3m15s | 750 MB | |
2.5 | 4.833 | 564 | 10s | 17s | 3m37s | 747 MB | |
2.4.1 | 4.719 | 564 | 21s | 14s | 5m12s | 774 MB | |
2.4 | 4.719 | 564 | 22s | 16s | 4m59s | 826 MB | |
2.3 | 5.019 | 564 | 22s | 16s | 4m57s | 841 MB | |
2.2.0 | 5.018 | 564 | 19s | 14s | 4m53s | 805 MB | |
2.0.0 | 23s | 10s | 7m13s | 764 MB | |||
2.1.4 | 5.102 | 564 | 13s | 32s | 4m44s | 789 MB | |
2.1.3 | 5.102 | 564 | 12s | 36s | 4m30s | 782 MB | |
2.1.0 | 5.666 | 564 | 13s | 28s | 4m57s | 738 MB | |
Tagged Value Model | |||||||
Impacted elements | Impacted diagrams | 1) Loading models | 2) Premerge calculation | 3) Writing merge file | Max RAM usage | ||
2.6 | 5.963 | 263 | 1m18s | 24s | 3m18s | 4.256 MB | |
2.5 | 5.964 | 288 | 1m24s | 32s | 3m39s | 3.839 MB | |
2.4.1 | 5.963 | 263 | 2m22s | 29s | 3m53s | 5.091 MB | |
2.4 | 5.963 | 263 | 2m17 | 25s | 4m12s | 4.266 MB | |
2.3 | 5.961 | 263 | 1m47s | 33s | 4m10s | 4.073 MB | |
2.2.0 | 5.961 | 263 | 1m46s | 31s | 3m57s | 4.541 MB | |
2.1.4 | 5.961 | 263 | 1m19s | 59s | 3m54s | 4.359 MB | |
2.1.3 | 5.960 | 263 | 1m18s | 1m25s | 3m57s | 4.428 MB | |
2.1 | 6.107 | 263 | 1m52s | 1m | 2m39s | 3.040 MB | |
2.0 | 6.113 | 263 | 1m52s | 1m | 5m54s | 4.219 MB |
1: The test environment got a disc encryption since the last performance tests. We assume that's why the large models take longer on writing/loading. Older versions of LemonTree now also need longer with that models on writing.
Measurement Methods
To understand the measurement methods we have used to compile this data or conduct measurements on your own model for a comparison, please refer to this page: