We run tests with different releases of LemonTree against different test-models to examine the improvement of the memory usage and performance.
This page gives an overview of the performance and memory usage and describes the test-models.
Performance and memory usage are continually improved in LemonTree.
Environment
| Win10, 16GB RAM i7-6700HQ CPU @2.60 GHz, 64 BIT |
Test description
We have different test setups which run against the recent releases of LemonTree.
1: Small model, with few changes in 3-way comparison
These models are the example models which come with the LemonTree installation.
2: Medium model with several changes in 3-way comparison
A larger model which is diffed in a 3-way comparison. There are changes in A (respective to Base) and B (respective to Base). This test-case can be regarded as standard use-case for LemonTree, when used with models that are under version control (GIT, SVN).
3: Large model with a lot of changes in 2-way comparison
A really large model with a lot changes and even more elements to be checked by LemonTree. This is quite challenging for LemonTree. In a 2-way comparison Model A is also regarded as Base model.
4: EAExample, compared with empty model
When comparing an empty model with another model the diff results in a lot of new objects which leads to special challenges for LemonTree. In older versions of LemonTree (< 1.5) this was not successful within the test environment.
5: Tagged Value model with a lot of TaggedValues in 2-way comparison
TaggedValues needed a lot of RAM in older versions of LemonTree (<1.8).
Testmodels
Test Name | Version | File size | Total packages | Total diagrams | Total elements | Total connectors | Total TaggedValues* | Description |
|---|---|---|---|---|---|---|---|---|
| Small Model | Base | 3 MB | 6 | 17 | 202 | 152 | 242 | |
| A | 3 MB | 6 | 17 | 205 | 154 | 242 | ||
| B | 3 MB | 6 | 17 | 203 | 153 | 242 | ||
| Medium Model | Base | 128 MB | 758 | 2.038 | 21.922 | 26.075 | 88.326 | |
| A | 128 MB | 739 | 1.988 | 21.685 | 25.594 | 88.810 | ||
| B | 133 MB | 739 | 1.987 | 21.670 | 25.568 | 88.331 | ||
| Large Model | Base | 233 MB | 8.490 | 6.733 | 114.962 | 69.652 | 32.788 | same as A |
| A | 233 MB | 8.490 | 6.733 | 114.962 | 69.652 | 32.788 | same as Base | |
| B | 195 MB | 8.059 | 6.252 | 107.111 | 64.345 | 29.157 | ||
| EaExample | Base | 1,4 MB | 1 | 0 | 0 | 0 | 0 | empty model |
| A | 1,4 MB | 1 | 0 | 0 | 0 | 0 | empty model | |
| B | 20 MB | 427 | 564 | 4.656 | 2.623 | 29.460 | ||
TV Model | Base | 142 MB | 338 | 909 | 46.380 | 22.819 | 316.458 | same as A |
| A | 142 MB | 338 | 909 | 46.380 | 22.819 | 316.458 | same as Base | |
| B | 152 MB | 347 | 982 | 50.810 | 27.289 | 336.759 |
Test results
Test name | LT Version | 1) Loading models | 2) Diff & Merge calculation | 3) Writing merge file | Max RAM usage |
|---|---|---|---|---|---|
| Small Model | 1.5.5 | 22s | 0.5s | 3s | 250 MB |
| 1.7.2 | 20s | 1s | 2s | 284 MB | |
| 2.0 | 22s | 1s | 1s | 257 MB | |
| 2.1.4 | 13s | 1s | 1s | 263 MB | |
| 2.2 | 18s | 1s | 1s | 292 MB | |
| 2.3 | 20s | 1s | 1s | 297 MB | |
| 2.4.1 | 19s | 1s | 1s | 463 MB | |
| 2.5 | 8s | 1s | 1s | 255 MB | |
| LT Version | 1) Loading models | 2) Diff & Merge calculation | 3) Writing merge file | Max RAM usage | |
| Medium model | 1.5.5 | 2m | 15s | 4m42s | 6.500 MB |
| 1.7.2 | 2m18s | 40s | 40s | 7.200 MB | |
| 2.0 | 2m16s | 20s | 1m | 5.454 MB | |
| 2.1.4 | 1m57s | 32s | 12s | 5.595 MB | |
| 2.2 | 2m23s | 13s | 20s | 6.184 MB | |
| 2.3 | 2m17s | 13s | 14s | 5.928 MB | |
| 2.4.1 | 2m10s | 19s | 15s | 5.940 MB | |
| 2.5 | 1m52s | 17s | 6s | 4.937 MB | |
| 1) Loading models | 2) Diff & Merge calculation | 3) Writing merge file | Max RAM usage | ||
| Large model | 1.5.5 | 2m30s | 37m | 68m | 9.100 MB |
| 1.7.2 | 2m45s | 25m | 37m | 10.100 MB | |
| 2.0 | 2m30s | 15m | 49m | 9.525 MB | |
| 2.1.4 | 2m22s | 17m58s | 22m38s | 10.011 MB | |
| 2.2 | 2m09s | 7m55s | 21m53s | 10.486 MB | |
| 2.3 | 3m13s1 | 8m13s | 29m14s1 | 10.548 MB | |
| 2.4.1 | 2m30s | 10m4s | 27m57s | 11.750 MB | |
| 2.5 | 2m21s | 8m26s | 19m10s | 11.237 MB | |
| 1) Loading models | 2) Diff & Merge calculation | 3) Writing merge file | Max RAM usage | ||
| EAExample | 1.5.5 | 25s | 25s | 15m40s | 770 MB |
| 1.7.2 | 26s | 60s | 6m | 980 MB | |
| 2.0 | 23s | 10s | 7m13s | 764 MB | |
| 2.1.4 | 13s | 32s | 4m44s | 789 MB | |
| 2.2 | 19s | 14s | 4m53s | 805 MB | |
| 2.3 | 22s | 16s | 4m57s | 841 MB | |
| 2.4.1 | 21s | 14s | 5m12s | 774 MB | |
| 2.5 | 10s | 17s | 3m37s | 747 MB | |
| 1) Loading models | 2) Diff & Merge calculation | 3) Writing merge file | Max RAM usage | ||
TV Model | 1.5.5 | 2m | 1m10s | 13m10s | 5.500 MB |
| 1.7.2 | 3m10s | 11m11s | 6m10s | 6.708 MB | |
| 2.0 | 1m52s | 1m | 5m54s | 4.219 MB | |
| 2.1.4 | 1m19s | 59s | 3m54s | 4.359 MB | |
| 2.2 | 1m46s | 31s | 3m57s | 4.541 MB | |
| 2.3 | 1m47s | 33s | 4m10s | 4.073 MB | |
| 2.4.1 | 2m22s | 29s | 3m53s | 5.091 MB | |
| 2.5 | 1m24s | 32s | 3m39s | 3.839 MB |
1: The test environment got a disc encryption since the last performance tests. We assume that's why the large models take longer on writing/loading. Older versions of LemonTree now also need longer with that models on writing.
Measurement Methods
To understand the measurement methods we have used to compile this data or conduct measurements on your own model for a comparison, please refer to this page: