Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

We run tests with different releases of LemonTree against For every release of LemonTree, we run a set of tests with different test-models to examine the improvement of change of both the memory usage and performanceconsumption and the total diff/merge times.

This page gives an overview of the performance and memory usage and describes the test-models.

Performance and memory usage are continually improved in LemonTree.

Environment

...

shows the results of these tests. 

Info

As shown by the test results, our goal is to continuously both improve the performance and decrease the memory consumption of LemonTree.

Nevertheless, changes to the supported feature-set may impact both performance and memory consumption negatively. The Remarks section under the test results notes such changes.

Environment

Our test machine has the following setup:

  • Operating System: Windows 10 (64 Bit)
  • Processor: i7-6700HQ CPU @2.60 GHz

...

  • Memory: 16 GB

Test

...

Case Description

The test results provided on this page are based on a fixed set of test models, which we diff with every released version of LemonTree.

1: Small model, with few changes in 3-way comparison

These models are the example models which come with the LemonTree installation.

2: Medium model with several changes in 3-way comparison

A larger model which is diffed in a 3-way comparison. There are changes in A (respective to Base) and B (respective to Base). This test-case can be regarded as standard use-case for LemonTree, when used with models that are under version control (GIT, SVN).

3: Large model with a lot of changes in 2-way comparison

A really large model with a lot changes and even more elements to be checked by LemonTree. This is quite challenging for LemonTree. In a 2-way comparison Model A is also regarded as Base model.

4: EAExample, compared with empty model

When comparing an empty model with another model the diff results in a lot of new objects which leads to special challenges for LemonTree. In older versions of LemonTree (< 1.5) this was not successful within the test environment.

5: Tagged Value model with a lot of TaggedValues in 2-way comparison

TaggedValues needed a lot of RAM in older versions of LemonTree (<1.8). 

Testmodels

...

Test Name

...

Version

...

File size

...

Total packages

...

Total diagrams

...

Total elements

...

Total connectors

...

Total TaggedValues*

...

Description

...

TV Model

...

* Total Tagged Values are calculated via SQLs in EA:

select count(*) from t_attributetag UNION
select count(*) from t_connectortag UNION
select count(*) from t_operationtag UNION
select count(*) from t_taggedvalue UNION
select count(*) from t_objectproperties

Test results

...

Test name

...

LT Version

...

Impacted elements

...

Details on the used test models can be found here:

Test Case Description

Measurement Methods

To understand the measurement methods we have used to compile this data or conduct measurements on your own model for a comparison, please refer to this page:

Measurement Methods

Info

By applying our measurement methods to your own models, you can calculate comparison values to approximate how LemonTree will perform for your models based on the data below.

Test results

1.7.220s1s2s284 MB1.8.026s1s1s284 MB2.0.026622s1s1s

257 MB

2.1.02669s1s1s257
Small Model
LT Version1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usage
3.1.4 (latest release)4s1s1s796 MB
3.1.34s1s1s804 MB
3.1.24s1s1s596 MB
3.1.14s1s1s596 MB
3.0.18s1s1s864 MB
3.06s1s1s910 MB
2.6.106s1s1s236 MB
2.6.96s1s1s232 MB
2.6.66s1s2s256 MB
2.6.56s1s1s233 MB
2.6.46s2s1s247 MB
2.6.38s2s1s233 MB
2.6.26s1s1s254 MB
2.6.17s1s1s237 MB
2.64s1s1s242 MB
2.58s1s1s255 MB
2.4.119s1s1s463 MB
2.419s1s1s300 MB
2.320s1s1s297 MB
2.218s1s1s292 MB
2.1.413s1s1s263 MB
2.1.313s1s1s265 MB
2.1.09s1s1s

257 MB

2.022s1s1s

257 MB

Medium Model
LT Version1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usage
3.1.4 (latest release)1m16s8s14s6.852 MB
3.1.3 1m23s9s15s6.871 MB
3.1.21m25s10s16s6.349 MB
3.1.11m25s10s16s6.349 MB
3.0.11m26s9s14s6.830 MB
3.01m16s9s13s7.297 MB
2.6.102m11s16s18s5.523 MB
2.6.91m39s16s18s5.447 MB
2.6.61m46s15s19s5.495 MB
2.6.51m47s16s18s5.365 MB
2.6.41m45s14s18s5.400 MB
2.6.32m12s16s6s5,301 MB
2.6.21m48s14s5s5.165 MB
2.6.12m17s15s9s4.988 MB
2.61m51s18s7s5.436 MB
2.51m52s17s6s4.937 MB
2.4.12m10s19s15s5.940 MB
2.42m07s19s17s5.908 MB
2.32m17s13s14s5.928 MB
2.22m23s13s20s6.184 MB
2.1.41m57s32s12s5.595 MB
2.1.31m51s36s13s5.630 MB
2.11m45s23s21s5.980 MB
2.02m16s20s1m5.454 MB
Large Model

1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usage
Small Model1.5.5.022s0.5s3s250 MB
3.1.4 (latest release)2m37s4m44s21m22s11.935 MB
3.1.32m39s5m6s21m41s12.269 MB
3.1.22m22s5m26s25m6s11.383 MB
3.1.12m22s5m26s25m6s11.383 MB
3.0.11m48s4m29s23m3s11.969 MB
3.02m4m13s20m49s11.726 MB
2.6.102m15s8m5s23m30s10.788 MB
2.6.92m09s8m7s22m24s11.306 MB
2.6.62m21s8m15s24m10.971 MB
2.6.52m10s8m36s23m37s11.251 MB
2.6.42m16s7m55s23m34s11.408 MB
2.6.32m11s7m57s17m24s11.447 MB
2.6.22m14s7m41s17m35s11.135 MB
2.6.12m18s7m46s22m42s11.374 MB
2.62m35s9m41s24m02s11.605 MB
2.52m21s8m26s19m10s11.237 MB
2.4.12m30s10m4s27m57s11.750 MB
2.42m52s10m49s24m14s11.809 MB
2.33m13s8m13s29m14s10.548 MB
2.2.02m09s7m55s21m53s10.486 MB
2.1.42m22s17m58s22m38s10.011 MB
2.1.3
23*613s1s1s265 MBLT VersionImpacted elementsImpacted diagrams
2m21s18m2s23m26s9.888 MB
2.12m47s14m19s24m16s9.406 MB
2.02m30s15m49m9.525 MB
EAExample

1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usage
Medium model
3.1.4 (latest release)10s9s3m5s1.
5.5.02m15s4m42s6.500 MB1.7.22m18s40s40s7.200 MB1.8.02m20s28s33s6.869 MB2.0.01.1843862m16s20s1m5.454 MB2.1.01.1843861m45s23s21s5.980 MB2.1.31.028*3861m51s36s13s5.630 MBImpacted elementsImpacted diagrams
499 MB
3.1.310s9s3m9s1.467 MB
3.1.210s10s3m12s1.633 MB
3.1.110s10s3m12s1.633 MB
3.0.111s9s3m6s1.439 MB
3.010s10s3m3s1.739 MB
2.6.109s11s3m21s763 MB
2.6.98s13s3m19s757 MB
2.6.69s15s3m36s748 MB
2.6.59s14s3m19s758 MB
2.6.49s15s3m19s746 MB
2.6.310s13s3m20s726 MB
2.6.29s12s3m18s770 MB
2.6.110s13s3m14s775 MB
2.68s12s3m15s750 MB
2.510s17s3m37s747 MB
2.4.121s14s5m12s774 MB
2.422s16s4m59s826 MB
2.322s16s4m57s841 MB
2.2.019s14s4m53s805 MB
2.0.023s10s7m13s764 MB
2.1.413s32s4m44s789 MB
2.1.312s36s4m30s782 MB
2.1.013s28s4m57s738 MB
Tagged Value Model

1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usage
Large model1.5.5.02m30s37m68m9.100 MB1.7.22m45s25m37m10.100 MB1.8.02m30s20m39m9.550 MB2.0.075.5286.9792m30s15m49m9
3.1.4 (latest release)1m22s16s3m47s6.982 MB
3.1.31m27s23s3m47s6.968 MB
3.1.21m13s25s4m2s5.399 MB
3.1.11m13s25s4m2s5.399 MB
3.0.11m12s20s4m53s5.668 MB
3.01m10s17s4m18s5.622 MB
2.6.101m22s29s5m25s3.525 MB
2.
1.075.5286.9792m47s14m19s24m16s9.406
6.91m18s26s5m20s4.300 MB
2.6.61m24s30s5m49s4.230 MB
2.
1.368.544*6.9792m21s18m2s23m26s9.888 MBImpacted elementsImpacted diagrams1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usageEAExample1.5.5.025s25s15m40s770 MB1.7.226s60s6m980 MB1.8.021s28s7m20s

590 MB

2.0.05.66656423s10s7m13s764 MB2.1.05.66656413s28s4m57s738 Impacted diagrams
6.51m24s26s4m55s4.567 MB
2.6.41m28s27s5m46s3.552 MB
2.6.31m19s50s4m15s4.558 MB
2.6.21m26s27s3m18s4.313 MB
2.6.11m30s26s3m16s4.257 MB
2.61m18s24s3m18s4.256 MB
2.51m24s32s3m39s3.839 MB
2.4.12m22s29s3m53s5.091 MB
2.42m1725s4m12s4.266 MB
2.31m47s33s4m10s4.073 MB
2.2.01m46s31s3m57s4.541 MB
2.1.41m19s59s3m54s4.359 MB
2.1.3
5.102*56412s36s4m30s782 MBImpacted elements
1m18s1m25s3m57s4.428 MB
2.11m52s1m2m39s3.040 MB
2.01m52s1m5m54s4.219 MB
TV Model (Triple Size)

1) Loading models2) Premerge calculation3) Writing merge fileMax RAM usage
TV Model
1.5.52m1m10s13m10s5.500 MB1.7.23m10s11m11s6m10s6.708 MB1.8.01m391m18s6m7s3.933 MB2.0.06.1132631m52s1m5m54s4.219 MB2.1.06.1132631m52s1m2m39s3.040 MB2.1.35.960*2631m18s1m25s3m57s4.428 MB

...

3.1.4 (latest release)4m22s45s14m34s12.079 MB
3.1.34m46s53s15m16s11.780 MB
3.1.23m33s57s17m46s10.733 MB
3.1.13m33s57s17m46s10.733 MB
3.0.13m26s48s16m12s10.962 MB
3.03m20s48s15m22s9.157 MB
2.6.103m33s1m14s20m48s9.379 MB
2.6.93m36s1m24s19m59s10.471 MB
2.6.63m30s1m27s17m41s10.355 MB
2.6.53m26s1m24s16m59s10.592 MB
2.6.43m31s1m20s17m39s9.662 MB
2.6.33m31s2m28s13m54s10.839 MB
2.6.23m56s1m15s10m14s9.156 MB
2.6.13m38s1m18s10m13s9.701 MB
2.63m29s1m29s10m29s9.165 MB
2.53m44s1m46s10m57s8.875 MB
2.4.16m35s1m41s13m20s9.040 MB
2.46m46s1m32s14m41s9.420 MB
2.34m16s1m29s12m23s8.589 MB
2.2.04m40s1m18s11m56s8.496 MB
2.1.44m45s2m58s11m27s8.685 MB
2.1.34m25s4m23s12m58s9.290 MB
2.17m19s3m21s13m9s8.000 MB

Remarks

  • The test environment got a disc encryption before the performance tests of version 2.1.3. We assume that's why the large models take longer on writing/loading. Older versions of LemonTree now also need longer with that models on writing.
  • LemonTree 2.6 Version 2.6.0 has increased the performance of diffing and merging, but consumes more memory than version 2.5.0. This had a quite severe impact on the performance results
    for our "Large Model" performance test, as it was approaching the cap of available memory for the test machine, which lead to page swapping. 
  • LemonTree 2.6.1 processes all previously unhandled custom properties of elements
  • LemonTree 2.6.4 has resolved issues affecting the memory consumption and the pre-merge times introduced in version 2.6.3, reducing both of those values back to values comparable to version 2.6.2.
    However, this version also (re-)introduces the merging of ZOrder values in diagrams, which significantly impacts the merge time for diffs with changes to a significant number of large diagrams. 
  • LemonTree 3.1.2 is a hotfix for obfuscation issues. As such, no actual performance test was conducted, and performance is assumed to be identical to 3.1.1