feat: fix ontology tree ui performance and enabled large disease ontologies filter on catalogue

[mswertz]

closes https://github.com/molgenis/GCC/issues/818
also in parts addresses https://github.com/molgenis/GCC/issues/819 (duplicate PR)

how to test:

• go to http://localhost:3000/catalogue/ssr-catalogue/testNetwork1/collections?page=1 and open 'Disease' and chose 'VII'
• see that it loads relatively quickly as compared to previous version of the PR
• look at other ontologies, they are more snappy
• also look at https://preview-emx2-pr-4539.dev.molgenis.org/apps/tailwind-components/#/input/Tree.story

N.B. with #4584 we should be able to use the emitSelectedChildren=false. Should probably enable it still in this PR, but added the feature to make it possible so I can continue in the other PR

todo:

• added population.mainMedicalCondition filter on resources list view
• addressed performance issue with showing large ontologies by changing Tree component
• moved search from Ontology to Tree to also address performance issue (and drop the modal, I don't like it)
• changed ontology options search from modal to inline

N.B. this is confusing, there can also be populationDisease. We would need to produce filter on both with an 'or'? Or shall we reduce the data model so there is only one place where this is defined? I propose on the subpopulations.

[mswertz]

okay, this takes too long, right?

[mswertz]

This one blocks on reducing diseases list? Otherwise I propose we don't do this now and first create ability to deal with very large ontologies

[hslh]

If the only way to get the behaviour requested by Kim is to remove items from the ontology, I don't think we should do it. Then it's indeed better to figure out a more general way to deal with large ontologies.

[EleanorHyde-UMCG]

In de directory onder ‘Diagnosis available’ wordt ICD-10 in alle lagen gelijk getoond, ook nog ORPHA gelijk (aparte tabblad maar alles wordt tegelijkertijd geladen) --> waarom is het sneller bij de directory? @mswertz

[connoratrug]

this takes long due to the large tree size, this can be divided into 3 separate issues:

1. the whole tree added to the dom ( can change this to lazy)
2. the tree builder uses full scan to find tree elements ( can build map once and use it as loopup list)
3. the tree is reactive ( primarily for search, but also because the whole app is reactive
4. heavy data fetch ( fetch all tree nodes)

(5. unbalanced tree ( i a single child list contains a large percentage of the tree items, lazy does not help once this list is visible ) )

1 and 2 can be 'fixed' pretty easy ,
3. means breaking out the vue reactive system, doing some work , and then re-attach then is more work and breaks with the whole reactive app architecture.
4. requires changes to the filter api to fetch root nodes
6. requires a 'paged'/'load more ' lazy loading of a single child list ( this might impact ui/ux ( controles, in controles, in ...), and and quickly get cultured .

the best way imo is to avoid having the vue/js code doing the heavy stuff ( this means lazy tree , only add visible nodes to the dom , filter in backend )

[mswertz]

this takes long due to the large tree size, this can be divided into 3 separate issues:

1. the whole tree added to the dom ( can change this to lazy)
2. the tree builder uses full scan to find tree elements ( can build map once and use it as loopup list)
3. the tree is reactive ( primarily for search, but also because the whole app is reactive
4. heavy data fetch ( fetch all tree nodes)

(5. unbalanced tree ( i a single child list contains a large percentage of the tree items, lazy does not help once this list is visible ) )

1 and 2 can be 'fixed' pretty easy , 3. means breaking out the vue reactive system, doing some work , and then re-attach then is more work and breaks with the whole reactive app architecture. 4. requires changes to the filter api to fetch root nodes 6. requires a 'paged'/'load more ' lazy loading of a single child list ( this might impact ui/ux ( controles, in controles, in ...), and and quickly get cultured .

the best way imo is to avoid having the vue/js code doing the heavy stuff ( this means lazy tree , only add visible nodes to the dom , filter in backend )

Indeed, the (1) and (2) were most easy. The 'find' that was what slowed it down most dramatically. In addition using 'show' instead 'if' for the invisible nodes indeed also lost some performance.

Replacing it with implementation that is more like the original molgenis-components fixed that issue and I think makes it acceptable. (I think the molgenis-components implementation of the tree has another benefit which is for showing the 'partial' on the parent of the parent, but that refactoring can be done later).

P.S. If we want to make it feel really snappy we could load also the hidden filter items, then the render is fast (fetch is the remaining time now). Alternatively, indeed do a partial load of only the 'roots' and then load subtrees as suggested in (4). Would require some extra backend filter options (e.g. retrieve where parent==null) and ability to delegate subtree filtering to backend (now we simply dump all subitems into the filter).

[mswertz]

Search in the ontology is not fast enough still. I don't understand why we copy the whole tree instead of simply annotating it with a 'visible' just as with molgenis-components. That would prevent complete reloading but simply hide elements.

[mswertz]

Search in the ontology is not fast enough still. I don't understand why we copy the whole tree instead of simply annotating it with a 'visible' just as with molgenis-components. That would prevent complete reloading but simply hide elements.

also fixed now.

[connoratrug]

why an extra file when the tree itself is a node

[mswertz]

tree state should be managed once for the whole Tree instead of per TreeNode.

[connoratrug]

the state can be managed outside of the tree , the tree can render the state

[connoratrug]

• the tree selection ( deselection ) is incorrect
• tests are missing

[mswertz]

• the tree selection ( deselection ) is incorrect
• tests are missing

Can you be more specific? I believe the selection/deselection is correct. Can you give an example to reproduce?

There were no tests before, so I cannot be bothered to add them now on this change.

[mswertz]

Can you be more specific? I believe the selection/deselection is correct. Can you give an example to reproduce?

I think I found it, updating the Tree.story for easy testing.

[mswertz]

There were no tests before, so I cannot be bothered to add them now on this change.

I updated the Tree.story

I addressed most of the comments, just the test: would an e2e test be suitable place?

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud