Tomcat Memory Usage Exceeds Configured Limits on VIVO 1.14

[jesielviana]

I am experiencing an issue with memory management on a VIVO 1.14 installation running on Debian 11. Despite configuring CATALINA_OPTS according to the installation manual with the following settings:

export CATALINA_OPTS="-Xms512m -Xmx512m -XX:MaxPermSize=128m"

The Tomcat process is consuming significantly more memory than configured. Monitoring tools report the following usage:

• VIRT: 97 GB
• RES: 41 GB
• SHR: 34 GB

This behavior suggests that the memory usage is not respecting the -Xmx setting defined in CATALINA_OPTS.

Additional Details:

OS: Debian 11
VIVO Version: 1.14
Java Version: OpenJDK 11
Tomcat Version: 9

Questions/Help Needed:

Are there additional configurations required for VIVO that override CATALINA_OPTS?
Is this a known issue, and are there recommended workarounds or fixes?
Could the excessive memory usage be caused by specific VIVO components or the application itself?

[chenejac]

@jesielviana thanks for reporting this issue.

Can you provide more details which monitoring tools you used? It looks those numbers of virtual, resident and shared memory size are quite big. Maybe to try to check memory usage of tomcat through its manager application?

[jesielviana]

Most of the memory is being used to load .dat files, for example: vivo/home/tdbContentModels/OSPG.dat
Print of pmap command:

There are hundreds of lines like these.

[brianjlowe]

Hi @jesielviana, these files are used by the the Jena TDB triple store. By default (on 64-bit machines) TDB uses memory mapped files; because this memory is separate from the Java heap space, the -Xmx parameter will not affect its size. TDB does have a mode that does its file caching inside the heap but VIVO would need to be modified slightly to set the tdb.file_mode parameter accordingly. There are also related parameters for setting the size of the cache. See the "File access - 'mapped' and 'direct' modes" and related info here: https://jena.apache.org/documentation/tdb/store-parameters.html .

The Jena page mentions that the default mapped mode "flex[es] the amount of memory for file system cache to balance demands from other programmes on the same hardware." Are you finding otherwise? Is the Tomcat process hogging memory needed for other processes? If not, letting the available memory be used for TDB's cache should lead to the best performance of the triple store.

[jesielviana]

Got it, in that case I'll change the tdb.file_mode parameter to “direct”, I think that's the best setting for our context.
Thank you @brianjlowe.