-
Notifications
You must be signed in to change notification settings - Fork 403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory resource constraints are not passed to IQ-TREE (via augur tree
)
#949
Comments
@AdrianLimaG Snakemake's resource settings help Snakemake's scheduler decide what rules can run simultaneously, but they don't automatically enforce limits on what the rule actually uses. It's left to the rule in question to pass through resource constraints to the commands it's running and for the commands to respect those. In this case, I don't think the rule running IQ-TREE is doing that, so IQ-TREE doesn't get the memo and defaults to using all available RAM (which seems insufficient for the size of the tree/alignment). |
augur tree
)
While I originally didn't think there was a way to ask IQ-TREE to limit its memory usage, looking again just now to confirm I see this in
So this is something we could update our Snakemake |
While most of our mem_mb definitions are only heuristics for Snakemake's scheduler and the commands themselves aren't limited or aware of the mem_mb defined, IQ-TREE *does* support memory limits. Resolves <#949>.
Possible fix in #953. |
Current Behavior
When runing nextstrain build with the following command
nextstrain build . --configfile build.yaml --cores all --resources mem_mb=5000
Everything works up until the iqtree2 command is run. Here my OS kills the operation due to its memory limitations.
My current set up has 15gb of ram and when I check the "dmesg" I see the iqtree was using around 19gb of virtual memory which is why the OS killed it.
I am just confused as to why iqtrees is trying to use this much memory when i defined in the resource for snakemake to use only 5gb. It seems as if snakemake is not passing this down.
Have i defined the resources properly? Are there any other steps i can take to prevent this from happening?
The text was updated successfully, but these errors were encountered: