mhcquant fails possibly due to ms2pip #336
Comments
|
The folder is both read and writable |
|
Apologies for the late response here. Additional question: |
|
The files are both read and write. I enabled the read/write all. The error could also be due to the high resources needed by https://github.com/nf-core/mhcquant/blob/master/conf/base.config Is there a way to provide a config file which ask for lesser resources? |
|
Yes you can overwrite this by providing a custom config file along nextflows For Comet there is also an option to split the database search into batches using |
|
I tried it, however, I still see the same issue I do not think it is resource related issue. |
|
The memory usage displayed in the log here Is not correct. Can you try playing around with memory? Comet definitely needs more than 2 GB. I would give it as much as you can. Also consider using additional resource flags. I am not sure if the custom config actually overwrites the ressourcen limits of nextflow.config. |
|
|
|
Very strange. I am also confused why the test run crashes at Comet and your run did run through CometAdapter 🤔 |
|
I just had to run the command again with |
|
Can you upload the full |
|
The nextflow.log you posted stopped at cometadapter again 🤔 . Can you do a non-resume run test run again and maybe give more cpus, maybe max(cpus of your instance) -1 via the |
|
I added a custom config file. It fails at Comet but different error this time. |
|
Can you try running the test profile using the current dev branch by using '-r dev'? Nextflow changed the way how to limit resources https://www.nextflow.io/docs/latest/reference/process.html#resourcelimits You probably need to update your nextflow version as well No custom config should be needed for this in the testrun |
|
I checked out the dev branch. |
|
Another thing I noticed that |
|
Did you progress on this issue? I saw you asked in nf-core Slack. Else I would suggest you ask during the nf-core helpdesk time-slots. If that doesn't help either feel free to drop me a DM and we can do a VC to fix it. Its quite hard to debug this from a distance. |
|
Hi, YES. The issue seems to be cluster-specific. I was able to run it on different cluster. |
|
Good to hear, just ping me or nf-core Slack again if you are in trouble again. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description of the bug
Hi,
I am trying to run mhcquant on a SLURM computing cluster. Upon executing the
nextflow run nf-core/mhcquant -profile singularityI see the error, I have created/home/shubham/.ms2pipeven then I am not able to run it successfully. What could be the issue?Command used and terminal output
(base) [shubham@cfx22885s-srcf-d11-17-west hlaPipeline]$ nextflow run nf-core/mhcquant -profile singularity --input 'samplesheet.tsv' --outdir results --fasta protein.fasta --ms2pip_model timsTOF2023 --ms2pip_model_dir /home/shubham/.ms2pip -resume
N E X T F L O W ~ version 24.04.4
Launching
https://github.com/nf-core/mhcquant[determined_sax] DSL2 - revision: 1b30692 [master]WARN: Nextflow self-contained distribution allows only core plugins -- User config plugins will be ignored: [email protected]
WARN: Access to undefined parameter
monochromeLogs-- Initialise it to a default value eg.params.monochromeLogs = some_value|\ | |__ __ /
/ \ |__) |__ } { | \| | \__, \__/ | \ |___ \-.,--,.,._,'nf-core/mhcquant v2.6.0-g1b30692
Core Nextflow options
revision : master
runName : determined_sax
containerEngine : singularity
launchDir : /oak/shubham/hlaPipeline
workDir : /oak/shubham/hlaPipeline/work
projectDir : /home/shubham/.nextflow/assets/nf-core/mhcquant
userName : shubham
profile : singularity
configFiles :
Input/output options
input : samplesheet.tsv
outdir : results
Database Options
fasta : protein.fasta
Database Search Settings
digest_mass_range : 800:2500
prec_charge : 2:3
fixed_mods :
default_params_file_comet:
Rescoring settings
ms2pip_model : timsTOF2023
ms2pip_model_dir : /home/shubham/.ms2pip
!! Only displaying parameters that differ from the pipeline defaults !!
If you use nf-core/mhcquant for your analysis please cite:
The pipeline
https://doi.org/10.1021/acs.jproteome.9b00313
The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
Software dependencies
https://github.com/nf-core/mhcquant/blob/master/CITATIONS.md
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
[ad/96526a] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:TDF2MZML (2) [100%] 2 of 2, cached: 2 ✔
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
[ad/96526a] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:TDF2MZML (2) [100%] 2 of 2, cached: 2 ✔
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:GUNZIP -
[dc/131eb0] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_DECOYDATABASE (protein) [100%] 1 of 1, cached: 1 ✔
[d5/48696c] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_COMETADAPTER (2) [100%] 2 of 2, cached: 2 ✔
[64/bfa4c8] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_PEPTIDEINDEXER (1) [100%] 2 of 2, cached: 2 ✔
[22/9de26a] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_IDMERGER (tumor_treated) [100%] 1 of 1, cached: 1 ✔
[68/26238c] process > NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE (tumor_treated) [100%] 1 of 1, failed: 1 ✘
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_PSMFEATUREEXTRACTOR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_PERCOLATORADAPTER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_IDFILTER_Q_VALUE -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_TEXTEXPORTER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_MZTABEXPORTER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:MULTIQC [ 0%] 0 of 1
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/mhcquant] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE (tumor_treated)'
Caused by:
Process
NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE (tumor_treated)terminated with an error exit status (1)Command executed:
ms2rescore_cli.py
--psm_file tumor_treated.idXML
--spectrum_path .
--output_path tumor_treated_ms2rescore.idXML
--processes 12
--ms2_tolerance 0.02 --ms2pip_model timsTOF2023 --ms2pip_model_dir /home/shubham/.ms2pip --rescoring_engine percolator --feature_generators deeplc,ms2pip
cat <<-END_VERSIONS > versions.yml$(echo $ (ms2rescore --version 2>&1) | grep -oP 'MS²Rescore (v\K[^\)]+' ))
"NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE":
MS²Rescore:
END_VERSIONS
Command exit status:
1
Command output:
Warning: OPENMS_DATA_PATH environment variable not found and no share directory was installed. Some functionality might not work as expected.
Command error:
/scg/apps/modules/lmod/lmod/init/bash: line 16: _lmod_vx: unbound variable
INFO: Converting SIF file to temporary sandbox...
Matplotlib created a temporary cache directory at /tmp/matplotlib-4afs_8gr because the default path (/home/shubham/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Fontconfig error: No writable cache directories
2024-10-19 11:18:31,311 INFO Percolator rescoring engine has been specified. Use the idXML containing rescoring features and run Percolator in a separate step.
2024-10-19 11:18:31,311 INFO MS²Rescore config:
2024-10-19 11:18:31,311 INFO {'$schema': './config_schema.json', 'ms2rescore': {'feature_generators': {'ms2pip': {'model': 'timsTOF2023', 'ms2_tolerance': 0.02, 'model_dir': '/home/shubham/.ms2pip'}, 'deeplc': {'deeplc_retrain': False, 'calibration_set_size': 0.15}}, 'rescoring_engine': {}, 'config_file': None, 'psm_file': 'tumor_treated.idXML', 'psm_file_type': 'infer', 'psm_reader_kwargs': {}, 'spectrum_path': '.', 'output_path': 'tumor_treated_ms2rescore.idXML', 'log_level': 'info', 'id_decoy_pattern': '^DECOY', 'psm_id_pattern': None, 'spectrum_id_pattern': None, 'lower_score_is_better': True, 'modification_mapping': {}, 'fixed_modifications': {}, 'processes': 12, 'rename_to_usi': False, 'fasta_file': None, 'write_report': True, 'ms2pip_model_dir': '/home/shubham/.ms2pip'}}
2024-10-19 11:18:50,520 INFO Found 90370 PSMs, of which 48.59% are decoys.
2024-10-19 11:18:51,736 WARNING Non-mapped modifications found: {'Oxidation'}
This can be ignored if Unimod modification label
2024-10-19 11:18:52,292 INFO Found 625 identified PSMs at 1% FDR before rescoring.
2024-10-19 11:18:52,407 INFO Adding MS²PIP-derived features to PSMs.
2024-10-19 11:18:52,552 INFO Running MS²PIP for PSMs from run (1/2)
20240520_HLAI_067_DDA_30min_200ms_7PASEF_Slot1-13_1_1813...Warning: OPENMS_DATA_PATH environment variable not found and no share directory was installed. Some functionality might not work as expected.
Traceback (most recent call last):
File "/home/shubham/.nextflow/assets/nf-core/mhcquant/bin/ms2rescore_cli.py", line 177, in
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/shubham/.nextflow/assets/nf-core/mhcquant/bin/ms2rescore_cli.py", line 173, in main
rescore_idxml(kwargs["psm_file"], kwargs["output_path"], config)
File "/home/shubham/.nextflow/assets/nf-core/mhcquant/bin/ms2rescore_cli.py", line 82, in rescore_idxml
rescore(config, psm_list)
File "/usr/local/lib/python3.10/site-packages/ms2rescore/core.py", line 80, in rescore
fgen.add_features(psm_list)
File "/usr/local/lib/python3.10/site-packages/ms2rescore/feature_generators/ms2pip.py", line 190, in add_features
ms2pip_results = correlate(
File "/usr/local/lib/python3.10/site-packages/ms2pip/core.py", line 178, in correlate
ms2pip_parallelized = _Parallelized(
File "/usr/local/lib/python3.10/site-packages/ms2pip/core.py", line 383, in init
validate_requested_xgb_model(
File "/usr/local/lib/python3.10/site-packages/ms2pip/_utils/xgb_models.py", line 21, in validate_requested_xgb_model
_download_model(model_file, xgboost_model_hashes[model_file], model_dir)
File "/usr/local/lib/python3.10/site-packages/ms2pip/_utils/xgb_models.py", line 94, in _download_model
os.mkdir(model_dir)
OSError: [Errno 30] Read-only file system: '/home/shubham/.ms2pip'
INFO: Cleaning up image...
Work dir:
/oak/shubham/hlaPipeline/work/68/26238c11b1b4af983e2e710adc8772
Tip: you can replicate the issue by changing to the process work dir and entering the command
bash .command.run-- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting
-- Check '.nextflow.log' file for details
Relevant files
No response
System information
nextflow/24.04.4
Executor Slurm
Engine Singularity
The text was updated successfully, but these errors were encountered: