Use eval to collect version

[ewels]

Nextflow 24.02.0-edge brings with it two new features which should give much cleaner syntax for collecting tool versions: channel topics using topic (see #4517) and eval. They are designed to work together nicely, but are technically separate features.

• Docs for eval: https://nextflow.io/docs/latest/process.html#output-type-eval
• Blog post about eval and topic: https://nextflow.io/blog/2024/nextflow-2404-highlights.html#process-eval-outputs
• Example pipeline change: Proposal: DSL2+ fetchngs#312

The blog post contains a basic example of how this can be used to collect versions:

process FASTQC {
  input:
  tuple val(meta), path(reads)

  output:
  tuple val(meta), path('*.html'), emit: html
  tuple val("${task.process}"), val('fastqc'), eval('fastqc --version'), topic: versions 

  """
  fastqc $reads
  """
}

workflow {
  Channel.topic('versions') 
    | unique()
    | collectFile(name: 'collated_versions.yml')
    | CUSTOM_DUMPSOFTWAREVERSIONS
}

Needs some thought about how we approach these changes - may be best to do a bulk change everywhere in one go, as it affects multiple code locations (modules and pipelines).

[muffato]

Nextflow 22.04 brings with it

Did you mean 24.04 ?

[ewels]

See comment: #4517 (comment)

[SPPearce]

Does this help fix #6696 ?

[asp8200]

An example that might need to be taken into consideration is that, in the nf-module for bwa, the command used to retrieve the version information is bwa 2>&1 and that command always returns exit code 1, and so I guess eval('bwa 2>&1') will always fail.

If the result of the version-retrieving command is pipped into sed like here, and the eval-statement over in the fastqc-module in the nf-core/repo would be something like eval('fastqc --version | sed \'/FastQC v/!d; s/.*v//'\')(Not sure about the syntax for escaping the single quotation mark (') but nevermind.), then Nextflow might not detect if the command fastqc --version | sed \'/FastQC v/!d; s/.*v//') has failed, because <some_failing_cmd> | sed '/FastQC v/!d; s/.*v//' seems to always return exit code 0.

[SPPearce]

Can we set pipefail globally? Is it already done?