Nextflow Language (DSL2)

A working reference for the untyped DSL2 interfaces used by many existing nf-core pipelines: processes, channels, operators, workflows, and modules. Nextflow is a Groovy-based DSL; DSL2 is the default and only DSL (DSL1 is removed, so nextflow.enable.dsl=2 is unnecessary). Targets stable 26.04.6. Source: https://github.com/nextflow-io/nextflow/tree/v26.04.6/docs and https://docs.seqera.io/nextflow/ . Incomplete biological/tool examples below are illustrative; they require the named tools, references and missing module definitions.

Current syntax conventions: NXF_SYNTAX_PARSER=v2 is the default in 26.04 (opt-in in 25.x). Strict syntax does not imply static typing. Typed processes/workflows remain a preview and require nextflow.enable.types = true in each applicable script; do not mix their syntax with untyped examples.

• **channel.of(...)** (lowercase namespace) is canonical; Channel.of(...) still works but is discouraged.
• Explicit closure parameters ({ v -> v * 2 }) are preferred over the implicit it.
• Name process outputs with **emit:; scale resources with task.attempt**.
• **output {} + publish:** (full feature in 25.10) is the new declarative way to publish results; publishDir still works and remains dominant in nf-core — both are shown below.
• Avoid removed/deprecated idioms: for/while loops (use each/collect), import/custom class (use functions or lib/), process shell: (use script:), include … addParams().

Table of Contents

• Script structure
• Processes
• Process directives
• Channels
• Operators
• Workflows
• Modules
• Dynamic resources and error handling
• Groovy essentials and gotchas

Script structure

A Nextflow script (.nf) uses process/workflow definitions and the supported subset of Groovy; strict syntax does not allow arbitrary Groovy. Every script enables DSL2 implicitly (it is the default since 22.03). A run begins at the **unnamed workflow {}** block (the entry workflow).

#!/usr/bin/env nextflow

params.input = 'data/*.fastq'        // pipeline parameter with a default

process FASTQC { /* ... */ }          // a process definition

workflow {                            // entry point
    reads = channel.fromPath(params.input, checkIfExists: true)
    FASTQC(reads)
}

Run with nextflow run main.nf --input 'data/*.fastq'. Parameters declared as params.x are overridable on the CLI (--x), in nextflow.config, or in a -params-file.

Processes

A process defines a task: a (usually Bash) script executed in its own isolated work directory. Nextflow stages declared inputs in and declared outputs out, so processes never read/write each other's files directly — they communicate only through channels.

process ALIGN {
    tag    "$meta.id"                 // label shown in the log/trace
    label  'process_high'             // maps to resources in config
    // Supply a pinned environment containing BOTH bwa and samtools.
    publishDir "${params.outdir}/bam", mode: 'copy'

    input:
    tuple val(meta), path(reads)      // a sample: metadata map + file(s)
    tuple path(reference), path(index_files)  // shared FASTA + BWA sidecars as a value

    output:
    tuple val(meta), path("*.bam"), emit: bam
    path  "versions.yml",           emit: versions

    when:
    meta.run_alignment != false       // skip task if false

    script:
    def prefix = task.ext.prefix ?: meta.id
    def args   = task.ext.args  ?: ''   // extra flags injected from config
    """
    bwa mem $args -t $task.cpus "$reference" ${reads} | samtools sort -o "${prefix}.bam"

    cat <<-END_VERSIONS > versions.yml
    "${task.process}":
        bwa: \$(bwa 2>&1 | sed -n 's/Version: //p')
        samtools: \$(samtools --version | sed '1!d; s/samtools //')
    END_VERSIONS
    """

    stub:
    """
    touch ${meta.id}.bam
    touch versions.yml
    """
}

Inputs

Declared one per line under input:. Each input consumes one item from a channel.

Inputs are positional and matched to channels in call order: ALIGN(reads_ch, index_ch).

Outputs

Declared under output:; each becomes a channel. Use emit: to name outputs so callers can reference ALIGN.out.bam instead of positional ALIGN.out[0].

Script, shell, exec

• **script:** (default) — a multi-line string run as Bash. Nextflow variables interpolate with $var/${expr}; escape shell vars you do NOT want Nextflow to touch as \$var.
• **shell:** — like script but Nextflow vars use !{var}, leaving $ for the shell. Deprecated as of 25.04 — use script: with \$ to escape shell vars.
• **exec:** — native Groovy, no external process (for in-line computation).
• A process can run any interpreter via a shebang (e.g. #!/usr/bin/env python).

process PY {
    input: val x
    output: stdout
    script:
    """
    #!/usr/bin/env python
    print(${x} ** 2)
    """
}

Conditional execution

• when: — skip the task when the expression is false (prefer filtering channels upstream when possible).
• A script: can branch with normal Groovy if/else returning different command strings.
• stub: replaces that task’s script under -stub-run; a task without one runs its real script. Containers, staging and eval() outputs may still run, so this is not a no-execution or no-cost dry run. nf-core modules require stubs.

Process directives

Set inside a process (or globally via config). Most-used:

Access the resolved values at runtime via task.* (task.cpus, task.memory, task.attempt, task.process, task.ext.args).

Channels

Channels are the asynchronous queues connecting processes. Two kinds:

• Queue channel: an ordered, consumable stream of items. Produced by most factories/operators and by process outputs. DSL2 automatically fans the stream out to separate downstream consumers; each receives all items. A single process invocation consumes one item per queue input for each task.
• Value channel (singleton): holds one value that can be read an unlimited number of times. Created by channel.value(), by operators like collect/first, or implicitly from a single value. A process input bound to a value channel is reused for every task.

Channel factories

channel.of(1, 2, 3)                                  // emit given values (ranges expand: 1..23)
channel.fromList([1, 2, 3])                          // emit list items
channel.value('ref.fa')                              // singleton value channel
channel.fromPath('data/*.bam')                       // one item per matching file
channel.fromPath('data/**/*.fastq', checkIfExists: true) // also: type:'file', hidden:true
channel.fromFilePairs('data/*_{1,2}.fastq.gz')       // -> [id, [r1, r2]] for paired reads
channel.topic('versions')                            // stable since 25.04; preview in 24.04/24.10
channel.empty()                                      // emits nothing

channel.fromSRA(...) exists but is deprecated as of 26.04 — use the documented Entrez Direct route to discover accessions, then an explicit validated samplesheet and download workflow.

arity is a process path input/output option, not a fromPath option. checkIfExists checks matching paths, not sample pairing or biological metadata.

fromFilePairs is the idiomatic way to group paired-end reads; it yields [ sampleId, [read1, read2] ], which you typically map into the nf-core [ meta, [reads] ] shape.

Operators

Operators transform/combine channels. Chain with .; the dataflow graph is built from these connections. The table includes legacy operators still used by untyped nf-core code. In 26.04, groupTuple, flatten, concat, distinct, first/last, the channel split* operators, set, and dump are deprecated; use the core operator migration guide for new code (groupBy, flatMap with Path split* methods, mix, unique, direct assignment, view, as applicable). Do not replace order-dependent logic mechanically.

// Build the nf-core [meta, reads] shape from a samplesheet
reads_ch = channel
    .fromPath(params.input, checkIfExists: true)
    .flatMap { sheet -> sheet.splitCsv(header: true) }
    .map { row -> tuple([id: row.sample, single_end: row.fastq_2 ? false : true],
                        row.fastq_2 ? [file(row.fastq_1, checkIfExists: true), file(row.fastq_2, checkIfExists: true)] : [file(row.fastq_1, checkIfExists: true)]) }

// Legacy tuple workflow: group results, then join on an exact meta key
counts.groupTuple()
       .join(metadata, failOnDuplicate: true, failOnMismatch: true) // exact keyed matching
       .view()

Workflows

A workflow composes processes and other workflows. The unnamed workflow is the entry point. Named workflows are reusable (sub)workflows.

workflow RNASEQ {
    take:                       // named inputs (untyped here)
    reads
    index

    main:                       // pipeline logic
    FASTQC(reads)
    ALIGN(reads, index)
    QUANT(ALIGN.out.bam)

    emit:                       // named outputs
    bam    = ALIGN.out.bam
    counts = QUANT.out.counts
    versions = FASTQC.out.versions.mix(ALIGN.out.versions)
}

workflow {                      // entry: wire inputs and call the named workflow
    reads = channel.fromFilePairs(params.reads, checkIfExists: true)
                   .map { id, pair -> tuple([id: id, single_end: false], pair) }
    index = channel.value(tuple(file(params.fasta, checkIfExists: true),
                                files(params.bwa_index, checkIfExists: true)))
    RNASEQ(reads, index)
    RNASEQ.out.counts.view()
}

• Call a process/workflow like a function: ALIGN(reads, index). Outputs are on .out (use emit: names: ALIGN.out.bam).
• A process can only be called once per workflow; to reuse it, include it again under an alias.
• Pipe syntax works for simple chains: reads | FASTQC.
• Declarative outputs (25.10+): assign channels in the entry workflow's publish: section and describe them in a top-level output {} block — the recommended replacement for the publishDir directive (which still works and dominates nf-core):

workflow {
    main:
    ch = ANALYZE(input)
    publish:
    results = ch                       // name the published channel
}
output {
    results { path 'analysis' }        // -> <outputDir>/analysis (default outputDir: results/)
}

Modules

Modules are .nf files whose processes/workflows are imported with include. This is the basis of nf-core's reusable components.

include { FASTQC }                     from './modules/fastqc/main.nf'
include { ALIGN as ALIGN_TUMOR;
          ALIGN as ALIGN_NORMAL }      from './modules/align/main.nf'
include { RNASEQ }                     from './subworkflows/rnaseq.nf'

• as aliases let you include the same component multiple times.
• Includes are resolved relative to the including file; .nf extension optional.
• Params should be passed explicitly (as inputs), not read globally inside modules — this keeps modules portable (an nf-core requirement).
• Since 26.04, registry modules can be installed with nextflow module install -version <version> <name> and included by registry name. This is distinct from nf-core modules install and its modules.json bookkeeping. Pin the registry version/checksum; see module registry. Local nf-core includes remain supported.

Dynamic resources and error handling

Make pipelines robust by retrying failures with more resources instead of over-provisioning everything. task.attempt increments on each retry.

process BIG_JOB {
    label 'process_high'
    cpus   { 4 * task.attempt }
    memory { 8.GB * task.attempt }
    time   { 4.h * task.attempt }

    errorStrategy { task.exitStatus in [137, 140, 143] ? 'retry' : 'terminate' }
    maxRetries 3
    // ...
}

• Exit codes 137/140/143 identify signals, not a definitive OOM diagnosis. Check scheduler/task logs before escalating resources; a cancellation, timeout or application failure can look similar. Bound retries and resource requests.
• errorStrategy 'ignore' lets the pipeline continue past a failed task; 'finish' stops launching new tasks but lets running ones complete.
• In nf-core, resource scaling lives in conf/base.config keyed on process_* labels (see references/developing.md).

Groovy essentials and gotchas

• Strings: single-quoted are literal; double-quoted interpolate ("${x}"). In script: blocks, escape shell variables as \$VAR.
• Define helper values with def inside script:/closures to avoid leaking globals.
• Maps use Groovy syntax: [ id: 'x', single_end: false ]; access as meta.id.
• Common gotchas:
• DSL2 allows a stream to feed multiple processes. The hazard is passing multiple independent queues into one process: completion order can mismatch samples. Join on an immutable sample key into one tuple channel; use channel.value(...) or toList() for a shared reference value.
• Legacy groupTuple without size buffers until the source closes. With size, it emits completed groups earlier and can drop incomplete groups unless remainder: true. Validate expected replicate/lane counts; never mistake a partial group for a complete sample.
• Legacy tuple join does not support duplicate keys. Set failOnDuplicate/failOnMismatch explicitly; for deliberate many-to-many matching use combine(by:) and validate cardinality. Do not assume parser mode alone supplies these checks.
• A full meta map used as a key must match exactly; extra/different fields can prevent joins. Keep sample keys immutable and check identities across branches.
• channel.topic consumers must not feed outputs back into the same topic: that cycle can hang the workflow.
• A process called twice without aliasing is an error; include ... as.
• Globs in output: match the work directory, not publishDir. Publishing is asynchronous; downstream processes must consume output channels. Prevent output filename collisions between samples.
• Prefer filtering channels over when: for clarity and caching.
• Strict syntax / language server: Nextflow provides a VS Code extension + nextflow lint and a stricter parser; nf-core is migrating pipelines to it. Keep scripts to documented DSL2 constructs and avoid deprecated DSL1 idioms (Channel.create(), .into{} overuse, top-level file() for inputs).