Skip to content

Workflow onComplete and onError sections #6275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,9 +321,15 @@ process {
This limitation can be avoided by using the {ref}`strict config syntax <updating-config-syntax>`.
:::

(config-workflow-handlers)=

## Workflow handlers

Workflow event handlers can be defined in the config file, which is useful for handling pipeline events without having to modify the pipeline code:
:::{deprecated} 25.10.0
Use a {ref}`trace observer <plugins-trace-observers>` in a plugin to add custom workflow handlers to a pipeline via configuration.
:::

You can define workflow event handlers in the config file:

```groovy
workflow.onComplete = {
Expand All @@ -337,4 +343,4 @@ workflow.onError = {
}
```

See {ref}`workflow-handlers` for more information.
This approach is useful for handling workflow events without modifying the pipeline code. See {ref}`workflow-handlers` for more information.
2 changes: 2 additions & 0 deletions docs/developer/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,8 @@ class MyExecutor extends Executor {
}
```

(plugins-trace-observers)=

### Trace observers

:::{versionchanged} 25.04
Expand Down
25 changes: 25 additions & 0 deletions docs/migrations/25-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,31 @@ This page summarizes the upcoming changes in Nextflow 25.10, which will be relea
This page is a work in progress and will be updated as features are finalized. It should not be considered complete until the 25.10 release.
:::

## Enhancements

<h3>New syntax for workflow handlers</h3>

The workflow `onComplete` and `onError` handlers were previously defined by calling `workflow.onComplete` and `workflow.onError` in the pipeline script. You can now define handlers as `onComplete` and `onError` sections in an entry workflow:

```nextflow
workflow {
main:
// ...

onComplete:
println "workflow complete"

onError:
println "error: ${workflow.errorMessage}"
}
```

This syntax is simpler and easier to use with the {ref}`strict syntax <strict-syntax-page>`. See {ref}`workflow-handlers` for details.

## Breaking changes

- The AWS Java SDK used by Nextflow was upgraded from v1 to v2, which introduced some breaking changes to the `aws.client` config options. See {ref}`the guide <aws-java-sdk-v2-page>` for details.

## Deprecations

- The use of workflow handlers in the configuration file has been deprecated. You should define workflow handlers in the pipeline script or a plugin instead. See {ref}`config-workflow-handlers` for details.
31 changes: 31 additions & 0 deletions docs/notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,22 @@ workflow.onComplete {
}
```

:::{versionadded} 25.10.0
:::

Entry workflows can define an `onComplete` section instead of using `workflow.onComplete`:

```nextflow
workflow {
main:
// ...

onComplete:
println "Pipeline completed at: $workflow.complete"
println "Execution status: ${ workflow.success ? 'OK' : 'failed' }"
}
```

(metadata-error-handler)=

### Error handler
Expand All @@ -39,6 +55,21 @@ workflow.onError {
Both the `onError` and `onComplete` handlers are invoked when an error condition is encountered. The first is called as soon as the error is raised, while the second is called just before the pipeline execution is about to terminate. When using the `finish` {ref}`process-error-strategy`, there may be a significant gap between the two, depending on the time required to complete any pending job.
:::

:::{versionadded} 25.10.0
:::

Entry workflows can define an `onError` section instead of using `workflow.onError`:

```nextflow
workflow {
main:
// ...

onError:
println "Error: Pipeline execution stopped with the following message: ${workflow.errorMessage}"
}
```

## Mail

The built-in function `sendMail` allows you to send a mail message from a workflow script.
Expand Down
22 changes: 14 additions & 8 deletions docs/reference/syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ Parameters supplied via command line options, params files, and config files tak

A workflow can be a *named workflow* or an *entry workflow*.

A *named workflow* consists of a name and a body, and may consist of a *take*, *main*, *emit*, and *publish* section:
A *named workflow* consists of a name and a body, and may consist of a *take*, *main*, and *emit* section:

```nextflow
workflow greet {
Expand All @@ -146,28 +146,34 @@ workflow greet {

- The emit section consists of one or more *emit statements*. An emit statement can be a [variable name](#variable), an [assignment](#assignment), or an [expression statement](#expression-statement). If an emit statement is an expression statement, it must be the only emit.

- The publish section can be specified but is intended to be used in the entry workflow (see below).


An *entry workflow* has no name and may consist of a *main* and *publish* section:
An *entry workflow* has no name and may consist of a *main*, *publish*, *onComplete*, and *onError* section:

```nextflow
workflow {
main:
greetings = channel.of('Bonjour', 'Ciao', 'Hello', 'Hola')
messages = greetings.map { v -> "$v world!" }
greetings.view { it -> '$it world!' }
greetings.view { v -> "$v world!" }

publish:
messages >> 'messages'
messages = messages

onComplete:
log.info 'Workflow completed successfully!'

onError:
log.error 'Workflow failed.'
}
```

- Only one entry workflow may be defined in a script.

- The `main:` section label can be omitted if the publish section is not specified.
- The `main:` section label can be omitted if the other sections are not specified.

- The publish section consists of one or more *publish statements*. A publish statement is an [assignment](#assignment), where the assignment target is the name of a workflow output.

- The publish section consists of one or more *publish statements*. A publish statement is a [right-shift expression](#binary-expressions), where the left-hand side is an expression that refers to a value in the workflow body, and the right-hand side is an expression that returns a string.
- The `onComplete` and `onError` sections consist of one or more [statements](#statements).

In order for a script to be executable, it must either define an entry workflow or be a code snippet as described [above](#script-declarations).

Expand Down
2 changes: 2 additions & 0 deletions modules/nf-lang/src/main/antlr/ScriptLexer.g4
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,8 @@ WHEN : 'when';
WORKFLOW : 'workflow';
EMIT : 'emit';
MAIN : 'main';
ONCOMPLETE : 'onComplete';
ONERROR : 'onError';
PUBLISH : 'publish';
TAKE : 'take';

Expand Down
28 changes: 15 additions & 13 deletions modules/nf-lang/src/main/antlr/ScriptParser.g4
Original file line number Diff line number Diff line change
Expand Up @@ -227,29 +227,27 @@ workflowDef

workflowBody
// explicit main block with optional take/emit blocks
: (sep TAKE COLON nls workflowTakes)?
sep MAIN COLON nls workflowMain
(sep EMIT COLON nls workflowEmits)?
(sep PUBLISH COLON nls workflowPublishers)?
: (sep TAKE COLON nls take=workflowTakes)?
sep MAIN COLON nls main=blockStatements
(sep EMIT COLON nls emit=workflowEmits)?
(sep PUBLISH COLON nls publish=workflowPublishers)?
(sep ONCOMPLETE COLON nls onComplete=blockStatements)?
(sep ONERROR COLON nls onError=blockStatements)?

// explicit emit block with optional take/main blocks
| (sep TAKE COLON nls workflowTakes)?
(sep MAIN COLON nls workflowMain)?
sep EMIT COLON nls workflowEmits
(sep PUBLISH COLON nls workflowPublishers)?
| (sep TAKE COLON nls take=workflowTakes)?
(sep MAIN COLON nls main=blockStatements)?
sep EMIT COLON nls emit=workflowEmits
(sep PUBLISH COLON nls publish=workflowPublishers)?

// implicit main block
| sep? workflowMain
| sep? main=blockStatements
;

workflowTakes
: identifier (sep identifier)*
;

workflowMain
: blockStatements
;

workflowEmits
: statement (sep statement)*
;
Expand Down Expand Up @@ -517,6 +515,8 @@ identifier
| WORKFLOW
| EMIT
| MAIN
| ONCOMPLETE
| ONERROR
| PUBLISH
| TAKE
;
Expand Down Expand Up @@ -709,6 +709,8 @@ keywords
| WORKFLOW
| EMIT
| MAIN
| ONCOMPLETE
| ONERROR
| PUBLISH
| TAKE
| NullLiteral
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ public void visitWorkflow(WorkflowNode node) {
visit(node.main);
visit(node.emits);
visit(node.publishers);
visit(node.onComplete);
visit(node.onError);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,21 @@ public class WorkflowNode extends MethodNode {
public final Statement main;
public final Statement emits;
public final Statement publishers;
public final Statement onComplete;
public final Statement onError;

public WorkflowNode(String name, Statement takes, Statement main, Statement emits, Statement publishers) {
public WorkflowNode(String name, Statement takes, Statement main, Statement emits, Statement publishers, Statement onComplete, Statement onError) {
super(name, 0, dummyReturnType(emits), dummyParams(takes), ClassNode.EMPTY_ARRAY, EmptyStatement.INSTANCE);
this.takes = takes;
this.main = main;
this.emits = emits;
this.publishers = publishers;
this.onComplete = onComplete;
this.onError = onError;
}

public WorkflowNode(String name, Statement main) {
this(name, EmptyStatement.INSTANCE, main, EmptyStatement.INSTANCE, EmptyStatement.INSTANCE, EmptyStatement.INSTANCE, EmptyStatement.INSTANCE);
}

public boolean isEntry() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ public void visitWorkflow(WorkflowNode node) {
resolver.visit(node.main);
resolver.visit(node.emits);
resolver.visit(node.publishers);
resolver.visit(node.onComplete);
resolver.visit(node.onError);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,14 +125,17 @@ public void visitParam(ParamNode node) {
@Override
public void visitWorkflow(WorkflowNode node) {
visitWorkflowTakes(node.takes);
visit(node.main);
visitWorkflowEmits(node.emits, node.main);
visitWorkflowPublishers(node.publishers, node.main);

var main = node.main instanceof BlockStatement block ? block : new BlockStatement();
visitWorkflowEmits(node.emits, main);
visitWorkflowPublishers(node.publishers, main);
visitWorkflowHandler(node.onComplete, "onComplete", main);
visitWorkflowHandler(node.onError, "onError", main);

var bodyDef = stmt(createX(
"nextflow.script.BodyDef",
args(
closureX(null, node.main),
closureX(null, main),
constX(getSourceText(node)),
constX("workflow")
)
Expand All @@ -157,8 +160,7 @@ private void visitWorkflowTakes(Statement takes) {
}
}

private void visitWorkflowEmits(Statement emits, Statement main) {
var code = (BlockStatement)main;
private void visitWorkflowEmits(Statement emits, BlockStatement main) {
for( var stmt : asBlockStatements(emits) ) {
var es = (ExpressionStatement)stmt;
var emit = es.getExpression();
Expand All @@ -167,37 +169,39 @@ private void visitWorkflowEmits(Statement emits, Statement main) {
}
else if( emit instanceof AssignmentExpression ae ) {
var target = (VariableExpression)ae.getLeftExpression();
code.addStatement(assignS(target, emit));
main.addStatement(assignS(target, emit));
es.setExpression(callThisX("_emit_", args(constX(target.getName()))));
code.addStatement(es);
main.addStatement(es);
}
else {
var target = varX("$out");
code.addStatement(assignS(target, emit));
main.addStatement(assignS(target, emit));
es.setExpression(callThisX("_emit_", args(constX(target.getName()))));
code.addStatement(es);
main.addStatement(es);
}
}
}

private void visitWorkflowPublishers(Statement publishers, Statement main) {
var code = (BlockStatement)main;
private void visitWorkflowPublishers(Statement publishers, BlockStatement main) {
for( var stmt : asBlockStatements(publishers) ) {
var es = (ExpressionStatement)stmt;
var publish = (BinaryExpression)es.getExpression();
var target = asVarX(publish.getLeftExpression());
es.setExpression(callThisX("_publish_", args(constX(target.getName()), publish.getRightExpression())));
code.addStatement(es);
main.addStatement(es);
}
}

private void visitWorkflowHandler(Statement code, String name, BlockStatement main) {
if( code instanceof BlockStatement block )
main.addStatement(stmt(callX(varX("workflow"), name, args(closureX(null, block)))));
}

@Override
public void visitProcess(ProcessNode node) {
visitProcessDirectives(node.directives);
visitProcessInputs(node.inputs);
visitProcessOutputs(node.outputs);
visit(node.exec);
visit(node.stub);

if( "script".equals(node.type) )
node.exec.visit(new TaskCmdXformVisitor(sourceUnit));
Expand Down Expand Up @@ -380,7 +384,7 @@ private Statement processStub(Statement stub) {
@Override
public void visitFunction(FunctionNode node) {
if( RESERVED_NAMES.contains(node.getName()) ) {
syntaxError(node, "`${node.getName()}` is not allowed as a function name because it is reserved for internal use");
syntaxError(node, "`" + node.getName() + "` is not allowed as a function name because it is reserved for internal use");
return;
}
moduleNode.getScriptClassDummy().addMethod(node);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,9 @@ public void visitWorkflow(WorkflowNode node) {
visitWorkflowOutputs(node.emits, "emit");
visitWorkflowOutputs(node.publishers, "output");

visit(node.onComplete);
visit(node.onError);

currentDefinition = null;
vsc.popScope();
}
Expand Down
Loading
Loading