Skip to content

Commit d532bb1

Browse files
committed
Workflow onComplete and onError sections
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
1 parent 99f9edf commit d532bb1

File tree

18 files changed

+238
-58
lines changed

18 files changed

+238
-58
lines changed

docs/config.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -321,9 +321,15 @@ process {
321321
This limitation can be avoided by using the {ref}`strict config syntax <updating-config-syntax>`.
322322
:::
323323

324+
(config-workflow-handlers)=
325+
324326
## Workflow handlers
325327

326-
Workflow event handlers can be defined in the config file, which is useful for handling pipeline events without having to modify the pipeline code:
328+
:::{deprecated} 25.10.0
329+
Use a {ref}`trace observer <plugins-trace-observers>` in a plugin to add custom workflow handlers to a pipeline via configuration.
330+
:::
331+
332+
Workflow event handlers can be defined in the config file:
327333

328334
```groovy
329335
workflow.onComplete = {
@@ -337,4 +343,4 @@ workflow.onError = {
337343
}
338344
```
339345

340-
See {ref}`workflow-handlers` for more information.
346+
While these handlers can also be defined in the pipeline code, this approach is useful for handling workflow events without modifying the pipeline code. See {ref}`workflow-handlers` for more information.

docs/developer/plugins.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,8 @@ class MyExecutor extends Executor {
327327
}
328328
```
329329

330+
(plugins-trace-observers)=
331+
330332
### Trace observers
331333

332334
:::{versionchanged} 25.04

docs/migrations/25-10.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,31 @@ This page summarizes the upcoming changes in Nextflow 25.10, which will be relea
88
This page is a work in progress and will be updated as features are finalized. It should not be considered complete until the 25.10 release.
99
:::
1010

11+
## Enhancements
12+
13+
<h3>New syntax for workflow handlers</h3>
14+
15+
The workflow `onComplete` and `onError` handlers were previously defined by calling `workflow.onComplete` and `workflow.onError` in the pipeline script. These handlers can now be defined as `onComplete` and `onError` sections in an entry workflow:
16+
17+
```nextflow
18+
workflow {
19+
main:
20+
// ...
21+
22+
onComplete:
23+
println "workflow complete"
24+
25+
onError:
26+
println "error: ${workflow.errorMessage}"
27+
}
28+
```
29+
30+
This syntax is simpler and easier to use with the {ref}`strict syntax <strict-syntax-page>`. See {ref}`workflow-handlers` for details.
31+
1132
## Breaking changes
1233

1334
- The AWS Java SDK used by Nextflow was upgraded from v1 to v2, which introduced some breaking changes to the `aws.client` config options. See {ref}`the guide <aws-java-sdk-v2-page>` for details.
35+
36+
## Deprecations
37+
38+
- The use of workflow handlers in the configuration file has been deprecated. These workflow handlers should be defined in the pipeline script or a plugin instead. See {ref}`config-workflow-handlers` for details.

docs/notifications.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,22 @@ workflow.onComplete {
2323
}
2424
```
2525

26+
:::{versionadded} 25.10.0
27+
:::
28+
29+
Entry workflows can define an `onComplete` section instead of using `workflow.onComplete`:
30+
31+
```nextflow
32+
workflow {
33+
main:
34+
// ...
35+
36+
onComplete:
37+
println "Pipeline completed at: $workflow.complete"
38+
println "Execution status: ${ workflow.success ? 'OK' : 'failed' }"
39+
}
40+
```
41+
2642
(metadata-error-handler)=
2743

2844
### Error handler
@@ -39,6 +55,21 @@ workflow.onError {
3955
Both the `onError` and `onComplete` handlers are invoked when an error condition is encountered. The first is called as soon as the error is raised, while the second is called just before the pipeline execution is about to terminate. When using the `finish` {ref}`process-error-strategy`, there may be a significant gap between the two, depending on the time required to complete any pending job.
4056
:::
4157

58+
:::{versionadded} 25.10.0
59+
:::
60+
61+
Entry workflows can define an `onError` section instead of using `workflow.onError`:
62+
63+
```nextflow
64+
workflow {
65+
main:
66+
// ...
67+
68+
onError:
69+
println "Error: Pipeline execution stopped with the following message: ${workflow.errorMessage}"
70+
}
71+
```
72+
4273
## Mail
4374

4475
The built-in function `sendMail` allows you to send a mail message from a workflow script.

docs/reference/syntax.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ Parameters supplied via command line options, params files, and config files tak
123123

124124
A workflow can be a *named workflow* or an *entry workflow*.
125125

126-
A *named workflow* consists of a name and a body, and may consist of a *take*, *main*, *emit*, and *publish* section:
126+
A *named workflow* consists of a name and a body, and may consist of a *take*, *main*, and *emit* section:
127127

128128
```nextflow
129129
workflow greet {
@@ -146,28 +146,34 @@ workflow greet {
146146

147147
- The emit section consists of one or more *emit statements*. An emit statement can be a [variable name](#variable), an [assignment](#assignment), or an [expression statement](#expression-statement). If an emit statement is an expression statement, it must be the only emit.
148148

149-
- The publish section can be specified but is intended to be used in the entry workflow (see below).
150149

151-
152-
An *entry workflow* has no name and may consist of a *main* and *publish* section:
150+
An *entry workflow* has no name and may consist of a *main*, *publish*, *onComplete*, and *onError* section:
153151

154152
```nextflow
155153
workflow {
156154
main:
157155
greetings = channel.of('Bonjour', 'Ciao', 'Hello', 'Hola')
158156
messages = greetings.map { v -> "$v world!" }
159-
greetings.view { it -> '$it world!' }
157+
greetings.view { v -> "$v world!" }
160158
161159
publish:
162-
messages >> 'messages'
160+
messages = messages
161+
162+
onComplete:
163+
log.info 'Workflow completed successfully!'
164+
165+
onError:
166+
log.error 'Workflow failed.'
163167
}
164168
```
165169

166170
- Only one entry workflow may be defined in a script.
167171

168-
- The `main:` section label can be omitted if the publish section is not specified.
172+
- The `main:` section label can be omitted if the other sections not specified.
173+
174+
- The publish section consists of one or more *publish statements*. A publish statement is an [assignment](#assignment), where the assignment target is the name of a workflow output.
169175

170-
- The publish section consists of one or more *publish statements*. A publish statement is a [right-shift expression](#binary-expressions), where the left-hand side is an expression that refers to a value in the workflow body, and the right-hand side is an expression that returns a string.
176+
- The onComplete and onError sections consist of one or more [statements](#statements).
171177

172178
In order for a script to be executable, it must either define an entry workflow or be a code snippet as described [above](#script-declarations).
173179

modules/nf-lang/src/main/antlr/ScriptLexer.g4

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -355,6 +355,8 @@ WHEN : 'when';
355355
WORKFLOW : 'workflow';
356356
EMIT : 'emit';
357357
MAIN : 'main';
358+
ONCOMPLETE : 'onComplete';
359+
ONERROR : 'onError';
358360
PUBLISH : 'publish';
359361
TAKE : 'take';
360362

modules/nf-lang/src/main/antlr/ScriptParser.g4

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -227,29 +227,27 @@ workflowDef
227227

228228
workflowBody
229229
// explicit main block with optional take/emit blocks
230-
: (sep TAKE COLON nls workflowTakes)?
231-
sep MAIN COLON nls workflowMain
232-
(sep EMIT COLON nls workflowEmits)?
233-
(sep PUBLISH COLON nls workflowPublishers)?
230+
: (sep TAKE COLON nls take=workflowTakes)?
231+
sep MAIN COLON nls main=blockStatements
232+
(sep EMIT COLON nls emit=workflowEmits)?
233+
(sep PUBLISH COLON nls publish=workflowPublishers)?
234+
(sep ONCOMPLETE COLON nls onComplete=blockStatements)?
235+
(sep ONERROR COLON nls onError=blockStatements)?
234236

235237
// explicit emit block with optional take/main blocks
236-
| (sep TAKE COLON nls workflowTakes)?
237-
(sep MAIN COLON nls workflowMain)?
238-
sep EMIT COLON nls workflowEmits
239-
(sep PUBLISH COLON nls workflowPublishers)?
238+
| (sep TAKE COLON nls take=workflowTakes)?
239+
(sep MAIN COLON nls main=blockStatements)?
240+
sep EMIT COLON nls emit=workflowEmits
241+
(sep PUBLISH COLON nls publish=workflowPublishers)?
240242

241243
// implicit main block
242-
| sep? workflowMain
244+
| sep? main=blockStatements
243245
;
244246

245247
workflowTakes
246248
: identifier (sep identifier)*
247249
;
248250

249-
workflowMain
250-
: blockStatements
251-
;
252-
253251
workflowEmits
254252
: statement (sep statement)*
255253
;
@@ -517,6 +515,8 @@ identifier
517515
| WORKFLOW
518516
| EMIT
519517
| MAIN
518+
| ONCOMPLETE
519+
| ONERROR
520520
| PUBLISH
521521
| TAKE
522522
;
@@ -709,6 +709,8 @@ keywords
709709
| WORKFLOW
710710
| EMIT
711711
| MAIN
712+
| ONCOMPLETE
713+
| ONERROR
712714
| PUBLISH
713715
| TAKE
714716
| NullLiteral

modules/nf-lang/src/main/java/nextflow/script/ast/ScriptVisitorSupport.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ public void visitWorkflow(WorkflowNode node) {
6666
visit(node.main);
6767
visit(node.emits);
6868
visit(node.publishers);
69+
visit(node.onComplete);
70+
visit(node.onError);
6971
}
7072

7173
@Override

modules/nf-lang/src/main/java/nextflow/script/ast/WorkflowNode.java

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,13 +43,21 @@ public class WorkflowNode extends MethodNode {
4343
public final Statement main;
4444
public final Statement emits;
4545
public final Statement publishers;
46+
public final Statement onComplete;
47+
public final Statement onError;
4648

47-
public WorkflowNode(String name, Statement takes, Statement main, Statement emits, Statement publishers) {
49+
public WorkflowNode(String name, Statement takes, Statement main, Statement emits, Statement publishers, Statement onComplete, Statement onError) {
4850
super(name, 0, dummyReturnType(emits), dummyParams(takes), ClassNode.EMPTY_ARRAY, EmptyStatement.INSTANCE);
4951
this.takes = takes;
5052
this.main = main;
5153
this.emits = emits;
5254
this.publishers = publishers;
55+
this.onComplete = onComplete;
56+
this.onError = onError;
57+
}
58+
59+
public WorkflowNode(String name, Statement main) {
60+
this(name, EmptyStatement.INSTANCE, main, EmptyStatement.INSTANCE, EmptyStatement.INSTANCE, EmptyStatement.INSTANCE, EmptyStatement.INSTANCE);
5361
}
5462

5563
public boolean isEntry() {

modules/nf-lang/src/main/java/nextflow/script/control/ScriptResolveVisitor.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,8 @@ public void visitWorkflow(WorkflowNode node) {
8888
resolver.visit(node.main);
8989
resolver.visit(node.emits);
9090
resolver.visit(node.publishers);
91+
resolver.visit(node.onComplete);
92+
resolver.visit(node.onError);
9193
}
9294

9395
@Override

0 commit comments

Comments
 (0)