Skip to content

Commit 8990cc5

Browse files
committed
Small updates docs
1 parent 0fcb732 commit 8990cc5

File tree

2 files changed

+41
-31
lines changed

2 files changed

+41
-31
lines changed

docs/nextflow/building_blocks.md

Lines changed: 39 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ These channels can then be used by operators or serve as an input for the proces
148148
````{tab} Exercise 1.1
149149
Inspect and edit the `exercises/01_building_blocks/template.nf` script. Create a channel consisting of multiple paired-end files. For more information, read [`fromFilePairs`](https://www.nextflow.io/docs/latest/channel.html#fromfilepairs).
150150
151-
Once the Nextflow script is saved, run it with: `nextflow run template.nf`.
151+
Once the Nextflow script is saved, run it with: `nextflow run exercises/01_building_blocks/template.nf`.
152152
153153
````
154154

@@ -165,6 +165,8 @@ This is a `tuple` qualifier which we will use a lot during this workshop and dis
165165
### 2. Operators
166166
Operators are necessary to transform the content of channels in a format that is necessary for usage in the processes. There is a plethora of different operators[[5](https://www.nextflow.io/docs/latest/operator.html?highlight=view#)], however only a handful are used extensively. Here are some examples that you might come accross:
167167
- `collect`: e.g. when using a channel consisting of multiple independent files (e.g. fastq-files) and need to be assembled for a next process (output in a list data-type).
168+
169+
Example: [`exercises/01_building_blocks/operator_collect.nf`](https://github.com/vibbits/nextflow-workshop/blob/main/exercises/01_building_blocks/operator_collect.nf)
168170
```
169171
Channel
170172
.from( 1, 2, 3, 4 )
@@ -177,6 +179,8 @@ Channel
177179

178180
- `mix`: e.g. when assembling items from multiple channels into one channel for a next process (e.g. multiqc)
179181

182+
Example: [`exercises/01_building_blocks/operator_mix.nf`](https://github.com/vibbits/nextflow-workshop/blob/main/exercises/01_building_blocks/operator_mix.nf)
183+
180184
```
181185
c1 = Channel.of( 1,2,3 )
182186
c2 = Channel.of( 'a','b' )
@@ -200,6 +204,8 @@ z
200204
- All functions available on the item, are available on the `it` variable within the closure.
201205
- When an element is a list or tuple, you can use the `it[0]`, `it[1]`, etc. syntax to access the individual elements of your item.
202206

207+
Example: [`exercises/01_building_blocks/operator_map.nf`](https://github.com/vibbits/nextflow-workshop/blob/main/exercises/01_building_blocks/operator_map.nf)
208+
203209
```
204210
Channel
205211
.of( 1, 2, 3, 4, 5 )
@@ -275,14 +281,6 @@ process < name > {
275281
}
276282
```
277283

278-
Each process is executed independently and isolated from any other process. They communicate via asynchronous FIFO queues, i.e. one process will wait for the output of another and then runs reactively when the channel has contents.
279-
280-
281-
282-
```{image} ../img/nextflow/asynchronous-FIFO.png
283-
:align: center
284-
```
285-
286284
Here are a couple of examples of processes:
287285

288286

@@ -383,7 +381,37 @@ The **output** declaration block defines the channels created by the process to
383381

384382
---
385383

386-
A script, as part of the process, can be written in any language (bash, Python, Perl, Ruby, etc.). This allows to add self-written scripts in the pipeline. The script can be written in the process itself, or can be present as a script in another folder and is run from the process here. An example can be found in `exercises/01_building_blocks/hellofrompython.nf`.
384+
385+
Each process is executed independently and isolated from any other process. They communicate via asynchronous FIFO queues, i.e. one process will wait for the output of another and then runs reactively when the channel has contents.
386+
387+
388+
389+
```{image} ../img/nextflow/asynchronous-FIFO.png
390+
:align: center
391+
```
392+
393+
Let's exemplify this by running the script [`exercises/01_building_blocks/fifo.nf`](https://github.com/vibbits/nextflow-workshop/blob/main/exercises/01_building_blocks/fifo.nf) and inspect the order that the channels are being processed.
394+
395+
```
396+
N E X T F L O W ~ version 20.10.0
397+
Launching `fifo.nf` [nauseous_mahavira] - revision: a71d904cf6
398+
[- ] process > whosfirst -
399+
This is job number 6
400+
This is job number 3
401+
This is job number 7
402+
This is job number 8
403+
This is job number 5
404+
This is job number 4
405+
This is job number 1
406+
This is job number 2
407+
This is job number 9
408+
executor > local (10)
409+
[4b/aff57f] process > whosfirst (10) [100%] 10 of 10
410+
```
411+
412+
---
413+
414+
A script, as part of the process, can be written in any language (bash, Python, Perl, Ruby, etc.). This allows to add self-written scripts in the pipeline. The script can be written in the process itself, or can be present as a script in another folder and is run from the process here. An example can be found in [`exercises/01_building_blocks/hellofrompython.nf`](https://github.com/vibbits/nextflow-workshop/blob/main/exercises/01_building_blocks/hellofrompython.nf).
387415

388416
```
389417
#!/usr/bin/env nextflow
@@ -413,24 +441,6 @@ In this case, the output would be in the directory starting `work/f6/4916cd...`
413441

414442
---
415443

416-
Earlier, we described that Nextflow uses an asynchronous FIFO principle. Let's exemplify this by running the script `fifo.nf` and inspect the order that the channels are being processed.
417-
418-
```
419-
N E X T F L O W ~ version 20.10.0
420-
Launching `fifo.nf` [nauseous_mahavira] - revision: a71d904cf6
421-
[- ] process > whosfirst -
422-
This is job number 6
423-
This is job number 3
424-
This is job number 7
425-
This is job number 8
426-
This is job number 5
427-
This is job number 4
428-
This is job number 1
429-
This is job number 2
430-
This is job number 9
431-
executor > local (10)
432-
[4b/aff57f] process > whosfirst (10) [100%] 10 of 10
433-
```
434444

435445
````{tab} Exercise 1.4
436446
A `tag` directive can be added at the top of the process definition and allows you to associate each process execution with a custom label. Hence, it is really useful for logging or debugging. Add a tag for `num` and `str` in the process of the script `exercises/01_building_blocks/firstscript.nf` and inspect the output.
@@ -477,7 +487,7 @@ workflow {
477487

478488
## Extra exercises
479489
````{tab} Extra exercise 1
480-
Use the `view` operator on the output of the `valuesToFile` process in the script `firstscript.nf`. For this, you will first need to add an `emit` argument to the output of the process. More information is available in the documentation [here](https://www.nextflow.io/docs/edge/dsl2.html#process-named-output).
490+
Use the `view` operator on the output of the `valuesToFile` process in the script `exercises/01_building_blocks/firstscript.nf`. For this, you will first need to add an `emit` argument to the output of the process. More information is available in the documentation [here](https://www.nextflow.io/docs/edge/dsl2.html#process-named-output).
481491
482492
````
483493
````{tab} Solution 1

docs/nextflow/first_pipeline.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ The final FastQC script, with some additional comments is provided in `exercises
128128

129129
Now we will add the next step in our pipeline, which is **trimming and filtering the low quality reads**. For this process, we will use the tool `trimmomatic`.
130130

131-
The `fastqc.nf` script was extended with the trimmomatic process and is available in `trimmomatic.nf`.
131+
The `fastqc.nf` script was extended with the trimmomatic process and is available in `exercises/03_first_pipeline/trimmomatic.nf`.
132132
- A number of parameters have been added related to the trimmomatic process
133133
- The process `trimmomatic` with its inputs and outputs and the script has been created
134134
- The `workflow` now also contains the process trimmomatic, called as a function
@@ -167,7 +167,7 @@ include { fastqc as fastqc_raw; fastqc as fastqc_trim } from "${projectDir}/modu
167167
```
168168
Now we're ready to use a process, defined in a module, multiple times in a workflow.
169169

170-
Investigate & run the script `modules.nf` which contains the following code snippet
170+
Investigate & run the script `exercises/03_first_pipeline/modules.nf` which contains the following code snippet
171171
```
172172
...
173173
include { fastqc as fastqc_raw; fastqc as fastqc_trim } from "${projectDir}/../../modules/fastqc"

0 commit comments

Comments
 (0)