You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,7 +20,7 @@
20
20
21
21
## Introduction
22
22
23
-
**nf-core/rnaseq** is a bioinformatics pipeline that can be used to analyse RNA sequencing data obtained from organisms with a reference genome and annotation. It takes a samplesheet and FASTQ files as input, performs quality control (QC), trimming and (pseudo-)alignment, and produces a gene expression matrix and extensive QC report.
23
+
**nf-core/rnaseq** is a bioinformatics pipeline that can be used to analyse RNA sequencing data obtained from organisms with a reference genome and annotation. It takes a samplesheet with FASTQ files or pre-aligned BAM files as input, performs quality control (QC), trimming and (pseudo-)alignment, and produces a gene expression matrix and extensive QC report.
24
24
25
25
Each row represents a fastq file (single-end) or a pair of fastq files (paired end). Rows with the same sample identifier are considered technical replicates and merged automatically. The strandedness refers to the library preparation and will be automatically inferred if set to `auto`.
78
78
79
+
The pipeline supports a two-step reprocessing workflow using BAM files from previous runs. Run initially with `--save_align_intermeds` to generate a samplesheet with BAM paths, then reprocess using `--skip_alignment` for efficient downstream analysis without repeating expensive alignment steps. This feature is designed specifically for pipeline-generated BAMs.
80
+
79
81
> [!WARNING]
80
82
> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_; see [docs](https://nf-co.re/docs/usage/getting_started/configuration#custom-configuration-files).
Copy file name to clipboardExpand all lines: docs/output.md
+12-6Lines changed: 12 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,6 +10,10 @@ nextflow run nf-core/rnaseq -profile test_full,<docker/singularity/institute>
10
10
11
11
The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.
12
12
13
+
:::tip
14
+
Many of the BAM files produced by this pipeline can be reused as input for future runs. This is particularly useful for reprocessing data or running downstream analysis steps without repeating computationally expensive alignment. See the [usage documentation](https://nf-co.re/rnaseq/usage#using-bam-files-as-input) for details on using BAM files as input.
15
+
:::
16
+
13
17
## Pipeline overview
14
18
15
19
The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes data using the following steps:
@@ -213,8 +217,8 @@ When `--remove_ribo_rna` is specified, the pipeline uses [SortMeRNA](https://git
213
217
<summary>Output files</summary>
214
218
215
219
-`star_salmon/`
216
-
-`*.Aligned.out.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the reference genome will be placed in this directory.
217
-
-`*.Aligned.toTranscriptome.out.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the transcriptome will be placed in this directory.
220
+
-`*.Aligned.out.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the reference genome will be placed in this directory. These files can be reused as `genome_bam` input in future pipeline runs.
221
+
-`*.Aligned.toTranscriptome.out.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the transcriptome will be placed in this directory. These files can be reused as `transcriptome_bam` input in future pipeline runs.
218
222
-`salmon.merged.gene_counts.tsv`: Matrix of gene-level raw counts across all samples.
219
223
-`salmon.merged.gene_tpm.tsv`: Matrix of gene-level TPM values across all samples.
220
224
-`salmon.merged.gene.SummarizedExperiment.rds`: RDS object that can be loaded in R that contains a [SummarizedExperiment](https://bioconductor.org/packages/release/bioc/html/SummarizedExperiment.html) container with the abundance TPM (`tpm`), estimated counts (`counts`) and gene length (`length`), estimated library size-scaled counts (`counts_scaled`), estimated length-scaled counts (`counts_length_scaled`) in the assays slot for genes.
@@ -276,8 +280,8 @@ The STAR section of the MultiQC report shows a bar plot with alignment rates: go
276
280
-`rsem.merged.transcript_tpm.tsv`: Matrix of isoform-level TPM values across all samples.
277
281
-`*.genes.results`: RSEM gene-level quantification results for each sample.
278
282
-`*.isoforms.results`: RSEM isoform-level quantification results for each sample.
279
-
-`*.STAR.genome.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the reference genome will be placed in this directory.
280
-
-`*.transcript.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the transcriptome will be placed in this directory.
283
+
-`*.STAR.genome.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the reference genome will be placed in this directory. These files can be reused as `genome_bam` input in future pipeline runs.
284
+
-`*.transcript.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the transcriptome will be placed in this directory. These files can be reused as `transcriptome_bam` input in future pipeline runs.
281
285
-`star_rsem/<SAMPLE>.stat/`
282
286
-`*.cnt`, `*.model`, `*.theta`: RSEM counts and statistics for each sample.
283
287
-`star_rsem/log/`
@@ -299,7 +303,7 @@ You can choose to align and quantify your data with RSEM by providing the `--ali
299
303
<summary>Output files</summary>
300
304
301
305
-`hisat2/`
302
-
-`<SAMPLE>.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the reference genome will be placed in this directory.
306
+
-`<SAMPLE>.bam`: If `--save_align_intermeds` is specified the original BAM file containing read alignments to the reference genome will be placed in this directory. These files can be reused as `genome_bam` input in future pipeline runs.
303
307
-`hisat2/log/`
304
308
-`*.log`: HISAT2 alignment report containing the mapping results summary.
305
309
-`hisat2/unmapped/`
@@ -323,7 +327,7 @@ The pipeline has been written in a way where all the files generated downstream
323
327
<summary>Output files</summary>
324
328
325
329
-`<ALIGNER>/`
326
-
-`<SAMPLE>.sorted.bam`: If `--save_align_intermeds` is specified the original coordinate sorted BAM file containing read alignments will be placed in this directory.
330
+
-`<SAMPLE>.sorted.bam`: If `--save_align_intermeds` is specified the original coordinate sorted BAM file containing read alignments will be placed in this directory. These files can be reused as `genome_bam` input in future pipeline runs.
327
331
-`<SAMPLE>.sorted.bam.bai`: If `--save_align_intermeds` is specified the BAI index file for the original coordinate sorted BAM file will be placed in this directory.
328
332
-`<SAMPLE>.sorted.bam.csi`: If `--save_align_intermeds --bam_csi_index` is specified the CSI index file for the original coordinate sorted BAM file will be placed in this directory.
329
333
-`<ALIGNER>/samtools_stats/`
@@ -864,6 +868,8 @@ A number of genome-specific files are generated by the pipeline because they are
864
868
- Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.yml`. The `pipeline_report*` files will only be present if the `--email` / `--email_on_fail` parameter's are used when running the pipeline.
865
869
- Reformatted samplesheet files used as input to the pipeline: `samplesheet.valid.csv`.
866
870
- Parameters used by the pipeline run: `params.json`.
871
+
-`samplesheets/`
872
+
-`samplesheet_with_bams.csv`: **Auto-generated complete samplesheet** (only created when using `--save_align_intermeds`) containing all samples with BAM file paths. For samples processed from FASTQ, includes paths to newly generated BAMs; for samples that were BAM input, preserves the original input paths. This comprehensive samplesheet can be used directly for future pipeline runs, enabling efficient reprocessing without re-alignment.
|`sample`| Custom sample name. This entry will be identical for multiple sequencing libraries/runs from the same sample. Spaces in sample names are automatically converted to underscores (`_`). |
98
-
|`fastq_1`| Full path to FastQ file for Illumina short reads 1. File has to be gzipped and have the extension ".fastq.gz" or ".fq.gz". |
99
-
|`fastq_2`| Full path to FastQ file for Illumina short reads 2. File has to be gzipped and have the extension ".fastq.gz" or ".fq.gz". |
100
-
|`strandedness`| Sample strand-specificity. Must be one of `unstranded`, `forward`, `reverse` or `auto`. |
|`sample`| Custom sample name. This entry will be identical for multiple sequencing libraries/runs from the same sample. Spaces in sample names are automatically converted to underscores (`_`). |
98
+
|`fastq_1`| Full path to FastQ file for Illumina short reads 1. File has to be gzipped and have the extension ".fastq.gz" or ".fq.gz". |
99
+
|`fastq_2`| Full path to FastQ file for Illumina short reads 2. File has to be gzipped and have the extension ".fastq.gz" or ".fq.gz". |
100
+
|`strandedness`| Sample strand-specificity. Must be one of `unstranded`, `forward`, `reverse` or `auto`. |
101
+
|`genome_bam`|**Optional**. Full path to genome-aligned BAM file. Typically from previous pipeline runs (see [output documentation](https://nf-co.re/rnaseq/output#star-salmon-and-kallisto) or [STAR/RSEM](https://nf-co.re/rnaseq/output#star-via-rsem)). |
102
+
|`transcriptome_bam`|**Optional**. Full path to transcriptome-aligned BAM file. Typically from previous pipeline runs (see [output documentation](https://nf-co.re/rnaseq/output#star-salmon-and-kallisto) or [STAR/RSEM](https://nf-co.re/rnaseq/output#star-via-rsem)). |
103
+
|`percent_mapped`|**Optional**. Percentage of reads that mapped during alignment (0-100). Useful for quality assessment and filtering. |
101
104
102
105
An [example samplesheet](../assets/samplesheet.csv) has been provided with the pipeline.
103
106
104
107
> **NB:** The `group` and `replicate` columns were replaced with a single `sample` column as of v3.1 of the pipeline. The `sample` column is essentially a concatenation of the `group` and `replicate` columns, however it now also offers more flexibility in instances where replicate information is not required e.g. when sequencing clinical samples. If all values of `sample` have the same number of underscores, fields defined by these underscore-separated names may be used in the PCA plots produced by the pipeline, to regain the ability to represent different groupings.
105
108
109
+
### BAM input for reprocessing workflow
110
+
111
+
The pipeline supports a **two-step workflow** for efficient reprocessing without expensive alignment steps. This feature is designed specifically for re-running with BAM files generated by previous runs of this same pipeline.
112
+
113
+
#### Step 1: Initial run with BAM generation
114
+
115
+
Run the pipeline normally, adding `--save_align_intermeds` to publish BAM files and generate a reusable samplesheet:
116
+
117
+
```bash
118
+
nextflow run nf-core/rnaseq \
119
+
--input samplesheet.csv \
120
+
--save_align_intermeds \
121
+
--outdir results_initial \
122
+
-profile docker
123
+
```
124
+
125
+
This creates `samplesheets/samplesheet_with_bams.csv` containing paths to the generated BAM files.
126
+
127
+
#### Step 2: Reprocessing run with BAM input
128
+
129
+
Use the auto-generated samplesheet to reprocess data, skipping alignment:
130
+
131
+
```bash
132
+
nextflow run nf-core/rnaseq \
133
+
--input samplesheets/samplesheet_with_bams.csv \
134
+
--skip_alignment \
135
+
--outdir results_reprocessed \
136
+
-profile docker
137
+
```
138
+
139
+
The pipeline will skip alignment and indexing steps, putting the BAM files through post-processing and quantification only.
> **⚠️ Warning**: This feature is designed specifically for BAM files generated by this pipeline. Using arbitrary BAM files from other sources is **not officially supported** and will likely only work via the two-step workflow described above. Users attempting to use other BAMs do so at their own risk.
154
+
155
+
**Key technical details:**
156
+
157
+
- The pipeline automatically indexes provided BAM files
158
+
- You can provide just `genome_bam`, just `transcriptome_bam`, or both
159
+
- Mixed samplesheets (some samples with FASTQ, others with BAM) are supported
160
+
- For BAM file locations from pipeline outputs, see the [output documentation](https://nf-co.re/rnaseq/output)
161
+
162
+
This workflow is ideal for tweaking downstream processing steps (quantification methods, QC parameters, differential expression analysis) without repeating time-consuming alignment.
163
+
106
164
## FASTQ sampling
107
165
108
166
If you would like to reduce the number of reads used in the analysis, for example to test pipeline operation with limited resource usage, you can make use of the FASTP option for trimming (see below). FASTP has an option to take the first `n` reads of input FASTQ file(s), so this can be used to reduce the reads passed to subsequent steps. For example, to pass only the first 10,000 reads for trimming you would set input paramters like:
0 commit comments