--- title: "Input formats: Spectronaut, DIA-NN, TMT and LFQ" author: "Sergio Ciordia" package: NADIA output: BiocStyle::html_document: toc: true toc_float: true number_sections: true vignette: > %\VignetteIndexEntry{Input formats: Spectronaut, DIA-NN, TMT and LFQ} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>", crop = NULL) ``` # Introduction NADIA accepts four input formats through `preprocess_spectronaut()`, `preprocess_diann()`, `preprocess_tmt()` and `preprocess_lfq()`. Although their source reports have different layouts, all four functions return a `proteomics_data` object with the same three components. Consequently, `process_proteomics()` and the downstream modules use the same interface regardless of how the data were acquired or searched. This vignette applies each preprocessor to an example report and explains the main distinction among them: **where NADIA obtains the experimental design**. Correctly linking every measured column or run to its condition and replicate is essential, because that mapping determines the groups compared later in the analysis. For example, `preprocess_spectronaut()` converts a long-format report—one row for each reported combination of LC-MS run and protein group—into an object of class `proteomics_data`. The `proteomics_data` object separates the information contained in the source report into three coordinated tables: - `metadata` contains one row per sample or LC-MS run, including the sample identifier, experimental condition, replicate and any available run-level information. - `protein_id` contains identification-level information for each protein group, including protein and gene annotations and, when available, run-specific evidence such as peptide and precursor counts, sequence coverage and identification scores. - `protein_quant` contains the protein-level quantification data. It includes the abundance measured for each protein group in each sample, together with the protein annotations and quantitative evidence required by the downstream workflow. ```{r input-format-overview, echo = FALSE, fig.align = "center", out.width = "100%", fig.cap = "Four report formats converge on the common `proteomics_data` structure used by the NADIA workflow.", fig.alt = "Workflow diagram showing Spectronaut, DIA-NN, TMT and LFQ reports entering their corresponding preprocessing functions. Spectronaut obtains the design from R.Condition and R.Replicate in the report. DIA-NN, TMT and LFQ obtain it from Abundance column suffixes or an annotation file supplied through annot_path. All four produce a proteomics_data object containing metadata, protein_id and protein_quant, which is passed to process_proteomics."} knitr::include_graphics("figures/Input_Formats_Workflow.svg") ```

Click the figure to enlarge it; press Esc or click outside the image to close it.

In the Spectronaut example used below, `metadata` contains 12 samples, whereas `protein_id` and `protein_quant` each contain 2,000 protein groups. The three tables have distinct but complementary roles. `metadata` defines the experimental design; `protein_quant` provides the abundance matrix used for normalisation, imputation and differential abundance analysis; and `protein_id` retains the more detailed identification evidence used for inspection and reporting. # Installation ```{r install, eval = FALSE} if (!requireNamespace("BiocManager", quietly = TRUE)) install.packages("BiocManager") BiocManager::install("NADIA") ``` ```{r library, message = FALSE} library(NADIA) ``` # DIA: Spectronaut ::: {.spectronaut-schema-note}

Spectronaut export schema included with NADIA

NADIA provides a ready-to-use schema for generating a compatible Spectronaut report. Locate the installed file from R with:

system.file("extdata", "NADIA_Report.rs", package = "NADIA")

`NADIA_Report.rs` is a binary file interpreted by Spectronaut, not a text file that lists the exported columns. Import it directly into Spectronaut and select it when generating the report. The resulting table will contain the structure and columns required by `preprocess_spectronaut()`. ::: Spectronaut exports a **long** report: one row per protein group *per run*, with the condition and replicate written in every row. Among the four input formats supported by NADIA, Spectronaut is the only one for which the experimental design does not need to be supplied separately. `preprocess_spectronaut()` extracts it directly from the `R.Condition` and `R.Replicate` columns in the report. ```{r spectronaut-input-structure, echo = FALSE, fig.align = "center", out.width = "100%", fig.cap = "Structure of the long-format Spectronaut report accepted by `preprocess_spectronaut()`.", fig.alt = "Diagram of a long-format Spectronaut report. Each row contains one LC-MS run and one protein group. R.FileName, R.Condition and R.Replicate identify the sample and experimental design; PG.ProteinGroups identifies the protein; PG.Quantity contains its abundance; and further columns contain run-level annotations."} knitr::include_graphics("figures/Spectronaut_Input_Structure.svg") ```

Click the figure to enlarge it; press Esc or click outside the image to close it.

The arguments accepted by `preprocess_spectronaut()` are summarised below. The input report and the order of the conditions are required; the remaining arguments control optional export and the aggregation of run-level annotations.

Show the complete preprocess_spectronaut() call and default arguments

```{r spectronaut-defaults, eval = FALSE} dia <- preprocess_spectronaut( # Required input and experimental conditions file_path = "spectronaut_report.tsv", condition_order = c("Control", "Treatment"), # Optional export; NULL keeps the result in memory export_dir = NULL, # Aggregation of annotations reported for multiple runs agg_coverage_run = "max", agg_coverage_global = "max", agg_mw = "max", agg_cscore_runwise = "mean", # Exported file names and progress messages timestamp_suffix = TRUE, verbose = TRUE ) ```

```{r dia} dia_file <- system.file("extdata", "nadia_dia_report.tsv.gz", package = "NADIA") dia <- preprocess_spectronaut(dia_file, condition_order = c("A", "B", "D"), verbose = FALSE) dia ``` Because the design already comes from the report, `condition_order` does not infer or redefine the condition assigned to each run. Instead, it selects the conditions to retain—excluding any other conditions present in the report—and sets their order in the subsequent stages of the analysis, including downstream comparisons. It should therefore list every condition that is to be analysed. The following output shows an example of the metadata table extracted and constructed from the imported Spectronaut report: ```{r dia-md} dia$metadata ``` Because the report is long and run-level, the same protein group may have multiple values for annotations such as sequence coverage, molecular weight and C-score. The `agg_*` arguments determine how those values are reduced when a single annotation value is required: ```{r dia-agg, eval = FALSE} dia_median <- preprocess_spectronaut( dia_file, condition_order = c("A", "B", "D"), agg_coverage_run = "median", # default "max" agg_cscore_runwise = "median", # default "mean" verbose = FALSE) ``` These arguments affect only the annotations stored alongside the quantitative data; they do not modify protein intensities. # DIA: DIA-NN A DIA-NN protein-group matrix (`report.pg_matrix.tsv`) is **wide**: it contains one row per protein group and one intensity column per run. This differs from the long Spectronaut report and resembles the wide Proteome Discoverer exports described below, which is why it requires a dedicated importer. ```{r diann-input-structure, echo = FALSE, fig.align = "center", out.width = "100%", fig.cap = "Structure of the wide DIA-NN protein-group matrix accepted by `preprocess_diann()`.", fig.alt = "Diagram of a wide DIA-NN protein-group matrix. Each row represents one protein group. Protein annotations occupy the first columns, followed by one Abundance column for each run. The experimental design is obtained from Condition_Replicate suffixes or from annot_path."} knitr::include_graphics("figures/DIANN_Input_Structure.svg") ```

Click the figure to enlarge it; press Esc or click outside the image to close it.

The arguments accepted by `preprocess_diann()` are shown below. The report and the condition order are required. `annot_path` is optional and is used when the run columns do not follow the `Abundance: _` convention.

Show the complete preprocess_diann() call and default arguments

```{r diann-defaults, eval = FALSE} diann <- preprocess_diann( # Required input and conditions to retain file_path = "report.pg_matrix.tsv", condition_order = c("Control", "Treatment"), # Optional sample annotation file; NULL uses the column suffixes annot_path = NULL, # Optional export; NULL keeps the result in memory export_dir = NULL, timestamp_suffix = TRUE, # Progress messages verbose = TRUE ) ```

```{r diann} diann_file <- system.file("extdata", "nadia_diann_report.tsv.gz", package = "NADIA") diann <- preprocess_diann(diann_file, condition_order = c("A", "B", "D"), verbose = FALSE) diann ``` ## Declaring the experimental design DIA-NN names each intensity column after the raw file it came from: ``` D:\DIA-NN\Projects\20241014_XII_CursoProtQ_DIA_500ng_sample_A1.raw ``` The text `sample_A1` may suggest condition A and replicate 1 to a reader, but that interpretation depends on a laboratory-specific naming convention. A regular expression that worked for this report could silently misclassify the next one. NADIA therefore does not infer the design from raw-file names. Instead, it can be declared in either of two ways: 1. **Rename the run columns** using the `Abundance: _` convention also used by Proteome Discoverer. The example distributed with NADIA follows this convention, so the call above needs no additional argument. 2. **Supply an annotation file through `annot_path`** and leave the report exactly as DIA-NN exported it. The `Column` field must contain the complete run headers, including the raw-file paths: ```{r diann-annot, eval = FALSE} preprocess_diann( "report.pg_matrix.tsv", condition_order = c("A", "B", "D"), annot_path = "design.tsv") ``` `design.tsv` must be a tab-separated file with one row for each run that should be imported and two columns named exactly `Column` and `Condition`. `Column` contains the complete header of the corresponding intensity column in the DIA-NN matrix, while `Condition` assigns that run to an experimental group. For example: ``` Column Condition D:\DIA-NN\Projects\experiment\sample_A1.raw A D:\DIA-NN\Projects\experiment\sample_A2.raw A D:\DIA-NN\Projects\experiment\sample_B1.raw B ``` The values in `Column` must match the report headers exactly, including the complete raw-file path when DIA-NN uses paths as column names. Runs omitted from the annotation file are not imported. `condition_order` then selects which of the annotated conditions are retained and sets their order in the subsequent analysis. If neither valid `Abundance: _` headers nor an annotation file are supplied, the error message lists the headers that must be entered in `Column`. Whichever route is used, NADIA constructs one row of `metadata` per retained run. The columns have the following meaning: - `R.FileName` preserves the complete intensity-column header read from the report. - `R.Condition` contains the experimental condition, obtained either from the renamed header or from `Condition` in the annotation file. - `R.Replicate` is read from a `_` suffix when that naming convention is used. For original DIA-NN headers without such a suffix, runs are numbered sequentially within each condition. - `Coding` is the sample identifier used to name the quantitative columns. It is the `_` suffix for renamed columns and the value of `Column` when `annot_path` is used. - `R.ProteinGroupsIdentified` reports how many protein groups have a non-missing intensity in that run. The following output shows an example of the metadata table extracted and constructed from the imported DIA-NN report: ```{r diann-md} diann$metadata ``` **Note on identification information.** The DIA-NN protein-group matrix contains little identification evidence. The two available fields are `N.Sequences`, which gives the total peptide-sequence count, and `N.Proteotypic.Sequences`, which gives the proteotypic peptide count. To preserve the common structure of the `proteomics_data` object, NADIA repeats these protein-level values across runs in `protein_id`, as it does for the global Proteome Discoverer metrics in TMT data. NADIA also calculates the number of protein groups with a reported intensity in each run and stores it as `R.ProteinGroupsIdentified` in the metadata table shown above. # TMT: Proteome Discoverer A TMT report from Proteome Discoverer is **wide**: it contains one row per protein group and one abundance column per sample. As with DIA-NN, the experimental design must be declared either through column names that follow the `Abundance: _` convention or through an annotation file supplied with `annot_path`. ```{r tmt-input-structure, echo = FALSE, fig.align = "center", out.width = "100%", fig.cap = "Structure of the wide Proteome Discoverer TMT report accepted by `preprocess_tmt()`.", fig.alt = "Diagram of a wide TMT report from Proteome Discoverer. Each row represents one protein group. Global protein and identification annotations occupy the first columns, followed by one Abundance column for each TMT channel. The design is obtained from Condition_Replicate suffixes or from annot_path."} knitr::include_graphics("figures/TMT_Input_Structure.svg") ```

Click the figure to enlarge it; press Esc or click outside the image to close it.

The arguments accepted by `preprocess_tmt()` are listed below. The report and the conditions to retain are required. `annot_path` provides an alternative way to declare the design when the abundance suffixes do not follow the naming convention.

Show the complete preprocess_tmt() call and default arguments

```{r tmt-defaults, eval = FALSE} tmt <- preprocess_tmt( # Required input and conditions to retain file_path = "tmt_report.tsv", condition_order = c("Control", "Treatment"), # Optional channel annotation file; NULL uses the column suffixes annot_path = NULL, # Optional export; NULL keeps the result in memory export_dir = NULL, timestamp_suffix = TRUE, # Progress messages verbose = TRUE ) ```

Click the figure to enlarge it; press Esc or click outside the image to close it.

Unlike DIA-NN and TMT, but like Spectronaut, this report contains **per-sample identification metrics**. For every sample, Proteome Discoverer can report a Mascot score, a PSM count and a peptide count in addition to the abundance. NADIA matches these column families to the corresponding sample by name rather than by their position in the report. The resulting identification values therefore retain differences among samples in `protein_id`. The arguments accepted by `preprocess_lfq()` are shown below. Only the report is required. Both `annot_path` and `condition_order` are optional; when omitted, the design and condition order are obtained from the abundance suffixes.

Show the complete preprocess_lfq() call and default arguments

```{r lfq-defaults, eval = FALSE} lfq <- preprocess_lfq( # Required input file_path = "lfq_report.tsv", # Optional design file and condition filtering/order annot_path = NULL, condition_order = NULL, # Optional export; NULL keeps the result in memory export_dir = NULL, timestamp_suffix = TRUE, # Progress messages verbose = TRUE ) ```

## Declaring the experimental design The design can be supplied in either of two ways: 1. **Use informative abundance-column suffixes.** Columns named `Abundance: A_1`, `Abundance: A_2` and `Abundance: B_1`, for example, identify the condition and replicate directly. This is the convention used by the example report distributed with NADIA. 2. **Supply an annotation file.** This route is used when the text after `Abundance: ` identifies a sample but does not encode its experimental condition and replicate. The report itself does not need to be renamed. `design.tsv` must be a tab-separated file with the columns `Column` and `Condition`. `Column` contains the exact sample identifier that follows the `Abundance: ` prefix in the report. For example: ``` Column Condition Sample01 Control Sample02 Control Sample03 Treatment Sample04 Treatment ``` Thus, the abundance column `Abundance: Sample01` is identified by `Sample01` in `Column`. Samples omitted from the annotation file are not imported. If `condition_order` is supplied, it additionally selects the conditions to retain and sets their order; otherwise, NADIA uses their order of appearance. The annotation file is passed to `preprocess_lfq()` as follows: ```{r lfq-annot, eval = FALSE} preprocess_lfq( "lfq_report.tsv", condition_order = c("Control", "Treatment"), annot_path = "design.tsv") ``` The example report distributed with NADIA contains three conditions with four replicates each. Its abundance-column suffixes already encode the complete design, so neither `annot_path` nor `condition_order` needs to be supplied: ```{r lfq-run} lfq_file <- system.file("extdata", "nadia_lfq_report.tsv.gz", package = "NADIA") lfq <- preprocess_lfq(lfq_file, verbose = FALSE) lfq ``` The following output shows an example of the metadata table extracted and constructed from the imported LFQ report: ```{r lfq-md} lfq$metadata ``` Any additional columns in `design.tsv` are ignored. Batch, TMT/LFQ experiment and other covariate information should instead be supplied through `covariate_df` in `process_proteomics()`, as explained in `vignette("batch-correction")`. # Where the design comes from The four importers produce the same object, but they obtain the experimental design in different ways: ::: {.input-format-summary} | Format | Layout | Design taken from | |---|---|---| | Spectronaut | long, run-level | the `R.Condition` / `R.Replicate` columns | | DIA-NN | wide | the `Abundance: _` suffixes, or `annot_path` | | TMT (PD) | wide | the `Abundance: _` suffixes, or `annot_path` | | LFQ (PD) | wide | the `Abundance: _` suffixes, or `annot_path` | ::: Only the long Spectronaut report carries its experimental design internally. The three wide formats use a common declaration strategy: either name the abundance columns according to the convention or provide a two-column annotation file. DIA-NN matches `Column` against its complete run headers, whereas the Proteome Discoverer importers match it against the text after `Abundance: `. `condition_order` then selects the conditions to retain and sets their order in the subsequent stages of the analysis. It is required for Spectronaut, DIA-NN and TMT. For LFQ it is optional; if omitted, NADIA uses the order in which the conditions are detected. # Naming the conditions Regardless of the input format, condition names become **column names in the design matrix**. The differential abundance model evaluates a contrast such as `"B-A"` as the expression `B - A`; it is not only a display label. This determines which condition names are safe. For example, conditions named `Tr-1` and `Ct-1` would produce `Tr-1-Ct-1`. R would interpret the hyphens as several subtraction operators rather than as characters within two condition names, so the intended contrast could not be constructed. The rule is simple: **each condition must have a syntactically valid R name.** In practice: | In the name | Verdict | Why | |---|---|---| | letters, digits (not first) | fine | `Ctrl`, `Treat24h` | | `.` full stop | **fine** | `Ct.1` gives the contrast `Tr.1-Ct.1` | | `_` underscore | **fine** | `Ct_a` gives `Tr_a-Ct_a` | | `-` hyphen | **rejected** | collides with the contrast separator | | space | **rejected** | not a syntactic name | | leading digit | **rejected** | `1Ct` is not a name; R would rewrite it `X1Ct` | Names that follow the rows marked **fine** can be used safely and allow the pipeline to complete without naming-related errors. If there is any doubt, consult the table above or check the proposed names with `make.names()`: ```{r names-test} candidates <- c("Ctrl", "Ct.1", "Ct_a", "Ct-1", "Ct 1", "1Ct") data.frame(name = candidates, valid = make.names(candidates) == candidates, what_r_would_do = make.names(candidates)) ``` The pipeline validates condition names before fitting the model and stops with an informative error if any are invalid. Sample identifiers must also remain consistent. A `Column` value in `design.tsv` that does not match the report produces an error, whereas samples omitted from the annotation file are excluded. If `metadata$Coding` is subsequently edited and no longer matches the corresponding `PG.Quantity_` column, the unmatched samples are removed from the analysis and the result contains fewer samples than expected. In all three **wide formats**, the shared suffix parser reads the replicate from the trailing digits of `_`. An underscore within the condition remains valid: `Ct_a_1` is parsed as condition `Ct_a`, replicate 1. When `annot_path` is used, the labels in its `Condition` field define the experimental conditions. A condition label can therefore be corrected in `design.tsv` without renaming any abundance column in the quantitative report; the values in `Column` must still match the corresponding report identifiers. # Four sources, one common object NADIA can start from four different report formats—Spectronaut, DIA-NN, TMT or LFQ—but all four preprocessing functions converge on the same `proteomics_data` object. Each importer adds a format-specific class while retaining this common parent class: ```{r classes} all4 <- list(Spectronaut = dia, DIA_NN = diann, TMT = tmt, LFQ = lfq) vapply(all4, function(x) class(x)[1], character(1)) vapply(all4, function(x) inherits(x, "proteomics_data"), logical(1)) ``` Every object contains the same three coordinated components: `metadata`, `protein_id` and `protein_quant`. ```{r elements} lapply(all4, names) ``` Their structure is common, but their contents reflect the information available in the original report. The main differences occur in `metadata` and `protein_id`: Spectronaut provides detailed run-level evidence, DIA-NN contains limited identification information, TMT reports global identification evidence for the multiplex and LFQ retains several identification metrics for each sample. The available metadata fields consequently also differ: ```{r md-cols} lapply(all4, function(x) colnames(x$metadata)) ``` At the quantitative level, all four `protein_quant` tables follow the same essential contract. In particular, the abundance of every sample is stored in a `PG.Quantity_` column linked to `metadata$Coding`. The numerical values and any supporting quantitative evidence still depend on the source, but the common organisation allows `process_proteomics()` to use the same interface for all four formats: ```{r pipeline} processed <- lapply(all4, process_proteomics, verbose = FALSE) lapply(processed, function(x) x$comparisons) ``` The resulting objects proceed through the same workflow, from normalisation and imputation to differential abundance analysis and visualisation. The available fields are handled according to the source report, while the shared object contract keeps the downstream interface unchanged. The presentation and export of the final results are described in `vignette("results-and-export")`. # Session information ```{r session} sessionInfo() ```