Install and load NADIA

Install NADIA through Bioconductor if it is not already available, and then load the package and example data.

if (!requireNamespace("BiocManager", quietly = TRUE)) {
  install.packages("BiocManager")
}
BiocManager::install("NADIA")

library(NADIA)
data(nadia_dia)

Run the processing workflow

process_proteomics() returns all analysis components in memory. Because export_dir is not supplied here, this call does not create any files.

res <- process_proteomics(
  nadia_dia,
  verbose = FALSE
)

The returned object contains the processed experiment, differential-abundance results, plotting tables, comparison names, and the parameters used in the run. The following code summarises the size and purpose of the main components used throughout this vignette.

component_summary <- data.frame(
  Component = c(
    "se_proc",
    "DEPs_results",
    "BoxPlot_Input",
    "PCA_Input",
    "comparisons",
    "parameters"
  ),
  Rows = c(
    nrow(res$se_proc),
    nrow(res$DEPs_results),
    nrow(res$BoxPlot_Input),
    nrow(res$PCA_Input),
    length(res$comparisons),
    length(res$parameters)
  ),
  Columns = c(
    ncol(res$se_proc),
    ncol(res$DEPs_results),
    ncol(res$BoxPlot_Input),
    ncol(res$PCA_Input),
    "\u2014",
    "\u2014"
  ),
  Purpose = c(
    "Processed assays and sample metadata",
    "Differential-abundance statistics",
    "Long-format intensities for boxplots",
    "Long-format intensities and significance flags for PCA",
    "Available pairwise comparisons",
    "Analysis settings and recorded function call"
  ),
  check.names = FALSE
)

knitr::kable(
  component_summary,
  caption = "Main components returned by process_proteomics().",
  align = c("l", "r", "r", "l")
)

The values are calculated directly with nrow() and ncol(), but the row count reflects the structure of each object rather than the number of unique proteins. DEPs_results contains 1,997 proteins across three comparisons (5,991 rows), BoxPlot_Input contains the same proteins across 12 samples and two assays (47,928 rows), and PCA_Input contains one row per protein and sample (23,964 rows). In the three data frames, Columns counts the available variables; in se_proc, its 1,997 rows and 12 columns correspond directly to proteins and samples, respectively. The final two components are not tabular: comparisons contains three contrast names and parameters contains 38 recorded analysis entries, so Columns is not applicable to them.

Choose between TSV and Parquet

TSV and Parquet store the same tabular information in different ways. TSV is tab-separated plain text that can be opened directly in a text editor or spreadsheet, making it convenient for manual inspection and exchange. However, column types must be inferred when it is read, and large files can require considerable disk space.

Parquet is a compressed, binary columnar format widely used in data analytics. Storing values by column allows software to read only the data it needs, making access faster and files smaller. Because Parquet is not human-readable, it requires compatible software: in R, arrow::read_parquet() can load it into memory, while DuckDB can query it directly.

The following table compares the main characteristics of the TSV and Parquet formats.

format_guide <- data.frame(
  Property = c(
    "File structure",
    "Readable as plain text",
    "Direct manual inspection",
    "Column types",
    "Storage requirement",
    "Repeated programmatic access",
    "Typical use"
  ),
  TSV = c(
    "Tab-delimited text",
    "Yes",
    "Text editor or spreadsheet",
    "Inferred when imported",
    "Larger for substantial tables",
    "Requires parsing the text",
    "Review, exchange, and small analyses"
  ),
  Parquet = c(
    "Compressed columnar binary",
    "No",
    "Requires Parquet-aware software",
    "Stored in the file schema",
    "Usually smaller",
    "Fast, including selected-column reads",
    "Data analytics and automated workflows"
  ),
  check.names = FALSE
)

knitr::kable(
  format_guide,
  caption = "Practical differences between TSV and Parquet exports.",
  align = c("l", "l", "l")
)

In summary, TSV is best for simple inspection and exchange, whereas Parquet is better suited to large datasets and repeated computational analysis because it is smaller, faster to read, and preserves column types. Use both when manual readability and efficient processing are equally important.

Create a set of TSV and/or Parquet files

process_proteomics() can write processed matrices and plotting inputs while it runs. Export is activated only when export_dir is supplied; with its default value of NULL, all results remain in memory and no directory or file is created. The value of export_format then selects TSV, Parquet, or a matched copy in both formats.

The following alternatives differ only in export_dir and export_format. Run the one that matches the intended downstream use; repeating the full analysis three times is not necessary.

# Export one TSV file for every enabled output.
process_proteomics(
  nadia_dia,
  export_dir = "results/tsv",
  export_format = "tsv",
  verbose = FALSE
)

# Export the same outputs only as compressed Parquet files.
process_proteomics(
  nadia_dia,
  export_dir = "results/parquet",
  export_format = "parquet",
  verbose = FALSE
)

# Export matched TSV and Parquet copies of every enabled output.
process_proteomics(
  nadia_dia,
  export_dir = "results/tsv_and_parquet",
  export_format = "both",
  verbose = FALSE
)

With the default export switches, NADIA writes five datasets: the normalised intensity matrix (export_normalized), the final imputed matrix (export_imputed), the differential-abundance results used by the volcano plots (export_volcano), the long-format intensities used by the boxplots (export_boxplot), and the imputed intensities with significance annotations used by PCA and heatmaps (export_pca). The matrix files contain one protein per row and one sample per intensity column, whereas the plotting inputs use long or results-oriented layouts suited to their downstream functions. File names include the normalisation and imputation methods so that outputs from different runs can be distinguished.

The next call writes all five default outputs as TSV files to a temporary directory and displays the types of files generated. In a real analysis, replace export_dir with a permanent project directory.

export_dir <- file.path(tempdir(), "nadia_vignette_tables")
unlink(export_dir, recursive = TRUE)

invisible(process_proteomics(
  nadia_dia,
  export_dir = export_dir,
  export_format = "tsv",
  verbose = FALSE
))

exported_files <- list.files(export_dir, full.names = TRUE)

file_contents <- ifelse(
  grepl("matrix_log2.*Imp", basename(exported_files)),
  "Imputed intensity matrix",
  ifelse(
    grepl("matrix_log2", basename(exported_files)),
    "Normalised intensity matrix",
    ifelse(
      grepl("VolcanoPlot", basename(exported_files)),
      "Differential-abundance results",
      ifelse(
        grepl("BoxPlot", basename(exported_files)),
        "Long-format boxplot input",
        "Long-format PCA input"
      )
    )
  )
)

export_manifest <- data.frame(
  File = basename(exported_files),
  Contents = file_contents,
  Size = sprintf("%.1f KB", file.info(exported_files)$size / 1024),
  check.names = FALSE
)

knitr::kable(
  export_manifest,
  caption = "Files created by the default TSV export.",
  align = c("l", "l", "r")
)

The filenames record the normalisation and imputation methods, reducing the risk of mixing tables from different runs. The plotting inputs are intentionally long: each row represents an observation required by its corresponding plot, whereas the two matrix files keep one protein per row.

Exports can also be assembled à la carte with the five switches shown above. Each export_* argument is TRUE by default; set it to FALSE to omit that dataset. The value of export_format is applied to every dataset that remains enabled. The following example requests matching TSV and Parquet copies of only the final imputed matrix and the differential-abundance results; the normalised matrix and the boxplot and PCA inputs are omitted.

process_proteomics(
  nadia_dia,
  export_dir = "results/selected_tables",
  export_format = "both",
  export_normalized = FALSE,
  export_imputed = TRUE,
  export_volcano = TRUE,
  export_boxplot = FALSE,
  export_pca = FALSE,
  verbose = FALSE
)

Write and read a .nadia file

The required input is the preprocessing object. The processing result and a Pattern Profiler result can be included when available.

The following example creates a .nadia archive from the bundled dataset and summarises selected file characteristics and metadata in the table below.

nadia_file <- file.path(tempdir(), "nadia_vignette_analysis.nadia")
unlink(c(nadia_file, paste0(nadia_file, ".wal")))

write_nadia(
  nadia_file,
  preprocessing = nadia_dia,
  result        = res,
  verbose       = FALSE
)

db <- read_nadia(nadia_file)

archive_summary <- data.frame(
  Property = c(
    "Archive size",
    "Samples",
    "Proteins",
    "Normalised assay",
    "Imputed assay",
    "Pattern Profiler stored"
  ),
  Value = c(
    sprintf("%.1f MB", file.size(nadia_file) / 1024^2),
    db$meta[["n_samples"]],
    db$meta[["n_proteins"]],
    db$meta[["norm_assay"]],
    db$meta[["imputed_assay"]],
    db$meta[["has_pattern_profiler"]]
  ),
  check.names = FALSE
)

knitr::kable(
  archive_summary,
  caption = "Summary of the example .nadia archive.",
  align = c("l", "l")
)

This initial archive contains the complete preprocessing and processing run; Pattern Profiler stored is therefore FALSE. Section 5.5 adds the optional clustering result to the same archive. If clustering has already been completed, it can instead be supplied directly to write_nadia() through pattern_profiler.

Restore and verify the result

NADIA provides three functions for restoring a .nadia archive and converting its contents back into the tables and objects generated during the original analysis:

• read_nadia() opens the archive and exposes its metadata, canonical tables, and stored views through a nadia_db object.
• nadia_result() reconstructs the proteomics_result returned by process_proteomics(), ready for NADIA plotting and reporting functions.
• nadia_preprocessing() restores the original proteomics_data object so that the processing workflow can be run again.

The following code verifies that the tables and objects restored from the .nadia archive are identical to those generated during the original analysis.

restored <- nadia_result(db)

roundtrip_check <- data.frame(
  Component = c(
    "Differential-abundance results",
    "PCA input",
    "Final imputed assay"
  ),
  Identical = c(
    identical(restored$DEPs_results, res$DEPs_results),
    identical(restored$PCA_Input, res$PCA_Input),
    identical(
      SummarizedExperiment::assay(restored$se_proc, "Impseqrob_min"),
      SummarizedExperiment::assay(res$se_proc, "Impseqrob_min")
    )
  ),
  check.names = FALSE
)

knitr::kable(
  roundtrip_check,
  caption = "Round-trip verification after restoring the archive.",
  align = c("l", "c")
)

All three checks return TRUE, confirming that the tables, their column types, and the final assay are reproduced exactly in this example.

Inspect stored tables and provenance

The .nadia file avoids unnecessary duplication. Values that can be reconstructed exactly, such as the log2 assay and PCA_Input, are exposed through SQL views rather than stored as additional copies. For imputation methods that leave observed values unchanged, only the replaced cells and their MAR or MNAR branch need to be stored.

The following table lists the SQL views from which NADIA can reconstruct the original content of the analysis.

archive_views <- subset(nadia_tables(nadia_file), type == "view")

view_description <- c(
  v_boxplot_input = "Boxplot input",
  v_deps_results = "Differential-abundance results",
  v_matrix_imputed = "Final imputed matrix",
  v_matrix_norm = "Normalised matrix",
  v_metadata = "Sample metadata",
  v_pca_input = "PCA input",
  v_protein_id = "Protein identification table",
  v_protein_quant = "Protein quantification table"
)

archive_views$Contents <- unname(view_description[archive_views$name])

knitr::kable(
  archive_views[, c("name", "rows", "Contents")],
  col.names = c("View", "Rows", "Reconstructed contents"),
  caption = "Canonical views available in the example archive.",
  align = c("l", "r", "l"),
  row.names = FALSE
)

The views provide the same analysis tables that users normally access in R. The archive also records package versions, function calls, source-file information, and analysis parameters.

provenance_parameters <- subset(
  db$parameters,
  name %in% c(
    "norm_method", "imp_method", "mar_method", "mnar_method",
    "de_method", "alpha", "logFC_threshold"
  )
)

knitr::kable(
  provenance_parameters[, c("step", "name", "value")],
  col.names = c("Stage", "Parameter", "Recorded value"),
  caption = "Selected parameters stored in the .nadia archive.",
  align = c("l", "l", "l"),
  row.names = FALSE
)

These values identify the main analysis choices rather than merely describing the exported table.

Query or extract the archive

Because a .nadia file is itself a DuckDB database, nadia_connect() can open a DBI connection for targeted SQL queries. DuckDB executes these queries directly, without reconstructing the complete analysis in R. The following example counts proteins in the three differential-abundance categories for each comparison.

con <- nadia_connect(nadia_file)

sql_summary <- DBI::dbGetQuery(
  con,
  paste(
    'SELECT "Comparison", "Change", COUNT(*) AS "Proteins"',
    'FROM v_deps_results',
    'GROUP BY "Comparison", "Change"',
    'ORDER BY "Comparison", "Change"'
  )
)

DBI::dbDisconnect(con, shutdown = TRUE)

knitr::kable(
  sql_summary,
  caption = "Protein counts queried directly from the .nadia archive.",
  align = c("l", "l", "r")
)

These SQL queries reproduce the counts of statistically significant proteins in each comparison using the stored Change classifications, which reflect the adjusted-p-value and fold-change criteria defined in the original analysis.

To work with software that cannot read .nadia files, NADIA provides nadia_export_parquet() to export each canonical view directly as a Parquet file. This export is performed by DuckDB directly and therefore does not require arrow. The following table lists all Parquet tables extracted from the example .nadia file.

parquet_dir <- file.path(tempdir(), "nadia_vignette_parquet")
unlink(parquet_dir, recursive = TRUE)
invisible(nadia_export_parquet(db, parquet_dir, verbose = FALSE))

parquet_manifest <- data.frame(
  File = list.files(parquet_dir),
  check.names = FALSE
)

knitr::kable(
  parquet_manifest,
  caption = "Parquet tables extracted from the .nadia archive.",
  align = "l"
)

Combine differential abundance with clustering

An existing .nadia file can optionally be extended with the results of a Pattern Profiler analysis. The following code generates the clustering result pp from the processed data and adds it to the file with nadia_add_pattern_profiler().

# Generate expression-pattern clusters.
pp <- pattern_profiler_analysis(
  res$se_proc,
  res$DEPs_results,
  assay_name = "Impseqrob_min",
  seed       = 123,
  verbose    = FALSE
)

# Add the clustering results to the existing .nadia file.
nadia_add_pattern_profiler(
  nadia_file,
  pattern_profiler = pp,
  verbose = FALSE
)

For downstream functional or enrichment analyses, NADIA can retrieve differential-abundance statistics together with expression-pattern assignments from the .nadia file. deps_with_clusters() combines these results in memory, whereas nadia_deps_with_clusters() reads the equivalent table from a .nadia archive. Both return the columns of DEPs_results followed by Cluster, Membership, and ClusterRank; rank 1 identifies the dominant cluster of a protein.

The following table shows six clustered proteins, combining their differential-abundance statistics with their dominant cluster and membership.

functional_input <- deps_with_clusters(
  res$DEPs_results,
  pp,
  assignment = "primary"
)

knitr::kable(
  head(functional_input[
    !is.na(functional_input$Cluster),
    c("Protein.IDs", "Comparison", "logFC", "adj.P.Val", "Change",
      "Cluster", "Membership")
  ], 6),
  caption = "Differential-abundance results with the dominant cluster of each protein.",
  align = c("l", "l", "r", "r", "l", "r", "r"),
  row.names = FALSE,
  digits = 4
)

Each row retains the original statistical results; Cluster identifies the dominant expression pattern, and Membership measures how strongly the protein belongs to it. Because soft clustering can associate a protein with several patterns, the next table compares the row counts for assignment = "primary" and assignment = "all".

soft <- deps_with_clusters(res$DEPs_results, pp, assignment = "all")

assignment_summary <- data.frame(
  Table = c(
    "Differential-abundance results",
    "assignment = \"primary\"",
    "assignment = \"all\""
  ),
  Rows = c(nrow(res$DEPs_results), nrow(functional_input), nrow(soft)),
  Proteins_without_cluster = c(
    NA_integer_,
    length(unique(functional_input$Protein.IDs[is.na(functional_input$Cluster)])),
    length(unique(soft$Protein.IDs[is.na(soft$Cluster)]))
  ),
  check.names = FALSE
)

knitr::kable(
  assignment_summary,
  col.names = c("Table", "Rows", "Proteins without a cluster"),
  caption = "Effect of the assignment mode on the number of rows.",
  align = c("l", "r", "r")
)

"primary" keeps one row per protein and comparison, whereas "all" adds rows for the retained secondary cluster assignments. With these default filters, both modes keep proteins without a cluster (NA), preserving the full set of tested proteins as a potential background for enrichment analysis.

The .nadia file also exposes the combined results through the SQL view v_deps_pattern_profiler, which is included in subsequent Parquet exports. The following query counts proteins by dominant cluster and differential-abundance category for comparison B-A; ClusterRank = 1 ensures that each protein is counted only in its dominant cluster.

con <- nadia_connect(nadia_file)

cluster_counts <- DBI::dbGetQuery(
  con,
  paste(
    'SELECT "Cluster", "Change", COUNT(*) AS "Proteins"',
    'FROM v_deps_pattern_profiler',
    'WHERE "ClusterRank" = 1 AND "Cluster" IS NOT NULL',
    'AND "Comparison" = \'B-A\'',
    'GROUP BY "Cluster", "Change"',
    'ORDER BY "Cluster", "Change"'
  )
)

DBI::dbDisconnect(con, shutdown = TRUE)

knitr::kable(
  cluster_counts,
  caption = "Differential-abundance categories per cluster in comparison B-A.",
  align = c("r", "l", "r")
)

The following table summarises the number of proteins in each differential-abundance category within each dominant cluster.

These counts show how differential-abundance categories are distributed across expression patterns. Storing both results in one .nadia file makes them available together through R or SQL for downstream analyses.

Save a complete widget page

Objects returned by results_list_widget(), protein_list_widget(), quant_list_widget(), and summary_list_widget() are complete HTML pages. htmltools::save_html() saves the page and copies its JavaScript and CSS dependencies into a neighbouring lib directory.

The following example creates and saves an interactive results page for comparison B-A. Replace widget_dir with a permanent project directory to keep the exported page.

widget_dir <- file.path(tempdir(), "nadia_vignette_widget")
dir.create(widget_dir, recursive = TRUE, showWarnings = FALSE)

results_page <- results_list_widget(
  res$DEPs_results,
  protein_quant = nadia_dia$protein_quant,
  comparisons = "B-A"
)

htmltools::save_html(
  results_page,
  file.path(widget_dir, "results_B-A.html")
)

The saved page retains the full NADIA toolbar, including filters and Excel export. Keep the HTML file and its lib directory together when moving or sharing the page so that it remains functional offline.

Save an individual widget as one file

results_list_reactable() and the Highcharts plotting functions return actual htmlwidget objects. They can be embedded into a single HTML file with htmlwidgets::saveWidget(..., selfcontained = TRUE).

The following example creates a results table for comparison B-A and saves it as a single HTML file, using the output directory from the previous example.

result_table <- results_list_reactable(
  res$DEPs_results,
  protein_quant = nadia_dia$protein_quant,
  comparisons = "B-A"
)

htmlwidgets::saveWidget(
  result_table,
  file.path(widget_dir, "results_B-A_standalone.html"),
  selfcontained = TRUE
)

The same approach can be used for a Highcharts figure by selecting one chart from the named list before saving it.

volcanoes <- volcano_highchart_list(
  res$DEPs_results,
  comparisons = "B-A",
  alpha = 0.05,
  lfc_thr = 1
)

htmlwidgets::saveWidget(
  volcanoes[["B-A"]],
  "volcano_B-A.html",
  selfcontained = TRUE
)

The resulting HTML file contains the widget and its dependencies, so it can be shared on its own and opened offline, at the cost of a larger file. Reactable tables retain sorting and search, but not the additional NADIA toolbar supplied by results_list_widget().