Package 'scSimEval'

Description

Times an expression and measures change in memory allocation.

Usage

benchmark_resource_usage(expr)

Arguments

Value

A list with elapsed seconds and peak memory allocated (MB).

Examples

benchmark_resource_usage({ Sys.sleep(0.01); 1 + 1 })

Description

A unified, high-performance, and beginner-friendly benchmarking runner designed for multi-dataset and multi-simulator studies (e.g., 16 datasets x 6 simulators). Automatically manages memory allocation with sparse matrix coercions, parallel execution across datasets, progress tracking, and produces both granular and cross-dataset aggregated benchmark tables ready for visualization.

Usage

benchmark_simulation_suite(
  datasets,
  simulators,
  compute_bivariate = FALSE,
  threads = 1,
  auto_sparse = TRUE,
  run_gc = TRUE,
  save_path = NULL,
  verbose = TRUE
)

Arguments

Value

An object of class scSimEval_multidataset_benchmark containing:

• granular_benchmark_table: Tidy data.frame containing all evaluated metrics across every dataset and simulator.

• benchmark_summary_table: Tidy data.frame containing cross-dataset averaged fidelity scores for each simulator (ready for plot_benchmark_bubble_matrix()).

• dataset_summary_table: Summary of reference dataset dimensions, sparsity, and library properties.

• methods: Vector of evaluated simulator names.

• datasets: Vector of evaluated dataset names.

• cross_dataset_stats: Summary statistics (mean, SD, SE) for each metric across datasets.

Examples

data(example_scrna, package = "scSimEval")
refs <- list("CohortA" = example_scrna$ref)
sims <- list("Splatter" = list("CohortA" = example_scrna$sim))
res <- benchmark_simulation_suite(refs, sims, verbose = FALSE)
print(res)

Description

Computes global and cell-type-stratified Pearson and Spearman correlations and Kullback-Leibler (KL) divergence between empirical reference and simulated single-cell chromatin accessibility profiles.

Usage

calc_accessibility_profile_concordance(
  ref_data,
  sim_data,
  cell_types = NULL,
  use_tfidf = TRUE
)

Arguments

Value

A list containing global PCC, global SCC, per-cell-type mean PCC/SCC, and mean accessibility KL divergence.

Examples

data(example_multiomics, package = "scSimEval")
r_rna <- example_multiomics$ref_multi$rna
r_atac <- example_multiomics$ref_multi$atac
calc_accessibility_profile_concordance(r_rna, r_atac)

Description

Fits a polynomial curve relating peak/bin mean accessibility to non-zero cell proportion (NZP / detection frequency) as modeled in the simATAC framework (Navidi et al., Genome Biology 2021): NZP = c0 + c1 * mean + c2 * mean^2.

Usage

calc_accessibility_sparsity_curve(data, poly_degree = 2)

Arguments

Value

A list with estimated coefficients (c0, c1, c2), R-squared, Spearman/Pearson correlations, and model fit summary.

Examples

data(example_scrna, package = "scSimEval")
calc_accessibility_sparsity_curve(example_scrna$ref)

Description

Compute All Bivariate (2D) Distance & Accuracy Metrics for Joint Distributions

Usage

calc_all_bivariate_metrics(ref_mat, sim_mat, metric_prefix = "", threads = 1)

Arguments

Value

Named list of bivariate metrics.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_all_bivariate_metrics(ref, sim)

Description

Compute All Univariate Distance & Accuracy Metrics for a Feature

Usage

calc_all_univariate_metrics(ref, sim, metric_prefix = "")

Arguments

Value

Named list of univariate distance metrics.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_all_univariate_metrics(ref, sim)

Description

Computes the Adjusted Mutual Information (AMI) between two clusterings or cell-type label assignments, correcting for chance expected mutual information under the hypergeometric model with fixed marginals (Vinh et al., JMLR 2010), as used in scMoMtF (Lan et al., PLOS Comput Biol 2024).

Usage

calc_ami(
  pred,
  truth,
  average_method = c("arithmetic", "max", "min", "geometric")
)

Arguments

Value

AMI score in [0, 1] (adjusted for chance).

References

Vinh, N. X., Epps, J., & Bailey, J. (2010). Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. Journal of Machine Learning Research, 11, 2837-2854.

Lan, W., Ling, T., Chen, Q. et al. scMoMtF: An interpretable multitask learning framework for single-cell multi-omics data analysis. PLOS Comput Biol 20(12): e1012679 (2024).

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_ami(pred, truth)

Description

Calculate Adjusted Rand Index (ARI)

Usage

calc_ari(pred, truth)

Arguments

Value

ARI value between -1 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_ari(pred, truth)

Description

Evaluates whether simulated paired scATAC-seq and scRNA-seq profiles capture the true regulatory coupling where regional chromatin opening gates target gene transcription (scMultiSim; Li et al., Nat Methods 2023).

Usage

calc_atac_rna_coupling(atac_data, rna_data, linked_pairs = NULL)

Arguments

Value

A list containing mean coupling correlation, positive coupling ratio, and mean R-squared.

Examples

data(example_multiomics, package = "scSimEval")
r_rna <- example_multiomics$ref_multi$rna
r_atac <- example_multiomics$ref_multi$atac
calc_atac_rna_coupling(r_rna, r_atac)

Description

Integrated from countsimQC (Soneson & Robinson). Treats dataset identity (reference vs. simulation) as cluster labels to test if the two datasets separate into distinct clusters or remain well-mixed.

Usage

calc_between_dataset_silhouette(ref, sim, subsample_size = 300)

Arguments

Value

A named list with global and local between-dataset silhouette widths.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_between_dataset_silhouette(ref, sim)

Description

Calculate Bhattacharyya Distance (BH)

Usage

calc_bhattacharyya(ref, sim, align = TRUE)

Arguments

Value

Statistical divergence between discrete probability measures.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_bhattacharyya(ref, sim)

Description

Calculate Calinski-Harabasz Index (CH)

Usage

calc_calinski_harabasz(data, cluster_labels)

Arguments

Value

Calinski-Harabasz index.

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 5))
dist_mat <- stats::dist(data)
calc_calinski_harabasz(data, cl)

Description

Evaluates the deviation and goodness-of-fit of single-cell clustering labels directly from raw or normalized count matrices without requiring ground-truth cell-type labels (Fang et al., Genome Biology 2022; benchmarked in scDesign3, Song et al., Nat Biotechnol 2024). CDI calculates the penalized log-likelihood (CDI-AIC and CDI-BIC) of count data under a cluster-specific Poisson or Negative Binomial GLM with cell size factors.

Usage

calc_cdi(
  counts,
  cluster_labels,
  size_factors = NULL,
  model = c("poisson", "nb"),
  top_features = NULL
)

Arguments

Value

A list containing:

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 10))
dist_mat <- stats::dist(data)
calc_cdi(data, cl)

Description

Integrated from scCluBench (Xu et al., AAAI 2026). Aligns predicted clusters to ground truth classes using the Hungarian bipartite matching algorithm, then computes the fraction of correctly mapped cells.

Usage

calc_clustering_accuracy(pred, truth)

Arguments

Value

Clustering accuracy between 0 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_clustering_accuracy(pred, truth)

Description

Tests the hypothesis that group-specific distance distributions of k-nearest neighbor cells have the same underlying unspecified distribution. Computes cell-specific p-values via Anderson-Darling tests (if kSamples installed) or two-sample Kolmogorov-Smirnov / Kruskal-Wallis tests, with optional delegation to CellMixS::cms.

Usage

calc_cms(
  data = NULL,
  batch_info,
  k = NULL,
  n_pcs = 30,
  coords = NULL,
  cell_min = 4,
  unbalanced = FALSE
)

Arguments

Value

A named list:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
calc_cms(coords, batch)

Description

Automatically discovers or evaluates co-expression modules and measures the preservation of intra-module vs. inter-module correlation structure.

Usage

calc_coexpression_module_fidelity(
  ref_mat,
  sim_mat,
  modules = NULL,
  n_modules = 5,
  top_genes = 200
)

Arguments

Value

A list summarizing module correlation preservation, RMSE, and modularity ratio fidelity.

Examples

data(example_scrna, package = "scSimEval")
calc_coexpression_module_fidelity(example_scrna$ref, example_scrna$sim)

Description

Calculate Completeness Score

Usage

calc_completeness(pred, truth)

Arguments

Value

Completeness score between 0 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_completeness(pred, truth)

Description

Calculate Cluster Connectivity

Usage

calc_connectivity(dist_mat, cluster_labels)

Arguments

Value

Connectivity metric.

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 10))
dist_mat <- stats::dist(data)
calc_connectivity(dist_mat, cl)

Description

Evaluates the preservation of co-regulated gene/feature modules between reference and simulated multi-omics datasets (Monzo et al., 2025; Arzalluz-Luque et al., 2022).

Usage

calc_coregulation_fidelity(
  ref_data,
  sim_data,
  modules,
  method = c("pearson", "spearman")
)

Arguments

Value

A list containing module correlation r, RMSE, MAE, and modularity fidelity.

Examples

data(example_scrna, package = "scSimEval")
modules <- list(Module_1 = rownames(example_scrna$ref)[seq_len(30)],
                Module_2 = rownames(example_scrna$ref)[31:60])
calc_coregulation_fidelity(example_scrna$ref, example_scrna$sim, modules)

Description

Evaluates the biological accuracy and reconstruction fidelity of cross-modality generation (e.g. predicting scATAC from scRNA or vice versa; Zhai et al., Genome Biology 2024). Computes cell-wise and feature-wise Pearson and Spearman correlations, cosine similarity, RMSE, and MAE between predicted and measured multi-omics profiles.

Usage

calc_cross_modal_generation(true_data, pred_data)

Arguments

Value

A list containing cell-wise and feature-wise correlation, cosine similarity, RMSE, and MAE.

Examples

data(example_multiomics, package = "scSimEval")
r_rna <- example_multiomics$ref_multi$rna
r_atac <- example_multiomics$ref_multi$atac
calc_cross_modal_generation(r_rna, r_atac)

Description

Evaluates whether cross-modality feature relationships (e.g. peak-to-gene links, promoter accessibility vs. gene expression, or mRNA vs. surface protein) are accurately captured by the simulation method compared to the empirical reference.

Usage

calc_cross_modality_correlation(
  ref_mod1,
  ref_mod2,
  sim_mod1,
  sim_mod2,
  feature_pairs = NULL,
  method = c("spearman", "pearson")
)

Arguments

Value

A named list of the 7 univariate accuracy metrics comparing the reference vs. simulated cross-modality correlation distributions.

Examples

data(example_multiomics, package = "scSimEval")
m <- example_multiomics
calc_cross_modality_correlation(m$ref_multi$rna, m$ref_multi$atac,
                                m$sim_multi$rna, m$sim_multi$atac)

Description

Calculate Davies-Bouldin Index (DB)

Usage

calc_davies_bouldin(data, cluster_labels)

Arguments

Value

Davies-Bouldin index (lower is better).

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 5))
dist_mat <- stats::dist(data)
calc_davies_bouldin(data, cl)

Description

Evaluates the accuracy of cell-type mixture proportion estimation / deconvolution methods between empirical reference and simulated cell populations (Chen et al., Bioinformatics 2021). Can accept precomputed proportion matrices OR cell-type vectors (and optional batch labels) to calculate proportion preservation across batches.

Usage

calc_deconvolution_accuracy(
  true_proportions,
  estimated_proportions,
  batch_ref = NULL,
  batch_sim = NULL
)

Arguments

Value

A list containing RMSE, MAE, Pearson correlation (r), Spearman correlation (rho), Jensen-Shannon Divergence (JSD), and Total Variation Distance (TVD).

Examples

data(example_scrna, package = "scSimEval")
calc_deconvolution_accuracy(example_scrna$ref, example_scrna$sim)

Description

Measures the difference in gene expression variance between true biological replicates and cell-shuffled pseudoreplicates (Squair et al., Nature Communications 2021). Positive delta variance indicates that biological replicates have higher inter-individual variability than randomly pooled pseudoreplicates.

Usage

calc_delta_variance(
  counts,
  replicates,
  conditions,
  cell_types = NULL,
  n_perm = 3
)

Arguments

Value

A named list:

Examples

data(example_scrna, package = "scSimEval")
calc_delta_variance(example_scrna$ref,
                    replicates = example_scrna$batch_info,
                    conditions = example_scrna$cell_types)

Description

Calculate Dunn Index

Usage

calc_dunn(dist_mat, cluster_labels)

Arguments

Value

Dunn index (ratio of smallest inter-cluster to largest intra-cluster distance).

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 10))
dist_mat <- stats::dist(data)
calc_dunn(dist_mat, cl)

Description

Integrated from countsimQC (Soneson & Robinson). Measures the normalized area between the eCDFs of reference and simulation across the shared support.

Usage

calc_ecdf_diffarea(ref, sim)

Arguments

Value

Normalized area between the two eCDFs.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_ecdf_diffarea(ref, sim)

Description

Integrated from HelenaLC/simulation-comparison.

Computes 2D bivariate density via MASS::kde2d over a shared bounding box, then measures optimal transport distance via emdist::emd2d.

Usage

calc_emd_2d(ref_mat, sim_mat, n = 25)

Arguments

Value

Normalized 2D Earth Mover's Distance.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_emd_2d(ref, sim)

Description

Calculates the posterior probability of zero counts being technical dropouts (excess zeros) versus biological sampling zeros under a Negative Binomial model.

Usage

calc_excess_zero_weights(counts)

Arguments

Value

A list with mean excess zero weight, gene-level weights, and estimated zero-inflation rate.

Examples

data(example_scrna, package = "scSimEval")
calc_excess_zero_weights(example_scrna$ref)

Description

Integrated from simpipe.

Usage

calc_fasano_franceschini(ref_mat, sim_mat, threads = 1)

Arguments

Value

Estimated 2D KS statistic.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_fasano_franceschini(ref, sim)

Description

Integrated from scCluBench (Xu et al., AAAI 2026). Computes the geometric mean of pairwise cluster precision and recall.

Usage

calc_fmi(pred, truth)

Arguments

Value

FMI score between 0 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_fmi(pred, truth)

Description

Computes the Fraction of Samples Closer Than The True Match (FOSCTTM) to quantify cell alignment error across single-cell multi-omics modalities in a shared latent or integrated space (Zhai et al., Genome Biology 2024; Liu et al., Nat Biotechnol 2023). A score of 0 represents perfect alignment (where the true paired cell is the nearest neighbor), while 0.5 corresponds to random chance.

Usage

calc_foscttm(x, y, metric = c("euclidean", "cosine"))

Arguments

Value

A list containing:

Examples

data(example_multiomics, package = "scSimEval")
r_rna <- example_multiomics$ref_multi$rna
r_atac <- example_multiomics$ref_multi$atac
calc_foscttm(r_rna, r_atac)

Description

Computes the single-cell analogue of Frechet Inception Distance (FID) on low-dimensional PCA embeddings between reference and simulated cell populations.

Usage

calc_frechet_singlecell_distance(
  ref_mat,
  sim_mat,
  cells_as_cols = TRUE,
  n_pcs = 15,
  max_features = 2000
)

Arguments

Value

A list with Frechet distance (FSD), mean discrepancy, and covariance trace discrepancy.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_frechet_singlecell_distance(ref, sim)

Description

Evaluates the realism (Precision) and coverage/diversity (Recall) of generated single-cell distributions relative to empirical reference data using k-NN hyperspheres.

Usage

calc_generative_precision_recall(
  ref_mat,
  sim_mat,
  cells_as_cols = TRUE,
  k = 5,
  max_cells = 500,
  max_features = 1000
)

Arguments

Value

A list containing Generative Precision, Generative Recall, and Generative F1.

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 10))
dist_mat <- stats::dist(data)
calc_generative_precision_recall(data, cl)

Description

Calculate Homogeneity Score

Usage

calc_homogeneity(pred, truth)

Arguments

Value

Homogeneity score between 0 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_homogeneity(pred, truth)

Description

Integrated from scCluBench (Xu et al., AAAI 2026).

Usage

calc_homogeneity_completeness_v_measure(pred, truth, beta = 1)

Arguments

Value

A named list containing homogeneity, completeness, and v_measure.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_homogeneity_completeness_v_measure(pred, truth)

Description

Computes the one-way random effects Intraclass Correlation Coefficient (ICC) per gene across donors or biological replicates to quantify hierarchical clustering.

Usage

calc_intraclass_correlation(counts, donor_labels, n_genes = 200)

Arguments

Value

A list with median ICC, mean ICC, and vector of per-gene ICC values.

Examples

data(example_scrna, package = "scSimEval")
calc_intraclass_correlation(example_scrna$ref, example_scrna$sim)

Description

Evaluates the diversity of batch labels within each cell's k-nearest neighborhood using the Inverse Simpson Index (1 / sum(p_b^2)). Implements a pure-R, lightweight formulation of LISI with optional distance weighting.

Usage

calc_isi(coords, batch_info, k = 30, weighted = TRUE)

Arguments

Value

A named list:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
calc_isi(coords, batch)

Description

Integrated from simpipe & SimBench.

Usage

calc_kde_test(ref_mat, sim_mat)

Arguments

Value

z-statistic testing discrepancy between bivariate kernel density estimates.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_kde_test(ref, sim)

Description

Decomposes gene expression variance / squared coefficient of variation (CV^2) into intrinsic transcriptional bursting noise and extrinsic cell-state noise.

Usage

calc_kinetic_noise_decomposition(counts, cell_states = NULL)

Arguments

Value

A list containing mean intrinsic noise, mean extrinsic noise, noise ratio, and gene-level vectors.

Examples

data(example_scrna, package = "scSimEval")
calc_kinetic_noise_decomposition(example_scrna$ref)

Description

Calculate Kolmogorov-Smirnov Distance (1D KS)

Usage

calc_ks(ref, sim)

Arguments

Value

Maximum vertical distance between empirical CDFs (between 0 and 1).

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_ks(ref, sim)

Description

Computes the Local Density Estimate (LDE) and Local Density Factor (LDF) using a Gaussian kernel over reachability distances in the k-NN neighborhood, adapted from Latecki et al. and CellMixS.

Usage

calc_ldf(coords, k = 15, h = 1, c = 1)

Arguments

Value

A named list:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
calc_ldf(coords)

Description

Quantifies cell-specific changes in the Local Density Factor (LDF) before and after data integration or between reference and simulated datasets.

Usage

calc_ldf_diff(coords_pre, coords_post, k = 15, h = 1, c = 1)

Arguments

Value

A named list:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
calc_ldf_diff(coords, coords)

Description

Direct port and generalization of scDesign3::perform_lrt (Song et al., Nat Biotechnol 2024). Performs the likelihood ratio test to compare two nested simulation models (e.g., cell-type/covariate model vs intercept-only null model, or spline trajectory vs linear model).

Usage

calc_likelihood_ratio_test(
  alter_model,
  null_model,
  df_alter = NULL,
  df_null = NULL
)

Arguments

Value

A data.frame containing:

Examples

calc_likelihood_ratio_test(stats::rnorm(50), stats::rnorm(50))

Description

Calculates the proportion of overlapping k-nearest neighbors between an original/reference embedding and an integrated/simulated embedding (adapted from Seurat LocalStruct and CellMixS locStructure).

Usage

calc_local_structure_metric(coords_pre, coords_post, k = 30)

Arguments

Value

A named list:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
calc_local_structure_metric(coords, coords)

Description

Calculate Median Absolute Deviation (MAD)

Usage

calc_mad(ref, sim, align = TRUE)

Arguments

Value

Numeric MAD value.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_mad(ref, sim)

Description

Calculate Mean Absolute Error (MAE)

Usage

calc_mae(ref, sim, align = TRUE)

Arguments

Value

Mean absolute difference.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_mae(ref, sim)

Description

Evaluates gene-wise and aggregate marginal goodness-of-fit (Poisson, Negative Binomial, or Gaussian) for simulated or fitted single-cell count matrices, computing total and mean AIC and BIC across genes (scDesign3; Song et al., 2024).

Usage

calc_marginal_aic_bic(
  counts,
  fitted_means,
  dispersions = NULL,
  distribution = c("poisson", "nb", "gaussian"),
  n_params_per_gene = NULL
)

Arguments

Value

A list containing:

Examples

calc_marginal_aic_bic(stats::rnorm(50), stats::rnorm(50))

Description

Evaluates the distributional discrepancy between empirical reference and generated/simulated single-cell high-dimensional profiles or latent embeddings.

Usage

calc_mmd(
  ref_mat,
  sim_mat,
  cells_as_cols = TRUE,
  sigma = NULL,
  max_cells = 500,
  max_features = 1000
)

Arguments

Value

A list with MMD, squared MMD, and the kernel bandwidth sigma used.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_mmd(ref, sim)

Description

Quantifies how well different single-cell omics layers mix in a unified latent embedding space without sacrificing biological clustering (Zhai et al., Genome Biology 2024; Luecken et al., Nat Methods 2022).

Usage

calc_modality_alignment(embedding, modalities, cell_types = NULL, k = 15)

Arguments

Value

A list containing modality_asw, modality_mixing_score, and mean_cross_modality_neighbor_frac.

Examples

data(example_multiomics, package = "scSimEval")
r_rna <- example_multiomics$ref_multi$rna
r_atac <- example_multiomics$ref_multi$atac
calc_modality_alignment(rbind(t(r_rna[seq_len(10),]), t(r_atac[seq_len(10),])), rep(c("RNA","ATAC"), each=ncol(r_rna)))

Description

Evaluates the statistical trade-off between model fit and parameter complexity for single-cell generative models as benchmarked in scDesign3 (Song et al., Nat Biotechnol 2024).

Usage

calc_model_aic_bic(loglik, n_params, n_obs)

Arguments

Value

A named vector or data.frame containing loglik, n_params, n_obs, AIC, and BIC.

Examples

calc_model_aic_bic(-100, 5, 100)

Description

Integrated from simPIC (Chugh et al., 2024; bluster::neighborPurity). For each cell, evaluates the proportion of its k-nearest neighbors that share the same cluster or cell-type identity.

Usage

calc_neighborhood_purity(data, cluster_labels, k = NULL, is_distance = FALSE)

Arguments

Value

A numeric vector of neighborhood purity scores (values from 0 to 1), with an attribute "mean_purity" containing the overall average.

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 10))
dist_mat <- stats::dist(data)
calc_neighborhood_purity(data, cl)

Description

Computes the Jaccard similarity index of top-k edge predictions between two networks or top correlated feature pairs between reference and simulated datasets.

Usage

calc_network_jaccard(edges1, edges2, k = NULL, directed = TRUE)

Arguments

Value

Jaccard similarity index in [0, 1] (|E1 cap E2| / |E1 cup E2|).

Examples

calc_network_jaccard(data.frame(from = c("g1", "g2"), to = c("g2", "g3")),
                     data.frame(from = c("g1", "g2"), to = c("g2", "g3")))

Description

Calculate Normalized Mutual Information (NMI)

Usage

calc_nmi(pred, truth)

Arguments

Value

NMI value between 0 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_nmi(pred, truth)

Description

Integrated from countsimQC (Soneson & Robinson). Evaluates whether the dataset label composition in k-NN neighborhoods departs significantly from the overall global dataset proportion using Chi-squared tests. Works for both 1D numeric vectors and 2D matrices.

Usage

calc_nn_mismatch(ref, sim, k = NULL, subsample_size = 300)

Arguments

Value

Fraction of points with significant neighbor composition mismatch (p <= 0.05).

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_nn_mismatch(ref, sim)

Description

Uses the standard Tukey 1.5 * IQR criterion on quantiles, as used in simpipe.

Usage

calc_outlier_proportion(x)

Arguments

Value

Outlier proportion (between 0 and 1).

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_outlier_proportion(ref)

Description

Calculate Distribution Overlapping Index (OV)

Usage

calc_overlap(ref, sim)

Arguments

Value

Area of overlap under probability densities (0 to 1).

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_overlap(ref, sim)

Description

Integrated from HelenaLC/simulation-comparison.

Usage

calc_peacock_2d(ref_mat, sim_mat)

Arguments

Value

Peacock test statistic.

Examples

ref <- matrix(stats::rnorm(100), 50, 2)
sim <- matrix(stats::rnorm(100), 50, 2)
calc_peacock_2d(ref, sim)

Description

Evaluates the preservation of chromatin peak-to-peak co-accessibility correlation matrices (cis-regulatory interactions) between reference and simulated scATAC-seq datasets.

Usage

calc_peak_coaccessibility_fidelity(
  ref_atac,
  sim_atac,
  top_n_peaks = 300,
  method = c("spearman", "pearson")
)

Arguments

Value

A list containing RV coefficient, matrix correlation, Frobenius distance, and MAE.

Examples

data(example_scrna, package = "scSimEval")
calc_peak_coaccessibility_fidelity(example_scrna$ref, example_scrna$sim)

Description

Evaluates whether the relative cell-to-cell ordering along a differentiation trajectory is preserved between reference and simulated data. Accepts either precomputed pseudotime vectors OR scRNA-seq expression matrices (which will be automatically inferred via PCA trajectory).

Usage

calc_pseudotime_correlation(
  ref_pseudotime,
  sim_pseudotime,
  method = c("spearman", "pearson")
)

Arguments

Value

Correlation coefficient between reference and simulated pseudotime trajectories.

Examples

data(example_scrna, package = "scSimEval")
calc_pseudotime_correlation(example_scrna$ref, example_scrna$sim)

Description

Calculate Root Mean Squared Error (RMSE)

Usage

calc_rmse(ref, sim, align = TRUE)

Arguments

Value

Quadratic error penalizing large discrepancies.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_rmse(ref, sim)

Description

Integrated from countsimQC (Soneson & Robinson). Tests whether values from the two datasets intermingle randomly when sorted. A significant left-sided result indicates that identical values cluster together into fewer runs than expected by chance, signaling distinct distributions.

Usage

calc_runs_test(ref, sim, alternative = c("left.sided", "two.sided"))

Arguments

Value

A named list with the runs test statistic and p-value.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_runs_test(ref, sim)

Description

Calculates the mixing metric originally proposed in Seurat (Stuart et al., Cell 2019) and adapted by CellMixS. For each cell, finds the rank of the k_pos-th neighbor from each batch in the sorted neighborhood, and computes the median rank across batches.

Usage

calc_seurat_mixing_metric(coords, batch_info, k = 300, k_pos = 5)

Arguments

Value

A named list:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
calc_seurat_mixing_metric(coords, batch)

Description

Detect Bimodal Distribution (BD) Genes (Bimodal Separation Index)

Usage

calc_signal_bd(exprs_mat, cell_types)

Arguments

Value

Numeric vector of Bimodality Index values.

Examples

data(example_scrna, package = "scSimEval")
calc_signal_bd(example_scrna$ref, example_scrna$cell_types)

Description

Detect Differential Distribution (DD) Genes (Kolmogorov-Smirnov Test)

Usage

calc_signal_dd(exprs_mat, cell_types)

Arguments

Value

Vector of adjusted p-values.

Examples

data(example_scrna, package = "scSimEval")
calc_signal_dd(example_scrna$ref, example_scrna$cell_types)

Description

Detect Differential Expression (DE) Genes (Mean Shift via Limma or T-Test)

Usage

calc_signal_de(exprs_mat, cell_types, p_sig = 0.05)

Arguments

Value

Vector of adjusted p-values.

Examples

data(example_scrna, package = "scSimEval")
calc_signal_de(example_scrna$ref, example_scrna$cell_types)

Description

Detect Differential Proportion / Zero-Inflation (DP) Genes (Chisq Test)

Usage

calc_signal_dp(exprs_mat, cell_types, threshold = 0)

Arguments

Value

Vector of adjusted p-values.

Examples

data(example_scrna, package = "scSimEval")
calc_signal_dp(example_scrna$ref, example_scrna$cell_types)

Description

Detect Differential Variability (DV) Genes (Bartlett Test)

Usage

calc_signal_dv(exprs_mat, cell_types)

Arguments

Value

Vector of adjusted p-values.

Examples

data(example_scrna, package = "scSimEval")
calc_signal_dv(example_scrna$ref, example_scrna$cell_types)

Description

Calculate Average Silhouette Width (ASW)

Usage

calc_silhouette(dist_mat, cluster_labels)

Arguments

Value

Mean silhouette width.

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 10))
dist_mat <- stats::dist(data)
calc_silhouette(dist_mat, cl)

Description

Computes the Root Mean Squared Error (RMSE) between sorted branch heights of hierarchical lineage trees. Accepts either precomputed hclust objects OR scRNA-seq count matrices + cell_types (which will be automatically inferred).

Usage

calc_tree_height_discrepancy(
  ref_tree,
  sim_tree,
  cell_types_ref = NULL,
  cell_types_sim = NULL
)

Arguments

Value

Root mean squared error between branch heights.

Examples

data(example_scrna, package = "scSimEval")
calc_tree_height_discrepancy(example_scrna$ref, example_scrna$sim,
                             cell_types_ref = example_scrna$cell_types,
                             cell_types_sim = example_scrna$cell_types)

Description

Calculate V-Measure

Usage

calc_v_measure(pred, truth, beta = 1)

Arguments

Value

V-measure score between 0 and 1.

Examples

pred <- factor(rep(c("A", "B"), each = 20))
truth <- factor(rep(c("A", "B"), each = 20))
calc_v_measure(pred, truth)

Description

Decomposes gene expression variance into biological cell-type heterogeneity, donor / biological replicate variation, and residual technical noise.

Usage

calc_variance_decomposition(
  counts,
  cell_metadata,
  donor_col = "donor",
  celltype_col = "cell_type",
  n_genes = 200
)

Arguments

Value

A list of mean percentage of variance explained by cell-type, donor, and residual noise.

Examples

data(example_scrna, package = "scSimEval")
md <- data.frame(donor = example_scrna$batch_info,
                 cell_type = example_scrna$cell_types)
calc_variance_decomposition(example_scrna$ref, md)

Description

Integrated from HelenaLC/simulation-comparison.

Usage

calc_wasserstein_1d(ref, sim, p = 1)

Arguments

Value

The 1D Wasserstein distance between the two empirical distributions.

Examples

ref <- stats::rnorm(50)
sim <- stats::rnorm(50)
calc_wasserstein_1d(ref, sim)

Description

Fits an empirical logistic dropout curve: logit(P(Y=0)) = beta_0 + beta_1 * log(mu) to model the dropout relationship with mean expression.

Usage

calc_zero_probability_curve(counts)

Arguments

Value

A list containing intercept, slope, midpoint (inflection point), and R-squared.

Examples

data(example_scrna, package = "scSimEval")
calc_zero_probability_curve(example_scrna$ref)

Description

Projects empirical biological reference and simulated count matrices into low-dimensional representations (UMAP, t-SNE, or PCA) for side-by-side and multi-method topological benchmarking. Automatically performs library-size normalization, variance-based feature selection, principal component analysis, and optional unsupervised clustering.

Usage

compute_dataset_embeddings(
  reference,
  simulated,
  reduction = c("umap", "tsne", "pca"),
  n_pcs = 30,
  perplexity = 30,
  n_neighbors = 15,
  min_dist = 0.3,
  seed = 42,
  cell_types = NULL,
  batch = NULL
)

Arguments

Value

A tidy data.frame containing cell coordinates (Dim1, Dim2), Dataset name, Dataset_Type ("Reference" vs "Simulated"), Cell_Type, Cluster, Library_Size, Detected_Features, and Batch.

Examples

data(example_scrna, package = "scSimEval")
emb <- compute_dataset_embeddings(
  reference = example_scrna$ref,
  simulated = list("Splatter" = example_scrna$sim),
  reduction = "umap",
  cell_types = example_scrna$cell_types
)
head(emb)

Description

Computes quantitative cluster separability (Silhouette width), unsupervised cluster recovery (Adjusted Rand Index), and distribution preservation metrics (Mean UMI, Mean detected features, sparsity percentage, and relative percentage error) between reference and simulated datasets.

Usage

compute_embedding_quality_metrics(
  embedding_data,
  metric_coords = c("Dim1", "Dim2")
)

Arguments

Value

A data.frame summarizing quantitative embedding quality metrics for each dataset.

Examples

data(example_scrna, package = "scSimEval")
emb <- compute_dataset_embeddings(
  reference = example_scrna$ref,
  simulated = list("Splatter" = example_scrna$sim),
  reduction = "umap",
  cell_types = example_scrna$cell_types
)
compute_embedding_quality_metrics(emb)

Description

Computes the overall performance leaderboard ranking simulation methods across all benchmark evaluation metrics. Metrics are first direction-inverted and min-max standardized into fidelity scores in [0, 1] (where 1.0 represents best observed performance). Simulators are then rank-ordered by mean overall fidelity score.

Usage

compute_method_leaderboard(benchmark_data)

Arguments

Value

A data.frame with columns:

• Overall_Rank: Integer rank (1 = top-performing simulator).

• Method: Simulator method name.

• Average_Fidelity: Formatted percentage string (e.g., "58.5%").

• Fidelity_Score: Numeric composite fidelity score in [0, 1] rounded to 4 decimals.

Examples

data(example_scrna, package = "scSimEval")
res <- evaluate_simulation_accuracy(example_scrna$ref, example_scrna$sim)
lb <- compute_method_leaderboard(res)

Description

Compares the non-linear relationship between peak mean accessibility and non-zero proportion (NZP) between reference and simulated scATAC-seq datasets.

Usage

evaluate_accessibility_sparsity_curve(ref_data, sim_data, poly_degree = 2)

Arguments

Value

A list of reference and simulated curve parameters, absolute discrepancies, and curve prediction RMSE.

Examples

data(example_scrna, package = "scSimEval")
evaluate_accessibility_sparsity_curve(example_scrna$ref, example_scrna$sim)

Description

Evaluates how realistically simulated technical batches mix or separate, computing kBET, LISI, Batch Silhouette, Shannon Entropy, PC Regression, CMS (Cell-Specific Mixing Score), Inverse Simpson Index (ISI), Seurat Mixing Metric, and optionally Local Density Differences (ldfDiff) and Local Structure Preservation.

Usage

evaluate_batch_metrics(
  data,
  batch_info,
  k = NULL,
  n_pcs = 30,
  pre_data = NULL,
  cell_types = NULL,
  verbose = FALSE
)

Arguments

Value

A named list of batch metrics:

Examples

data(example_scrna, package = "scSimEval")
evaluate_batch_metrics(example_scrna$ref, example_scrna$batch_info,
                       cell_types = example_scrna$cell_types)

Description

Evaluates both unsupervised cluster separation (ASW, Dunn, Connectivity, DB, CH, Neighborhood Purity, CDI) and supervised concordance against ground truth labels (Clustering Accuracy ACC, Hungarian F1/Precision/Recall, ARI, NMI, AMI, FMI, Homogeneity, Completeness, V-measure).

Usage

evaluate_clustering_metrics(
  data,
  cluster_info = NULL,
  pred_clusters = NULL,
  dist_mat = NULL,
  cell_types = NULL,
  ref_data = NULL
)

Arguments

Value

A named list of all clustering metrics.

Examples

data <- matrix(stats::rnorm(200), 20, 10)
cl <- factor(rep(c("A", "B"), each = 5))
dist_mat <- stats::dist(data)
evaluate_clustering_metrics(data, cl)

Description

Evaluates whether simulated or integrated multi-batch single-cell data preserve cell-type identity across distinct experimental batches or donors. Trains a k-NN classifier on each individual batch and tests prediction accuracy and macro F1 score on all other remaining batches.

Usage

evaluate_cross_batch_prediction(coords, cell_types, batch_info, k = 5)

Arguments

Value

A list containing:

Examples

coords <- matrix(stats::rnorm(100), 50, 2)
batch <- factor(rep(c("B1", "B2"), length.out = 50))
ct <- factor(rep(c("TypeA", "TypeB"), length.out = 50))
evaluate_cross_batch_prediction(coords, ct, batch)

Description

Trains a model using ONLY Modality 1 (e.g., scATAC-seq) and predicts cell-type identities defined in Modality 2 (e.g., scRNA-seq) to test cross-modal biological coherence.

Usage

evaluate_cross_modal_prediction(mod1_data, cell_types)

Arguments

Value

A list with cross-modal classification accuracy and macro F1 score.

Examples

data(example_multiomics, package = "scSimEval")
r_rna <- example_multiomics$ref_multi$rna
r_atac <- example_multiomics$ref_multi$atac
evaluate_cross_modal_prediction(r_rna, example_multiomics$cell_types)

Description

Evaluates the preservation of differentially expressed genes (DEGs) and biological signal between empirical reference and simulated datasets across three landmark single-cell benchmarking frameworks:

• Simpipe (Duo et al., 2024): True DEG ratio, Distribution score (p-value uniformity via Pearson Chi-Square goodness-of-fit test on remaining genes after DEG removal), and supervised machine learning classification (Accuracy, Precision, Recall, F1) using simulated DEGs to predict cell identity.

• SimBench (Cao et al., 2021): Symmetric Mean Absolute Percentage Error (SMAPE) on DEG proportions, SimBench DE fidelity score, log2 fold-change effect size Pearson and Spearman correlation, and top-N DEG Jaccard overlap.

• Shaky Foundations (Crowell et al., 2023): Group silhouette separation width, silhouette discrepancy, and Percent Variance Explained (PVE) by group assignment.

Usage

evaluate_deg_fidelity(
  ref_data,
  sim_data,
  ref_celltypes,
  sim_celltypes,
  group1 = NULL,
  group2 = NULL,
  fdr_cutoff = 0.05,
  logfc_cutoff = 0.5,
  top_n_de = 100,
  classifier = c("knn", "svm", "rf"),
  run_pvalue_uniformity = TRUE
)

Arguments

Value

A list containing:

• deg_summary_table: Comprehensive data.frame uniting all 15 metrics across Simpipe, SimBench, and Shaky Foundations.

• deg_genes_ref: Character vector of significant DEGs in reference.

• deg_genes_sim: Character vector of significant DEGs in simulation.

• contrast: String describing the evaluated group contrast.

• logfc_ref: Named numeric vector of reference log2 fold-changes.

• logfc_sim: Named numeric vector of simulated log2 fold-changes.

• ml_classification: Detailed classifier performance metrics.

• pvalue_uniformity: Chi-square goodness-of-fit test results.

Examples

data(example_scrna, package = "scSimEval")
evaluate_deg_fidelity(example_scrna$ref, example_scrna$sim,
                      ref_celltypes = example_scrna$cell_types,
                      sim_celltypes = example_scrna$cell_types)

Description

Trains a supervised classification model on reference single-cell chromatin accessibility profiles (scATAC-seq / scCAS) and evaluates cell-type prediction / projection accuracy on simulated cells against ground-truth cell-type labels (Chen et al., Nat Mach Intell 2022).

Usage

evaluate_epigenomic_annotation(
  ref_data,
  sim_data,
  ref_celltypes,
  sim_celltypes,
  method = c("knn", "centroid"),
  n_pcs = 20,
  k = 5
)

Arguments

Value

A list containing overall accuracy, balanced accuracy, macro F1, macro precision, macro recall, Cohen's kappa, and per-class performance metrics.

Examples

data(example_multiomics, package = "scSimEval")
evaluate_epigenomic_annotation(
  example_multiomics$ref_multi$atac,
  example_multiomics$sim_multi$atac,
  ref_celltypes = example_multiomics$cell_types,
  sim_celltypes = example_multiomics$cell_types
)

Description

Orchestrates comprehensive benchmarking across all 7 evaluation categories using strictly the user's provided inputs:

• ref_multi: List of reference omics matrices (e.g. list(rna = ..., atac = ...)).

• sim_multi: List of simulated omics matrices (e.g. list(rna = ..., atac = ...)).

• cell_types: Vector/factor of cell-type identities.

• batch_info: Vector/factor of batch or donor annotations.

• Computational resource usage: cpu_time, memory_mb, system_time.

• Trajectory features inferred directly from scRNA-seq counts.

Usage

evaluate_multiomics_accuracy(
  ref_multi,
  sim_multi,
  cell_types = NULL,
  batch_info = NULL,
  pairing = c("paired", "unpaired"),
  is_paired = NULL,
  memory_mb = NULL,
  elapsed_time = NULL,
  peak_memory_mb = NULL,
  feature_pairs = NULL,
  compute_bivariate = FALSE,
  threads = 1,
  verbose = TRUE
)

Arguments

Details

Multiomics Metric Measurement Capacity by Experimental Design & Metadata:

• Paired Multiomics (e.g. 10x Chromium Multiome, SHARE-seq, SNARE-seq):

  • Count Matrices Alone (RNA + ATAC): Measures approximately 48 metrics spanning both unimodal distribution layers (Categories I, II, VI, VIII across both modalities) plus Category VII Cross-Modal Coupling (FOSCTTM alignment score, Match@1 nearest neighbor accuracy, cross-modal generative prediction accuracy/F1, RV coefficient, and peak-to-gene regulatory correlation).

  • + Cell Types: Unlocks 15–20 additional metrics for cross-modal cell-type concordance and cell-type specific epigenetic promoter/enhancer activity.

  • + Batch Info: Unlocks 10–14 additional metrics evaluating technical confounder integration across both RNA and ATAC modalities.

  • + Both (Full Metadata): Unlocks the complete multiomics suite of all 62 metrics across all 8 categories.

• Unpaired Multiomics (independent scRNA and scATAC from separate cells):

  • Count Matrices Alone (RNA + ATAC): Measures approximately 38–42 metrics spanning unimodal distributions for both layers plus population-level cross-modal metrics (co-expression module fidelity, Network Jaccard similarity, cross-modality correlation).

  • + Cell Types (Crucial for Unpaired): Unlocks 15–20 additional metrics for cluster-matched label transfer concordance and cell-type matched regulatory linkage.

  • + Batch Info: Unlocks 7–12 additional metrics evaluating batch integration.

Value

A list containing detailed results and a unified tidy master summary table.

Examples

data(example_multiomics, package = "scSimEval")
evaluate_multiomics_accuracy(example_multiomics$ref_multi, example_multiomics$sim_multi)

Description

Orchestrates simultaneous benchmarking across a collection of single-cell datasets, including paired multiomics (e.g., scRNA-seq and scATAC-seq from the same biological experiment) and unimodal datasets (e.g., standalone scRNA-seq or scATAC-seq). Produces a unified, consolidated summary table, comparative ranking matrix, and dataset overview.

Usage

evaluate_multiple_datasets(
  datasets,
  pair_by_prefix = TRUE,
  compute_bivariate = FALSE,
  threads = 1,
  verbose = TRUE
)

Arguments

Value

An S3 object of class scSimEval_consolidated containing:

• consolidated_summary_table: Master tidy data.frame with columns Dataset, Data_Name, Modality, Category, Property, Metric, and Value.

• consolidated_score_matrix: Wide comparison matrix comparing all datasets across metrics.

• dataset_overview: Aggregate summary table with total metrics and mean scores per dataset.

• dataset_results: Named list containing detailed evaluation results for each individual dataset.

Examples

data(example_scrna, package = "scSimEval")
res <- evaluate_simulation_accuracy(example_scrna$ref, example_scrna$sim)

Description

Trains a classifier on simulated features (80% train / 20% test) to verify whether the simulated biological signals are sufficiently predictive of cell identities. If de_features is NULL, the top variable features are automatically selected.

Usage

evaluate_predictive_de_model(
  data,
  group,
  de_features = NULL,
  n_top = 50,
  method = c("knn", "rf", "svm")
)

Arguments

Value

A list containing Accuracy, Macro Precision, Macro Recall, and Macro F1 score.

Examples

data(example_scrna, package = "scSimEval")
evaluate_predictive_de_model(example_scrna$ref, group = example_scrna$cell_types)

Description

Compares proportions of genes exhibiting DE, DV, DD, DP, and BD between empirical reference and simulated datasets.

Usage

evaluate_simbench_signals(
  ref_mat,
  sim_mat,
  ref_celltypes,
  sim_celltypes,
  p_sig = 0.05,
  bi_cutoff = 0.3
)

Arguments

Value

A tidy data.frame comparing biological signal proportions.

Examples

data(example_scrna, package = "scSimEval")
evaluate_simbench_signals(example_scrna$ref, example_scrna$sim,
                          ref_celltypes = example_scrna$cell_types,
                          sim_celltypes = example_scrna$cell_types)

Description

Computes distribution distances, cellular and feature-level properties, zero-inflation curves, and manifold distances between reference and simulated datasets.

Usage

evaluate_simulation_accuracy(
  ref_data,
  sim_data,
  compute_bivariate = TRUE,
  threads = 1,
  memory_mb = NULL,
  elapsed_time = NULL,
  peak_memory_mb = NULL,
  verbose = TRUE
)

Arguments

Details

Metric Measurement Capacity by Data & Metadata Mode:

• Count Matrix Alone (No Metadata): Measures approximately 23 baseline metrics spanning Category I (Distributional Properties: KS, Wasserstein, MAD, MAE, RMSE, OV, Bhattacharyya, ECDF difference, Runs test, 2D Fasano-Franceschini, 2D Peacock, KDE test, MMD, Fréchet distance), Category II (Correlations & Zero-Inflation: BCV discrepancy, dropout curve midpoint/slope, mean-variance R², cell/gene Pearson correlations), and Category VI (Trajectory & Lineage Dynamics: pseudotime correlation, tree height RMSE).

• + Cell Types (Optional): Unlocks 12–15 additional metrics spanning Category III (Cellular Structure & Concordance: ARI, NMI, AMI, Silhouette width, Dunn, Davies-Bouldin, Calinski-Harabasz, V-measure, Neighborhood Purity, Generative Precision/Recall) and Category V (Biological Signal & Downstream Fidelity: SimBench DE/DP/DM/DB/DV signal fidelity, top DEG Jaccard overlap, Log2FC correlation, classifier accuracy and Macro-F1).

• + Batch Annotations (Optional): Unlocks 7 additional metrics spanning Category IV (Batch Effects & Confounder Mixing: Batch Silhouette width, local neighborhood Shannon Entropy, PCR R², CMS CellMixS, Inverse Simpson Index ISI, Seurat Mixing Metric, LDF difference).

• Full Metadata (Count Matrix + Cell Types + Batch): Unlocks the complete unimodal benchmarking suite of up to 55 metrics.

Value

A list containing tidy summary tables and detailed metric lists.

Examples

data(example_scrna, package = "scSimEval")
res <- evaluate_simulation_accuracy(example_scrna$ref, example_scrna$sim)

Description

Evaluates trajectory preservation using pseudotime correlation, branch height discrepancy, and univariate distribution distance metrics on pseudotime. If raw count matrices are provided, pseudotime and lineage trees are automatically inferred directly from the scRNA-seq data.

Usage

evaluate_trajectory_metrics(
  ref_data,
  sim_data,
  cell_types_ref = NULL,
  cell_types_sim = NULL
)

Arguments

Value

A named list of trajectory accuracy metrics.

Examples

data(example_scrna, package = "scSimEval")
evaluate_trajectory_metrics(example_scrna$ref, example_scrna$sim)

Description

Compares empirical dropout trends between reference and simulated datasets.

Usage

evaluate_zero_probability_curve(ref_counts, sim_counts)

Arguments

Value

Named numeric vector of curve parameter differences.

Examples

data(example_scrna, package = "scSimEval")
evaluate_zero_probability_curve(example_scrna$ref, example_scrna$sim)

Description

A named factor vector containing technical batch annotations ("Batch1", "Batch2") for 80 single cells, matching the column names of example_scrna and example_scatac. Useful as a reference template for formatting batch metadata inputs in scSimEval.

Usage

data(example_batch_info)

Format

A named factor with 80 elements across 2 levels: "Batch1" (40 cells) and "Batch2" (40 cells). Names correspond to individual cell barcodes ("Cell_01" to "Cell_80").

Source

Derived from example_scrna$batch_info.

Examples

data(example_batch_info)
head(example_batch_info)
table(example_batch_info)

Description

A named factor vector containing discrete cell-type labels ("TypeA", "TypeB") for 80 single cells, matching the column names of example_scrna and example_scatac. Useful as a reference template for formatting cell-type metadata inputs in scSimEval.

Usage

data(example_cell_types)

Format

A named factor with 80 elements across 2 levels: "TypeA" (40 cells) and "TypeB" (40 cells). Names correspond to individual cell barcodes ("Cell_01" to "Cell_80").

Source

Derived from example_scrna$cell_types.

Examples

data(example_cell_types)
head(example_cell_types)
table(example_cell_types)

Description

A unified single-cell multiomics dataset pairing scRNA-seq and scATAC-seq profiles across the same 80 cells, accompanied by cell-type labels, batch annotations, and logged computational scalability metrics.

Usage

data(example_multiomics)

Format

A list with 5 elements:

ref_multi

Named list of reference matrices: rna (60 x 80) and atac (60 x 80).

sim_multi

Named list of simulated matrices: rna (60 x 80) and atac (60 x 80).

cell_types

A factor of cell-type identities ("TypeA", "TypeB") for the 80 cells.

batch_info

A factor of batch annotations ("Batch1", "Batch2") for the 80 cells.

resource_stats

A list of computational scalability metrics: cpu_time, memory_mb, system_time, elapsed_time.

Source

Synthetic multiomics benchmark data generated for testing scSimEval.

Examples

data(example_multiomics)
names(example_multiomics$ref_multi)
names(example_multiomics$sim_multi)

Description

A synthetic single-cell ATAC sequencing peak accessibility dataset designed for testing, benchmarking, and demonstrating evaluation metrics in scSimEval. Contains paired empirical reference accessibility counts and simulated accessibility counts across 60 peaks and 80 cells matching the cells in example_scrna.

Usage

data(example_scatac)

Format

A list with 4 elements:

ref

A numeric matrix of reference peak accessibility counts (60 peaks x 80 cells).

sim

A numeric matrix of simulated peak accessibility counts (60 peaks x 80 cells).

cell_types

A factor of cell-type identities ("TypeA", "TypeB") for the 80 cells.

batch_info

A factor of batch annotations ("Batch1", "Batch2") for the 80 cells.

Source

Synthetic benchmark data generated via binomial sampling with cell-type specific accessibility shifts.

Examples

data(example_scatac)
dim(example_scatac$ref)
dim(example_scatac$sim)
table(example_scatac$batch_info)

Description

A synthetic single-cell RNA sequencing dataset designed for testing, benchmarking, and demonstrating evaluation metrics in scSimEval. Contains paired empirical reference counts and simulated counts across 60 genes and 80 cells from two cell types and two batches.

Usage

data(example_scrna)

Format

A list with 4 elements:

ref

A numeric matrix of reference count data (60 genes x 80 cells).

sim

A numeric matrix of simulated count data (60 genes x 80 cells).

cell_types

A factor of cell-type identities ("TypeA", "TypeB") for the 80 cells.

batch_info

A factor of batch annotations ("Batch1", "Batch2") for the 80 cells.

Source

Synthetic benchmark data generated via negative binomial sampling with cell-type specific expression shifts.

Examples

data(example_scrna)
dim(example_scrna$ref)
dim(example_scrna$sim)
table(example_scrna$cell_types)

Description

Computes library size, log-library size, zero fraction / detection frequency, pairwise cell correlation (on top HVGs), cell k-NN hubness, pairwise PCA distance, and outlier proportions.

Usage

extract_cell_properties(
  data,
  max_cells_cor = 1000,
  n_top_hvgs = 500,
  verbose = FALSE
)

Arguments

Value

A named list of cell-level summary vectors.

Examples

data(example_scrna, package = "scSimEval")
extract_cell_properties(example_scrna$ref)

Description

Computes key dimensional and biological properties from single-cell or multiomics count matrices, including cell counts, feature counts, sparsity percentage, biological group/cell type counts, batch counts, median library size, median detected features per cell, and mean expression level.

Usage

extract_dataset_summary(
  mat,
  role = "Reference",
  method_name = "Dataset",
  modality = "scRNA-seq",
  cell_types = NULL,
  batch_info = NULL
)

Arguments

Value

A data.frame with 1 row summarizing the dataset properties.

Examples

data(example_scrna, package = "scSimEval")
extract_dataset_summary(example_scrna$ref, role = "Reference", method_name = "Empirical")

Description

Computes mean abundance, standard deviation, variance, coefficient of variation (CV), feature dropout (zero fraction), dispersion, Biological Coefficient of Variation (BCV), gene-gene correlations, and outlier proportions.

Usage

extract_feature_properties(data, n_top_cor = 400, verbose = FALSE)

Arguments

Value

A named list of feature-level summary vectors.

Examples

data(example_scrna, package = "scSimEval")
extract_feature_properties(example_scrna$ref)

Description

Fast Column-Wise Variances for Dense or Sparse Single-Cell Matrices

Usage

fast_col_vars(mat)

Arguments

Value

Numeric vector of column variances.

Description

Computes ||X_i - Y_j||^2 using BLAS matrix product: ||x||^2 + ||y||^2 - 2 <x, y>.

Usage

fast_cross_distance_sq(X, Y)

Arguments

Value

Matrix of squared Euclidean distances (m x n).

Description

Fast Truncated PCA on Top Highly Variable Features

Usage

fast_pca(mat, n_pcs = 20, n_hvg = 2000, scale = FALSE)

Arguments

Value

Matrix of cell principal component embeddings (cells x n_pcs).

Description

Fast Row-Wise Variances for Dense or Sparse Single-Cell Matrices

Usage

fast_row_vars(mat)

Arguments

Value

Numeric vector of row variances.

Description

Fast Zero Fraction per Cell (Column-wise)

Usage

fast_sparse_zero_fraction_cell(mat)

Arguments

Value

Numeric vector of zero fraction per cell.

Description

Fast Zero Fraction per Feature (Row-wise)

Usage

fast_sparse_zero_fraction_feature(mat)

Arguments

Value

Numeric vector of zero fraction per feature.

Description

Builds a hierarchical clustering lineage tree between cell-type centroids from scRNA-seq expression data.

Usage

infer_scrna_lineage_tree(data, cell_types, method = "ward.D2")

Arguments

Value

An hclust tree object representing the cell-type lineage tree.

Examples

data(example_scrna, package = "scSimEval")
infer_scrna_lineage_tree(example_scrna$ref, example_scrna$cell_types)

Description

Estimates cell differentiation pseudotime directly from single-cell expression counts using diffusion/principal curve projection along the first principal component of top highly variable genes.

Usage

infer_scrna_pseudotime(data, n_top = 500)

Arguments

Value

Numeric vector of pseudotime values in [0, 1] for each cell.

Examples

data(example_scrna, package = "scSimEval")
infer_scrna_pseudotime(example_scrna$ref)

Description

Launches the Shiny web application embedded in the scSimEval package. Provides an intuitive graphical interface to upload biological reference and simulated datasets (single-cell scRNA-seq, scATAC-seq, or paired multiomics), specify computational scalability metrics (elapsed time and peak memory), interactively inspect the flagship 62-measure comparative bubble matrix, explore diagnostic figures, and export high-resolution figures (600 DPI publication quality), multi-page PDF reports, Excel (.xlsx) workbooks, and complete results ZIP packages.

Usage

launch_scSimEval_app(
  port = NULL,
  host = "127.0.0.1",
  launch.browser = interactive(),
  maxRequestSize = 10 * 1024^3
)

Arguments

Value

Invisibly returns the Shiny app process object.

Examples

# Locate the embedded Shiny app directory bundled with the package
system.file("shiny", "scSimEvalApp", package = "scSimEval")

# Launch the interactive studio (interactive sessions only)
if (interactive()) {
  launch_scSimEval_app()
}

Description

Produces a multi-dimensional benchmarking bubble matrix comparing simulation methods. Methods appear as rows; evaluation metrics appear as columns grouped under the eight canonical evaluation categories (I-VIII) displayed as colored header strips at the top.

Usage

plot_benchmark_bubble_matrix(
  data,
  method_classes = NULL,
  category_colors = NULL,
  metrics_order = NULL,
  methods_order = NULL,
  rank_methods = TRUE,
  compact_strips = TRUE,
  normalize_scores = TRUE,
  show_missing_dots = TRUE,
  bubble_size_range = c(2, 7.8),
  title = NULL,
  subtitle = NULL,
  base_size = 11,
  show_score_labels = FALSE
)

plot_bubble_matrix(
  data,
  method_classes = NULL,
  category_colors = NULL,
  metrics_order = NULL,
  methods_order = NULL,
  rank_methods = TRUE,
  compact_strips = TRUE,
  normalize_scores = TRUE,
  show_missing_dots = TRUE,
  bubble_size_range = c(2, 7.8),
  title = NULL,
  subtitle = NULL,
  base_size = 11,
  show_score_labels = FALSE
)

Arguments

Details

Bubble encoding:

• Size: Normalized fidelity score [0, 1] – larger = better performance.

• Fill color: Evaluation category (one of the eight canonical groups).

• Circle (shape 21): Regular performance (score < 0.96).

• Square (shape 22): Top performer (score >= 0.96).

• Small grey dot: Metric not available/computed for that method.

Score normalization is direction-aware: distance/error metrics are inverted so that 1 always means best performance, while concordance/correlation metrics are used directly (higher = better).

Value

A ggplot object rendering the multi-dimensional bubble matrix.

Examples

# Tidy benchmark data frame of standardized scores
bm_df <- data.frame(
  Method   = rep(c("Splat", "scDesign3", "SymSim", "dyngen"), each = 4),
  Category = rep(c("Accuracy", "Accuracy", "Cellular Structure", "Scalability"), 4),
  Metric   = rep(c("KS Distance", "Wasserstein", "Silhouette ASW", "CPU Time"), 4),
  Score    = c(0.8, 0.7, 0.9, 0.6, 0.5, 0.8, 0.6, 0.5, 0.8, 0.7, 0.6, 0.5, 0.7, 0.6, 0.9, 0.5)
)
p <- plot_benchmark_bubble_matrix(bm_df)
p

# Load benchmark summary across simulators
demo_file <- system.file("shiny/scSimEvalApp/data/demo_benchmark_data.rds", package = "scSimEval")
if (file.exists(demo_file)) {
  demo <- readRDS(demo_file)
  p <- plot_benchmark_bubble_matrix(
    data = demo$benchmark_summary_table,
    title = "Single-Cell Multiomics Simulation Benchmark",
    method_classes = list(
      "scRNA-seq"  = c("Splatter", "SymSim"),
      "scATAC-seq" = c("simATAC", "SCRIP"),
      "Multiomics" = c("scDesign3", "dyngen")
    )
  )
  print(p)
}

Description

Generates publication-ready faceted barplots comparing simulator methods across all individual evaluation metrics belonging to a specified benchmark category. Each metric is presented in its own sub-panel with exact numeric score labels, direction-awareness indicators (indicating whether higher or lower values are optimal), and standardized or original raw score formulation.

Usage

plot_category_metric_bars(
  benchmark_data,
  category,
  score_type = c("normalized", "raw"),
  palette = NULL,
  ncol = NULL,
  base_size = 11
)

Arguments