---
title: "Comprehensive Catalog of the 62 Evaluation Metrics"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Comprehensive Catalog of the 62 Evaluation Metrics}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r knitr-setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# Overview of the Evaluation Architecture

**`scSimEval`** evaluates single-cell simulation methods using an objective, ground-truth-free framework. It includes **62 distinct evaluation measures** organized into **8 core biological and computational categories**.

Every metric operates directly on the real biological reference data and the synthetic data generated by single-cell and multiomics simulators (scRNA-seq, scATAC-seq, and paired/unpaired co-assays). All 62 metrics in this catalog match identically with the metric identifiers and labels evaluated within the package functions, exported tables, and the interactive **scSimEval Studio** Shiny application.

---

# Summary Table of the 8 Categories

| Category | Domain | Number of Metrics | Primary Focus |
|:---:|:---|:---:|:---|
| **I** | **Distributional Properties** | **14** | Statistical distance between real and simulated single-cell count distributions and manifolds |
| **II** | **Correlations & Zero-Inflation** | **6** | Biological overdispersion (BCV), dropout curves, cell correlations, and gene co-expression |
| **III** | **Cellular Structure & Concordance** | **10** | Unsupervised cluster separation (ASW, Dunn, DB, CH) and supervised cell type preservation (ARI, NMI, AMI, V-measure, purity, precision) |
| **IV** | **Batch Effects & Confounder Mixing** | **7** | Technical batch mixing (CellMixS, LISI, Shannon entropy, PCR $R^2$, Seurat mixing, batch ASW, and ldfDiff) |
| **V** | **Biological Signal & Downstream Fidelity** | **15** | Differentially expressed gene (DEG) preservation, fold-change correlation, null uniformity, classifier accuracy, and Shaky Foundations variance |
| **VI** | **Trajectory & Lineage Dynamics** | **2** | Inferred developmental pseudotime rank correlation and cell lineage branching tree topology |
| **VII** | **Cross-Modal Coupling & Modularity** | **6** | Paired and joint ATAC-RNA regulatory links, FOSCTTM, Match@1, label transfer, and $R_V$ coefficient |
| **VIII** | **Computational Scalability** | **2** | Execution runtime in seconds and peak RAM memory allocation in MiB |
| **Total** | | **62** | Complete single-cell and multiomics simulation evaluation suite |

---

# Complete Master Reference Table (62 Metrics)

Below is the definitive cross-reference table mapping each metric's internal identifier, user-facing display label in the Shiny app and figures, category, evaluation direction, and scope:

| # | Metric ID | Shiny App Display Label | Category | Direction | Description |
|:--:|:---|:---|:---|:---:|:---|
| 1 | `KS` | KS Distance | (I) Distributional Properties | Lower is better | Kolmogorov-Smirnov supremum distance between cumulative distributions |
| 2 | `Wasserstein` | Wasserstein Dist. | (I) Distributional Properties | Lower is better | 1D Earth Mover's Distance measuring work required to transform distributions |
| 3 | `MAD` | MAD | (I) Distributional Properties | Lower is better | Median Absolute Deviation discrepancy across cell and gene properties |
| 4 | `MAE` | MAE | (I) Distributional Properties | Lower is better | Mean Absolute Error across summary statistics |
| 5 | `RMSE` | RMSE | (I) Distributional Properties | Lower is better | Root Mean Squared Error penalizing large deviations in moment statistics |
| 6 | `OV` | Overlap Coeff. | (I) Distributional Properties | Higher is better | Kernel density curve area intersection between reference and simulated distributions |
| 7 | `Bhattacharyya` | Bhattacharyya Dist. | (I) Distributional Properties | Lower is better | Statistical divergence between discrete count probability distributions |
| 8 | `ECDF_DiffArea` | ECDF Diff. Area | (I) Distributional Properties | Lower is better | Integrated area difference between empirical cumulative distribution functions |
| 9 | `Runs_Statistic` | Runs Statistic | (I) Distributional Properties | Lower is better | Wald-Wolfowitz runs test absolute $z$-score testing random interleaving |
| 10 | `Fasano_Franceschini_2D_KS` | Fasano-Franceschini 2D | (I) Distributional Properties | Lower is better | 2D extension of the KS statistic for bivariate feature distributions |
| 11 | `Peacock_2D_KS` | Peacock 2D | (I) Distributional Properties | Lower is better | 2D Peacock bivariate distribution test statistic |
| 12 | `KDE_Bivariate_zstat` | 2D KDE z-stat | (I) Distributional Properties | Lower is better | 2D Kernel Density Estimation divergence $z$-statistic between joint distributions |
| 13 | `MMD` | MMD | (I) Distributional Properties | Lower is better | Maximum Mean Discrepancy testing multivariate single-cell manifolds in RKHS |
| 14 | `Frechet_SingleCell_Distance` | Frechet SC Dist. | (I) Distributional Properties | Lower is better | Fréchet single-cell distance between multivariate Gaussian manifolds in PC space |
| 15 | `bcv_discrepancy` | Biological CV (BCV) | (II) Correlations & Zero-Inflation | Lower is better | Biological Coefficient of Variation discrepancy after Poisson noise subtraction |
| 16 | `dropout_curve_midpoint_x0` | Dropout Midpoint (x0) | (II) Correlations & Zero-Inflation | Lower is better | Logistic dropout curve midpoint ($x_0$) estimation error |
| 17 | `dropout_curve_slope_k` | Dropout Slope (k) | (II) Correlations & Zero-Inflation | Lower is better | Logistic dropout curve slope ($k$) steepness error |
| 18 | `mean_variance_r2` | Mean-Variance R2 | (II) Correlations & Zero-Inflation | Higher is better | Mean-variance trend fit ($R^2$) and dropout model goodness-of-fit |
| 19 | `cell_pearson_cor` | Cell Pearson Corr. | (II) Correlations & Zero-Inflation | Higher is better | Cell-to-cell Pearson correlation matrix concordance |
| 20 | `feature_pearson_cor` | Feature Pearson Corr. | (II) Correlations & Zero-Inflation | Higher is better | Gene-to-gene Pearson co-expression network correlation concordance |
| 21 | `silhouette_sim` | Silhouette ASW | (III) Cellular Structure & Concordance | Higher is better | Average Silhouette Width (ASW) cluster separation fidelity |
| 22 | `dunn_sim` | Dunn Index | (III) Cellular Structure & Concordance | Higher is better | Dunn index measuring ratio of inter-cluster separation to cluster diameter |
| 23 | `davies_bouldin_sim` | Davies-Bouldin | (III) Cellular Structure & Concordance | Lower is better | Davies-Bouldin compactness and separation discrepancy |
| 24 | `calinski_harabasz_sim` | Calinski-Harabasz | (III) Cellular Structure & Concordance | Higher is better | Calinski-Harabasz variance ratio criterion between and within clusters |
| 25 | `ari` | ARI | (III) Cellular Structure & Concordance | Higher is better | Adjusted Rand Index comparing unsupervised clusters against cell type labels |
| 26 | `nmi` | NMI | (III) Cellular Structure & Concordance | Higher is better | Normalized Mutual Information between unsupervised clusters and known cell types |
| 27 | `ami` | AMI | (III) Cellular Structure & Concordance | Higher is better | Adjusted Mutual Information adjusted for chance cluster overlap |
| 28 | `v_measure` | V-Measure | (III) Cellular Structure & Concordance | Higher is better | Harmonic mean of cluster homogeneity and completeness |
| 29 | `neighborhood_purity` | Neighborhood Purity | (III) Cellular Structure & Concordance | Higher is better | Proportion of $k$-nearest neighbors sharing identical annotated cell type labels |
| 30 | `Generative_Precision` | Generative Precision | (III) Cellular Structure & Concordance | Higher is better | Hungarian optimal matching accuracy between simulated and real cell partitions |
| 31 | `batch_silhouette` | Batch Silhouette | (IV) Batch Effects & Confounder Mixing | Higher is better | Average Silhouette Width for batch confounders (lower batch clustering is better) |
| 32 | `shannon_entropy` | Shannon Entropy | (IV) Batch Effects & Confounder Mixing | Higher is better | Batch Shannon diversity entropy across local cell neighborhoods |
| 33 | `pcr_r2` | PC Regression R2 | (IV) Batch Effects & Confounder Mixing | Lower is better | Principal Component Regression variance explained by technical batch labels ($R^2_{\text{batch}}$) |
| 34 | `cms` | CMS Score | (IV) Batch Effects & Confounder Mixing | Higher is better | CellMixS cell-specific mixing score testing random batch distributions |
| 35 | `isi` | LISI / ISI | (IV) Batch Effects & Confounder Mixing | Higher is better | Local Inverse Simpson Index measuring effective batches per neighborhood |
| 36 | `seurat_mixing_metric` | Seurat Mixing | (IV) Batch Effects & Confounder Mixing | Lower is better | Seurat batch mixing metric measuring average rank of same-batch neighbors |
| 37 | `ldf_diff` | ldfDiff | (IV) Batch Effects & Confounder Mixing | Lower is better | Local Density Factor discrepancy measuring manifold density distortion |
| 38 | `SimBench_SMAPE` | SimBench SMAPE | (V) Biological Signal & Downstream Fidelity | Lower is better | Symmetric Mean Absolute Percentage Error in differentially expressed log2FC |
| 39 | `SimBench_DE_Fidelity_Score` | DE Fidelity (1-SMAPE) | (V) Biological Signal & Downstream Fidelity | Higher is better | SimBench composite differential expression fidelity score ($1 - \text{SMAPE}$) |
| 40 | `Log2FC_Pearson_Corr` | Log2FC Pearson Corr | (V) Biological Signal & Downstream Fidelity | Higher is better | Pearson correlation of estimated log2 fold-changes across marker genes |
| 41 | `Log2FC_Spearman_Corr` | Log2FC Spearman Corr | (V) Biological Signal & Downstream Fidelity | Higher is better | Spearman rank correlation of estimated log2 fold-changes |
| 42 | `Top_DEG_Jaccard_Overlap` | Top DEG Jaccard | (V) Biological Signal & Downstream Fidelity | Higher is better | Jaccard index overlap of top differentially expressed genes between real and sim |
| 43 | `DEG_Ratio` | DEG Ratio (Sim/Real) | (V) Biological Signal & Downstream Fidelity | Matches 1.0 | Ratio of identified DEG count in simulated data vs real reference data |
| 44 | `PValue_Uniformity_Chisq` | P-Val Unif. (Chisq) | (V) Biological Signal & Downstream Fidelity | Lower is better | $\chi^2$ divergence test verifying uniform p-value distribution under the null hypothesis |
| 45 | `Distribution_Score` | Distribution Score | (V) Biological Signal & Downstream Fidelity | Higher is better | Composite null p-value distribution preservation score without false positives |
| 46 | `Classifier_Accuracy` | Classifier Accuracy | (V) Biological Signal & Downstream Fidelity | Higher is better | Random forest / SVM cell type classification accuracy on marker transcripts |
| 47 | `Classifier_Macro_F1` | Classifier Macro-F1 | (V) Biological Signal & Downstream Fidelity | Higher is better | Macro-averaged F1 classification score across rare and common cell populations |
| 48 | `Classifier_Macro_Recall` | Classifier Macro-Recall | (V) Biological Signal & Downstream Fidelity | Higher is better | Macro-averaged recall score across biological cell identities |
| 49 | `Silhouette_Sim` | Group Silhouette | (V) Biological Signal & Downstream Fidelity | Higher is better | Shaky Foundations cell-type silhouette similarity in DEG marker space |
| 50 | `Silhouette_Discrepancy` | Silhouette Discrepancy | (V) Biological Signal & Downstream Fidelity | Lower is better | Shaky Foundations absolute silhouette width discrepancy relative to reference |
| 51 | `PVE_Group_Sim` | Group PVE | (V) Biological Signal & Downstream Fidelity | Higher is better | Shaky Foundations Proportion of Variance Explained (PVE) by cell groups |
| 52 | `PVE_Discrepancy` | Group PVE Discrepancy | (V) Biological Signal & Downstream Fidelity | Lower is better | Discrepancy in proportion of variance explained by biological condition |
| 53 | `pseudotime_correlation` | Pseudotime Corr. (rho) | (VI) Trajectory & Lineage Dynamics | Higher is better | Spearman rank correlation of inferred developmental pseudotime ordering |
| 54 | `tree_height_rmse` | Lineage Tree RMSE | (VI) Trajectory & Lineage Dynamics | Lower is better | Root Mean Squared Error of lineage branching hierarchy tree heights |
| 55 | `cross_modal_accuracy` | Cross-Modal Accuracy | (VII) Cross-Modal Coupling & Modularity | Higher is better | Cross-modal cell type label transfer accuracy (RNA $\rightarrow$ ATAC) |
| 56 | `cross_modal_F1` | Cross-Modal F1 | (VII) Cross-Modal Coupling & Modularity | Higher is better | Macro-averaged F1 score of cross-modal cell type label transfer |
| 57 | `FOSCTTM` | FOSCTTM | (VII) Cross-Modal Coupling & Modularity | Lower is better | Fraction of Samples Closer Than The True Match in joint multi-omics space |
| 58 | `match_at_1` | Match@1 | (VII) Cross-Modal Coupling & Modularity | Higher is better | Top-1 cross-modal nearest neighbor exact cell pairing rate |
| 59 | `rv_coefficient` | RV Coefficient | (VII) Cross-Modal Coupling & Modularity | Higher is better | Vector correlation coefficient measuring chromatin co-accessibility and expression coupling |
| 60 | `module_correlation_r` | Module Corr. (r) | (VII) Cross-Modal Coupling & Modularity | Higher is better | Peak-to-gene linkage and co-regulatory module correlation concordance |
| 61 | `elapsed_time_seconds` | Elapsed Time (s) | (VIII) Computational Scalability | Lower is better | Wall-clock execution runtime in seconds required for model training and synthesis |
| 62 | `peak_memory_mb` | Peak RAM (MiB) | (VIII) Computational Scalability | Lower is better | Maximum resident set size / peak memory allocated during execution |

---

# Category 1: Distribution Properties (14 Metrics)

Evaluates whether single-cell count distributions (library size, mean expression, gene variance, and zero fraction) and higher-dimensional data manifolds in simulated data match real experimental data.

### 1. Kolmogorov-Smirnov (KS) Distance (`KS`)
* **Direction:** Lower is better (inverted during score normalization).
* **Formula:** $D_{KS} = \sup_x |F_{ref}(x) - F_{sim}(x)|$
* **Biological Meaning:** Measures the maximum vertical distance between the empirical cumulative distribution functions of real reference and simulated properties. A value of 0 indicates identical distributions.

### 2. Wasserstein Metric (Earth Mover's Distance 1D) (`Wasserstein`)
* **Direction:** Lower is better.
* **Formula:** $W_1(P, Q) = \int_{-\infty}^{\infty} |F_{ref}(x) - F_{sim}(x)| \, dx$
* **Biological Meaning:** Represents the minimal cost required to transform the simulated distribution into the reference distribution, integrating differences across the full range of counts.

### 3. Median Absolute Deviation (`MAD`)
* **Direction:** Lower is better ($|\text{MAD}_{ref} - \text{MAD}_{sim}|$).
* **Formula:** $\text{MAD} = \text{median}(|x - \text{median}(x)|)$
* **Biological Meaning:** Robust non-parametric estimate of count spread that resists distortion by extreme sequencing outliers.

### 4. Mean Absolute Error (`MAE`)
* **Direction:** Lower is better.
* **Formula:** $\text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - y_i|$
* **Biological Meaning:** Average absolute deviation between cell-level and gene-level summary statistics.

### 5. Root Mean Squared Error (`RMSE`)
* **Direction:** Lower is better.
* **Formula:** $\text{RMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^n (x_i - y_i)^2}$
* **Biological Meaning:** Standard quadratic scoring rule penalizing large discrepancies between real and simulated statistical moments.

### 6. Density Curve Overlap (`OV`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $\text{Overlap} = \int \min(f_{ref}(x), f_{sim}(x)) \, dx$
* **Biological Meaning:** Shared area under the kernel density estimates of real and simulated data. A value of $1.0$ represents identical density profiles.

### 7. Bhattacharyya Distance (`Bhattacharyya`)
* **Direction:** Lower is better.
* **Formula:** $D_B(P, Q) = -\ln(BC(P, Q))$, where $BC(P, Q) = \sum \sqrt{p(x) q(x)}$
* **Biological Meaning:** Measures statistical overlap and divergence between discrete single-cell count probability distributions.

### 8. Empirical Cumulative Distribution Area Difference (`ECDF_DiffArea`)
* **Direction:** Lower is better.
* **Formula:** $\Delta_{\text{ECDF}} = \int |F_{ref}(x) - F_{sim}(x)| \, dx$
* **Biological Meaning:** Directly integrates the area enclosed between real and simulated empirical cumulative distribution functions.

### 9. Wald-Wolfowitz Runs Test Statistic (`Runs_Statistic`)
* **Direction:** Lower is better ($|Z_{\text{runs}}|$, where values near 0 indicate identical orderings).
* **Biological Meaning:** Non-parametric test checking whether interleaved reference and simulated observations mix randomly, confirming they originate from the same continuous distribution.

### 10. 2D Fasano-Franceschini Test Statistic (`Fasano_Franceschini_2D_KS`)
* **Direction:** Lower is better.
* **Biological Meaning:** Computationally efficient bivariate generalization of the Kolmogorov-Smirnov test over quadrants, evaluating paired properties (e.g. library size vs. cell zero fraction).

### 11. 2D Peacock Bivariate Test Statistic (`Peacock_2D_KS`)
* **Direction:** Lower is better.
* **Biological Meaning:** Rigorous two-dimensional Kolmogorov-Smirnov test measuring supremum divergence across all four quadrants in bivariate planes (e.g. mean vs. variance).

### 12. 2D Bivariate Kernel Density Estimation z-Statistic (`KDE_Bivariate_zstat`)
* **Direction:** Lower is better.
* **Biological Meaning:** Non-parametric two-dimensional kernel density divergence statistic assessing the global shape and contours of paired feature distributions.

### 13. Maximum Mean Discrepancy (`MMD`)
* **Direction:** Lower is better.
* **Formula:** $\text{MMD}^2(P, Q) = \mathbb{E}[k(x, x')] - 2\mathbb{E}[k(x, y)] + \mathbb{E}[k(y, y')]$
* **Biological Meaning:** Kernel-based two-sample test comparing multivariate single-cell distributions in reproducing kernel Hilbert space without projecting down to 1D summaries.

### 14. Fréchet Single-Cell Distance (`Frechet_SingleCell_Distance`)
* **Direction:** Lower is better.
* **Formula:** $d_F^2 = \|\mu_{ref} - \mu_{sim}\|_2^2 + \text{Tr}(\Sigma_{ref} + \Sigma_{sim} - 2(\Sigma_{ref}\Sigma_{sim})^{1/2})$
* **Biological Meaning:** Multivariate Gaussian manifold divergence in principal component space between empirical and synthesized cell populations.

---

# Category 2: Correlations and Zero Patterns (6 Metrics)

Evaluates whether simulators accurately capture biological overdispersion, zero-inflation dynamics, cell-to-cell dependencies, and gene co-expression networks.

### 15. Biological Coefficient of Variation Distance (`bcv_discrepancy`)
* **Direction:** Lower is better ($|\text{BCV}_{ref} - \text{BCV}_{sim}|$).
* **Formula:** $\text{BCV} = \sqrt{\frac{\sigma^2}{\mu^2} - \frac{1}{\mu}}$
* **Biological Meaning:** Isolates true biological variability across cells after subtracting Poisson technical sampling noise.

### 16. Logistic Dropout Curve Midpoint Error (`dropout_curve_midpoint_x0`)
* **Direction:** Lower is better ($|x_{0,\text{ref}} - x_{0,\text{sim}}|$).
* **Formula:** $P(\text{zero}) = \frac{1}{1 + \exp(-k(x - x_0))}$
* **Biological Meaning:** Evaluates the expression threshold ($x_0$) at which a gene has a 50% probability of exhibiting technical dropout (zero count).

### 17. Logistic Dropout Curve Slope Error (`dropout_curve_slope_k`)
* **Direction:** Lower is better ($|k_{\text{ref}} - k_{\text{sim}}|$).
* **Biological Meaning:** Evaluates the rate of transition ($k$) from unobserved dropouts to consistently detected counts as expression levels increase.

### 18. Mean-Variance Fit / Dropout Goodness-of-Fit (`mean_variance_r2`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Quantifies how accurately the non-linear relationship between mean expression and zero frequency / variance is preserved in simulated counts.

### 19. Cell-Cell Pearson Correlation Distance (`cell_pearson_cor`)
* **Direction:** Higher is better (or Frobenius discrepancy, inverted).
* **Biological Meaning:** Confirms that pairwise cell similarity matrices and cellular manifold neighborhoods are accurately replicated.

### 20. Feature/Gene-Gene Pearson Correlation Distance (`feature_pearson_cor`)
* **Direction:** Higher is better.
* **Biological Meaning:** Assesses whether gene co-expression modules and transcriptional regulatory dependencies maintain their pairwise correlation strength.

---

# Category 3: Cell Structure and Clustering (10 Metrics)

Evaluates whether simulated datasets preserve distinct cell types, cluster separation, and biological partitions without artificial blurring.

### 21. Average Silhouette Width Discrepancy (`silhouette_sim`)
* **Direction:** Higher is better (similarity) / Lower is better (discrepancy).
* **Formula:** $s(i) = \frac{b(i) - a(i)}{\max(a(i), b(i))}$
* **Biological Meaning:** Measures how well-separated distinct cell types are relative to within-cluster compactness.

### 22. Dunn Index Discrepancy (`dunn_sim`)
* **Direction:** Higher is better / Lower discrepancy.
* **Formula:** $D = \frac{\min_{i \neq j} d(C_i, C_j)}{\max_k \text{diam}(C_k)}$
* **Biological Meaning:** Ratio between the smallest inter-cluster distance and largest cluster diameter, evaluating worst-case cluster separation.

### 23. Davies-Bouldin Index Discrepancy (`davies_bouldin_sim`)
* **Direction:** Lower is better.
* **Formula:** $DB = \frac{1}{k} \sum_{i=1}^k \max_{j \neq i} \left( \frac{\sigma_i + \sigma_j}{d(c_i, c_j)} \right)$
* **Biological Meaning:** Measures similarity between clusters; lower scores reflect compact clusters that are widely separated.

### 24. Calinski-Harabasz Index Discrepancy (`calinski_harabasz_sim`)
* **Direction:** Higher is better / Lower discrepancy.
* **Formula:** $CH = \frac{\text{SS}_B / (k - 1)}{\text{SS}_W / (N - k)}$
* **Biological Meaning:** Variance ratio criterion measuring overall between-cluster dispersion relative to within-cluster dispersion.

### 25. Adjusted Rand Index Concordance (`ari`)
* **Direction:** Higher is better (range: $[-1, 1]$, where $1.0$ is perfect match).
* **Biological Meaning:** Evaluates agreement between unsupervised clusters and known biological cell type labels, corrected for chance agreement.

### 26. Normalized Mutual Information (`nmi`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $\text{NMI}(U, V) = \frac{2 I(U; V)}{H(U) + H(V)}$
* **Biological Meaning:** Measures shared mutual information between unsupervised cluster assignments and biological cell types.

### 27. Adjusted Mutual Information (`ami`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Information-theoretic concordance between cell partitions adjusted for expected random overlap across unbalanced clusters.

### 28. $V$-Measure (`v_measure`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $V = \frac{2 \cdot h \cdot c}{h + c}$
* **Biological Meaning:** Harmonic mean of cluster homogeneity (clusters contain single cell types) and completeness (all cells of a given type are assigned together).

### 29. Neighborhood Purity (`neighborhood_purity`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Fraction of $k$-nearest neighbors in embedding space that share the same biological cell identity.

### 30. Generative Precision / Hungarian Optimal Matching (`Generative_Precision`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Solves the linear sum assignment problem (Hungarian algorithm) to find optimal one-to-one pairings between simulated clusters and real biological cell types.

---

# Category 4: Batch Effects and Technical Confounders (7 Metrics)

Evaluates whether technical batch variation is realistically synthesized without washing out biological signals.

### 31. Batch Silhouette ASW (`batch_silhouette`)
* **Direction:** Higher is better for mixed batches (or discrepancy relative to reference batch ASW).
* **Biological Meaning:** Average Silhouette Width calculated using batch labels; near-zero values indicate well-mixed batches where cells do not cluster solely by technical batch.

### 32. Batch Shannon Entropy (`shannon_entropy`)
* **Direction:** Higher is better (range: $[0, \ln(B)]$).
* **Formula:** $H = -\sum_{b=1}^B p_b \ln(p_b)$
* **Biological Meaning:** Measures batch composition diversity in local single-cell neighborhoods; high entropy indicates thorough mixing across batches.

### 33. Principal Component Regression Batch Variance (`pcr_r2`)
* **Direction:** Lower is better (or low discrepancy relative to reference).
* **Biological Meaning:** Quantifies the proportion of global gene expression variance linearly explained by technical batch covariates ($R^2_{\text{batch}}$).

### 34. CellMixS Mixing Score (`cms`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Tests whether the distribution of batch labels in local neighborhoods follows random expectations derived from overall batch proportions.

### 35. Local Inverse Simpson Index (`isi`)
* **Direction:** Higher is better (range: $[1, B]$).
* **Formula:** $\text{LISI} = \frac{1}{\sum_{b=1}^B p_b^2}$
* **Biological Meaning:** Effective number of distinct batches represented in each cell's nearest-neighbor neighborhood.

### 36. Seurat Mixing Metric (`seurat_mixing_metric`)
* **Direction:** Lower is better.
* **Biological Meaning:** Computes the average rank of neighbors belonging to the same batch; lower ranks reflect thorough mixing without batch clustering.

### 37. Local Density Factor Discrepancy (`ldf_diff`)
* **Direction:** Lower is better.
* **Biological Meaning:** Measures whether local density factor differences between real and simulated data remain small, ensuring local manifold geometries are not distorted by batch artifacts.

---

# Category 5: Marker Genes and Biological Signals (15 Metrics)

Assesses whether differentially expressed genes (DEGs), fold-changes, null p-value uniformity, and downstream classification models are faithfully preserved across established benchmarking frameworks (**SimBench**, **Simpipe**, and **Shaky Foundations**).

### 38. Symmetric Mean Absolute Percentage Error (`SimBench_SMAPE`)
* **Direction:** Lower is better (range: $[0, 200\%]$).
* **Formula:** $\text{SMAPE} = \frac{100\%}{n} \sum_{i=1}^n \frac{|y_i - \hat{y}_i|}{(|y_i| + |\hat{y}_i|)/2}$
* **Biological Meaning:** Relative percentage error in log2 fold-changes of differentially expressed genes between reference and simulated datasets.

### 39. SimBench Composite DE Fidelity Score (`SimBench_DE_Fidelity_Score`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $\text{Score} = 1 - \frac{\text{SMAPE}}{200}$
* **Biological Meaning:** Unified SimBench score capturing overall differential expression effect size retention.

### 40. Log2 Fold-Change Pearson Correlation (`Log2FC_Pearson_Corr`)
* **Direction:** Higher is better (range: $[-1, 1]$).
* **Biological Meaning:** Measures linear proportionality of estimated DEG effect sizes between real and simulated experiments.

### 41. Log2 Fold-Change Spearman Correlation (`Log2FC_Spearman_Corr`)
* **Direction:** Higher is better (range: $[-1, 1]$).
* **Biological Meaning:** Measures whether the relative ranking of marker genes by effect size is preserved.

### 42. Top DEG Jaccard Overlap (`Top_DEG_Jaccard_Overlap`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $J(A, B) = \frac{|A \cap B|}{|A \cup B|}$
* **Biological Meaning:** Set intersection overlap of the top differentially expressed marker genes between real and simulated data.

### 43. DEG Detection Ratio (`DEG_Ratio`)
* **Direction:** Matches 1.0 (range: $[0, \infty)$).
* **Formula:** $\text{Ratio} = \frac{N_{\text{DEG, sim}}}{N_{\text{DEG, ref}}}$
* **Biological Meaning:** Evaluates sensitivity and false discovery balance by checking whether simulated data generates the expected quantity of significant marker genes.

### 44. Uniform P-Value Distribution Discrepancy (`PValue_Uniformity_Chisq`)
* **Direction:** Lower is better.
* **Biological Meaning:** $\chi^2$ goodness-of-fit test assessing whether non-differentially expressed null genes yield uniform p-values without anti-conservative false positive spikes.

### 45. Null P-Value Distribution Preservation Score (`Distribution_Score`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Composite score verifying that null p-value uniformity is preserved without systematic type I error inflation.

### 46. Machine Learning Cell Type Classification Accuracy (`Classifier_Accuracy`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Accuracy of a machine learning classifier trained on marker genes in identifying cell types.

### 47. Macro-Averaged F1 Score (`Classifier_Macro_F1`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$
* **Biological Meaning:** Harmonic balance between precision and recall across all cell types, preventing common populations from masking rare cell types.

### 48. Macro-Averaged Recall Score (`Classifier_Macro_Recall`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Unweighted average sensitivity across biological cell types in simulated datasets.

### 49. Shaky Foundations Group Silhouette Similarity (`Silhouette_Sim`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Silhouette width concordance in differential marker space according to the Shaky Foundations methodology.

### 50. Shaky Foundations Silhouette Discrepancy (`Silhouette_Discrepancy`)
* **Direction:** Lower is better.
* **Biological Meaning:** Absolute difference in cell-group silhouette separation between real and simulated datasets.

### 51. Shaky Foundations Group PVE Similarity (`PVE_Group_Sim`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Preservation of the Proportion of Variance Explained (PVE) by biological condition/cell type.

### 52. Shaky Foundations PVE Discrepancy (`PVE_Discrepancy`)
* **Direction:** Lower is better.
* **Biological Meaning:** Absolute discrepancy in the proportion of total variance explained by biological group structure.

---

# Category 6: Trajectory & Lineage Dynamics (2 Metrics)

Evaluates whether continuous developmental progressions and branching lineage hierarchies are accurately reconstructed.

### 53. Pseudotime Rank Correlation (`pseudotime_correlation`)
* **Direction:** Higher is better (range: $[-1, 1]$).
* **Biological Meaning:** Spearman rank correlation of cell progression ordering along developmental trajectories (e.g. stem $\rightarrow$ progenitor $\rightarrow$ differentiated), inferred directly from expression counts.

### 54. Lineage Tree Height Discrepancy (`tree_height_rmse`)
* **Direction:** Lower is better.
* **Biological Meaning:** Root Mean Squared Error of lineage tree branch lengths and branching points between real and simulated developmental trees.

---

# Category 7: Cross-Modal Coupling & Modularity (6 Metrics)

Evaluates the joint regulatory linkage between chromatin accessibility (scATAC-seq) and gene expression (scRNA-seq) in multiomics datasets.

### 55. Cross-Modal Cell Type Transfer Accuracy (`cross_modal_accuracy`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Accuracy of predicting cell types in Modality 2 (scATAC-seq) using a classifier trained strictly on Modality 1 (scRNA-seq).

### 56. Cross-Modal Cell Type Transfer Macro-F1 Score (`cross_modal_F1`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Macro-averaged F1 score evaluating cross-modality cell identity transfer across rare and common cell states.

### 57. Fraction of Samples Closer Than The True Match (`FOSCTTM`)
* **Direction:** Lower is better (range: $[0, 0.5]$, where $0.0$ indicates perfect alignment and $0.5$ represents random chance).
* **Formula:** $\text{FOSCTTM} = \frac{1}{2N} \left( \sum_{i=1}^N \frac{|\{j : d(x_i, y_j) < d(x_i, y_i)\}|}{N} + \sum_{j=1}^N \frac{|\{i : d(y_j, x_i) < d(y_j, x_j)\}|}{N} \right)$
* **Biological Meaning:** Measures single-cell alignment accuracy in joint multi-omics latent space for co-assayed cells.

### 58. Top-1 Cross-Modal Match Rate (`match_at_1`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Biological Meaning:** Proportion of single cells whose true co-assayed profile is its nearest neighbor (rank 1) in joint embedding space.

### 59. Regulatory Coupling RV Coefficient (`rv_coefficient`)
* **Direction:** Higher is better (range: $[0, 1]$).
* **Formula:** $R_V(X, Y) = \frac{\text{Tr}(X X^T Y Y^T)}{\sqrt{\text{Tr}((X X^T)^2) \text{Tr}((Y Y^T)^2)}}$
* **Biological Meaning:** Vector matrix correlation measuring global co-inertia and shared regulatory geometry between chromatin accessibility and transcriptional output.

### 60. Cross-Modal Module Correlation Fidelity (`module_correlation_r`)
* **Direction:** Higher is better (range: $[-1, 1]$).
* **Biological Meaning:** Evaluates whether peak-to-gene correlations and cis-regulatory co-accessibility modules maintain biological concordance.

---

# Category 8: Computational Scalability (2 Metrics)

Evaluates computational runtime and memory resource consumption across simulators.

### 61. Wall-Clock Execution Time (`elapsed_time_seconds`)
* **Direction:** Lower is better.
* **Biological Meaning:** Total elapsed runtime in seconds required to train the generative model and synthesize simulated single-cell profiles.

### 62. Peak Memory Allocation (`peak_memory_mb`)
* **Direction:** Lower is better.
* **Biological Meaning:** Maximum resident memory (RAM in MiB) consumed during simulation execution.

---

# Automated Score Normalization Pipeline

To compare metrics with completely different units (seconds, megabytes, p-values, correlation coefficients) fairly on the same scale, `scSimEval` executes an automated two-step normalization:

```
Raw Metric Value
      │
      ▼
Step 1: Direction Inversion (for lower-is-better metrics)
      Inverted Value = Maximum - Value
      │
      ▼
Step 2: Min-Max Normalization
      Standardized Score = (Value - Minimum) / (Maximum - Minimum)
      │
      ▼
Standardized Score in [0.00, 1.00]
(Higher is always better; 1.00 = Top Performer)
```

In visual representations (such as `plot_benchmark_bubble_matrix()`), standardized scores are mapped directly to bubble sizes, and top performers ($\ge 0.96$) receive bold highlighted square glyphs.

```{r session-info}
sessionInfo()
```