What the geometry says

The benchmark ranks embeddings; it does not say why one lands where it does. The representation-geometry probes measure the same 20 × 26 embeddings instead — how many directions each one uses, how alike its cells are, how much of the input’s expression neighbourhood it keeps — and those measurements line up with the ranking in an instructive way.

Everything on this page is descriptive. The probes characterise what an embedding looks like; the associations with performance are model-level correlations across 20 representations, not causal claims about training.

Three findings

Foundation models use a tiny fraction of their dimensions. The normalised participation ratio npr — effective dimension over nominal dimension — is 0.999 for PCA and above 0.89 for Harmony and both Seurat strategies, but below 0.05 for every zero-shot foundation model: GenePT-w 0.0009, scFoundation 0.004, scBERT 0.010, UCE 0.010, CellPLM 0.014. A 512-dimensional or 1,536-dimensional embedding is, in variance terms, a handful of axes. That alone does not predict the ranking — scFoundation and UCE are among the top three and among the most collapsed (Spearman ρ between npr and mean biological rank = 0.30, P = 0.2) — so collapse is a diagnostic to report, not a verdict.

The worst-ranked models are anisotropic: every cell points the same way. Mean cosine similarity over random cell pairs, aniso_cos, is highest for GenePT-w (1.00), Geneformer (0.98), scBERT (0.98), scPRINT (0.97) and CELLama (0.89) — five of the six lowest-ranked zero-shot methods. Across all 20 representations, higher anisotropy accompanies worse biological conservation (ρ = −0.61, P = 0.005), and what matters is the between-type similarity (ρ = −0.67): an embedding whose different cell types are nearly parallel has little room left to separate them. The within-minus-between gap, aniso_cos_ct_gap, correlates positively with performance (ρ = 0.69, P < 0.001).

Preserving the input’s expression neighbourhoods is what the ranking rewards. rnx_mean measures how well an embedding’s within-batch k-nearest-neighbour graph reproduces the one built directly on the raw counts (analytic Pearson residuals over 2,000 genes, no dimensionality reduction, no labels). scVI trained de novo (0.200), PCA (0.186) and Harmony (0.173) preserve the most; among the frozen foundation models, scFoundation (0.163) ranks first and sixth overall. Across the 20 representations, rnx_mean tracks the biological-conservation ranking with ρ = 0.83 (P < 10⁻⁵), and among the thirteen frozen foundation models alone with ρ = 0.92. The association holds (ρ = 0.80 and 0.90) when the three neighbourhood-based metrics — Acc@kNN, cLISI, GC — are excluded from the outcome, so it is not the same quantity measured twice.

Intrinsic dimension (id_twonn_*) is the probe that does not track performance (ρ = −0.38, P = 0.10). CellFM is the outlier — a TwoNN estimate near 160 where every other method is below 30.

Explore the statistics

Select All tissues for the mean of each statistic over tissues, or one tissue for its values. There is no overall score and no rank: these are measurements. Column headings carry a one-line definition; click one to sort.

Loading the geometry tables…

Reading the columns

Column

Meaning

npr, pr

Participation ratio of the standardised embedding — the effective number of directions carrying variance — and its ratio to the nominal dimension.

aniso_spec

Share of the embedding’s total squared norm on its first singular direction. Near 1 when one common direction dominates.

aniso_cos, aniso_cos_within_ct, aniso_cos_between_ct, aniso_cos_ct_gap

Mean cosine similarity over 100,000 sampled cell pairs — mixed, same cell type, different cell types, and the difference.

rnx_mean

Chance-corrected R_NX between within-batch kNN graphs of the embedding and of Pearson-residual expression space, averaged over k = 15, 30, 50. rnx_eligible_cell_fraction is the share of cells in batches large enough to count.

id_twonn_raw, id_twonn_z

TwoNN intrinsic dimension, on the embedding as stored and after standardising every coordinate.

eta2_celltype, eta2_batch

Partial η² of cell type and of batch in the per-coordinate model z ~ cell type + batch, averaged over coordinates. Both need more than one batch and are blank on the seven single-batch tissues.

n_cells_used, embedding_dim

Cells scored (at most 20,000 per tissue, the same subsample for every method) and the nominal dimension.

Precise definitions, including how the expression reference for R_NX is built, are on Representation-geometry probes. The same statistics for your own embeddings come from scfoundry geometry.

Caveats

The performance outcome behind every correlation above is the mean per-tissue rank on the biological-conservation families only (cluster concordance, continuous separation, local neighbourhood structure), so that integration methods and zero-shot methods can be placed on one axis without the batch family favouring the former. Correlations are Spearman’s ρ over 20 model-level means, so the P values are those of a small sample.

The paper’s analysis fitted the two-factor variance model only where a batch factor exists, which is why eta2_celltype is blank on single-batch tissues here; scfoundry geometry reports the cell-type term on any dataset and leaves only the batch term blank.

Download the data

See also