Representation-geometry probes

The statistics that scfoundry geometry computes, defined precisely enough to reimplement. They describe an embedding; none of them is a score.

Common preprocessing

Alignment. The embedding file and its raw-count input are matched by file name and aligned cell by cell on obs_names. A mismatch is an error, not a warning.

Subsampling. Datasets larger than --max-cells (20,000) are subsampled once, without replacement, with a seed derived from --seed and the dataset name. Every method scored on that dataset therefore sees the same cells, and the pair sets below are the same pairs.

Two views of the embedding. Let Z (n × d) be the embedding as stored. Zₛ is Z with every coordinate centred and scaled to unit variance; coordinates with zero variance are set to 0 and counted in zero_var_dims. Which view a probe uses is stated below — the choice matters, because standardising removes exactly the scale differences that anisotropy measures.

Effective dimension

pr, npr — on Zₛ.

With C the covariance matrix of Zₛ,

PR  = (tr C)² / tr(C²) = (Σ λᵢ)² / Σ λᵢ²
nPR = PR / d

where λᵢ are the eigenvalues of C. PR equals d when all eigenvalues are equal and 1 when a single direction carries all the variance. tr(C²) is the squared Frobenius norm of C, computed exactly for embeddings of up to 1,600 dimensions and by a Hutchinson estimator with 128 probes above that; the notes column records which.

Anisotropy

aniso_spec — on Z, uncentred.

aniso_spec = σ₁² / ‖Z‖²_F

the share of the embedding’s total energy on its first singular direction, from a randomised SVD (one component, five power iterations). It is 1 when every cell is a multiple of one vector; a common offset shared by all cells drives it up, which is why the embedding is not centred first.

aniso_cos, aniso_cos_within_ct, aniso_cos_between_ct, aniso_cos_ct_gap — on Z.

The mean cosine similarity ⟨zᵢ, zⱼ⟩ / (‖zᵢ‖‖zⱼ‖) over three fixed sets of 100,000 cell pairs, drawn with a seed derived from the dataset name so that every method is evaluated on the same pairs:

• mixed — i and j uniform over all cells, i ≠ j;

• within type — a cell type drawn uniformly over types, then two distinct cells of it;

• between types — two distinct cell types drawn uniformly, then one cell of each.

Sampling types uniformly rather than cells keeps abundant types from dominating both within- and between-type sets. aniso_cos_ct_gap is within minus between. Pairs involving a zero vector are dropped and the count of valid pairs is recorded.

Expression-neighbourhood preservation

rnx_k15, rnx_k30, rnx_k50, rnx_mean, and the per-batch table — on Z.

The question is whether cells that are neighbours in expression space are still neighbours in the embedding. The reference is built from the raw counts, per batch, without dimensionality reduction and without labels:

1. Eligible batches. Batches of at least 101 cells (--min-batch-cells). A dataset with no eligible batch gets no R_NX. rnx_eligible_cell_fraction is the share of the subsampled cells in eligible batches.

2. Gene candidates. All genes except mitochondrial, ribosomal-protein and ERCC spike-in genes.

3. Analytic Pearson residuals (Lause, Berens and Kobak, 2021), per batch: with μᵢⱼ = (row sum × column sum) / batch total, θ = 100, and the residual (xᵢⱼ − μᵢⱼ) / √(μᵢⱼ + μᵢⱼ²/θ) clipped to ±√n_b, where n_b is the batch size.

4. Gene selection. Within each eligible batch, candidate genes are ranked by residual variance and the rank is expressed as a percentile; the 2,000 genes with the highest median percentile across batches are kept (--n-genes), so that one large batch cannot dictate the panel.

5. Reference graph. Within each eligible batch, Euclidean k-nearest neighbours in the 2,000-gene residual space, k = 50 — exact for batches up to 5,000 cells, PyNNDescent above that.

The embedding graph is built the same way on Z, within the same batch. For each k ∈ {15, 30, 50}:

Q_NX(k) = mean over cells of |N_ref(i, k) ∩ N_emb(i, k)| / k
R_NX(k) = ((N − 1) · Q_NX(k) − k) / (N − 1 − k)

with N the batch size. Q_NX is the neighbourhood overlap; R_NX (Lee and Verleysen, 2009) rescales it so that 0 is the overlap of a random embedding and 1 perfect preservation. Each dataset’s value is the cell-weighted mean over its eligible batches; rnx_mean is the mean over the three k. Building the graphs within batches is what makes the criterion independent of batch effects: an embedding is only asked to preserve the neighbourhoods that exist inside one technical batch.

Intrinsic dimension

id_twonn_raw on Z, id_twonn_z on Zₛ.

The TwoNN estimator (Facco et al., 2017). With r₁ and r₂ the distances from each cell to its first and second nearest neighbours (Euclidean, exact up to 5,000 cells) and μ = r₂ / r₁,

d = N / Σᵢ log μᵢ

over the cells with r₁ > 0 and r₂ > r₁. It estimates the dimension of the manifold the cells lie on, which is unrelated to the number of coordinates: CellFM’s embedding, for instance, has a nominal dimension of 1,536 and a TwoNN estimate near 160, while most methods fall below 30.

Variance decomposition

partial_eta2_celltype, partial_eta2_batch — on Zₛ, over the labelled cells.

For every coordinate zⱼ fit the linear model

zⱼ ~ 1 + cell type + batch

with one-hot factors, by QR decomposition. For each factor, the reduced model without it is fitted, and

partial η² = SS_effect / (SS_effect + SSE_full),   SS_effect = SSE_reduced − SSE_full

so each factor is adjusted for the other. The reported value is the mean over coordinates with non-degenerate variance. On single-batch data the model is zⱼ ~ 1 + cell type and partial_eta2_batch is NaN; the paper’s tables, which fitted only the two-factor model, leave both terms undefined on such tissues.

Seeds

All randomness is seeded from --seed (default 0) combined with the dataset name, so that a probe rerun on the same data reproduces its numbers, and every method sees the same subsample and the same cell pairs:

Randomness	Seed derived from
Subsample of cells	dataset, --max-cells
Anisotropy pair sets	dataset
Hutchinson probes, randomised SVD	dataset, method
Nearest neighbours for TwoNN	dataset (on Z); dataset, method (on Zₛ)
PyNNDescent (reference graph)	dataset, batch
PyNNDescent (embedding graph)	dataset, batch, method

References

• Facco E., d’Errico M., Rodriguez A., Laio A. (2017). Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific Reports 7, 12140.

• Lee J. A., Verleysen M. (2009). Quality assessment of dimensionality reduction: rank-based criteria. Neurocomputing 72, 1431–1443.

• Lause J., Berens P., Kobak D. (2021). Analytic Pearson residuals for normalization of single-cell RNA-seq UMI data. Genome Biology 22, 258.

• Ethayarajh K. (2019). How contextual are contextualized word representations? EMNLP. The mean-cosine measure of anisotropy.

See also

• Geometry — running the probes.

• What the geometry says — the statistics on the benchmark.