Demo datasets¶

Four small .h5ad files live in the code repository under data/demo/. They are not part of the scfoundry package, so download them once into your workspace:

mkdir -p demo
for f in colon_1000 colon_50 liver_1shot_support liver_1shot_query; do
  curl -L -o demo/$f.h5ad https://github.com/Svvord/scFoundry/raw/main/data/demo/$f.h5ad
done

They already satisfy the input contract, so you can exercise every task before preparing your own data. Together they are about 112 MB.

File	Cells × genes	Cell types	Batches	Used by
`colon_1000.h5ad`	1,000 × 35,461	7	7	`embed`, `benchmark`, `geometry`, `finetune` (reference)
`colon_50.h5ad`	50 × 35,461	6	7	`finetune` (held-out query)
`liver_1shot_support.h5ad`	5 × 30,257	5	4	`transfer` (reference)
`liver_1shot_query.h5ad`	75 × 30,257	5	6	`transfer` (query)

The colon pair¶

colon_1000.h5ad is a 1,000-cell sample of human colon with CELLxGENE-style metadata. Its obs carries 42 columns, including everything the tasks need:

>>> adata.obs["cell_type"].cat.categories.tolist()
['epithelial cell', 'enteroendocrine cell', 'M cell of gut',
 'intestinal crypt stem cell', 'intestine goblet cell', 'colonocyte',
 'BEST4+ enterocyte']

>>> adata.obs["batch_id"].cat.categories.tolist()[:4]
["F66|10x 3' v2", "F67|10x 3' v2", "F72|10x 5' v2", "F73|10x 5' v2"]

Notice how batch_id is constructed: donor crossed with assay. That is a real technical batch axis — the same donors were run on both 10x 3′ v2 and 5′ v2 chemistry — rather than a donor split that would confound batch with biology.

colon_50.h5ad is a 50-cell companion drawn from the same tissue with the same gene space. It exists so the fine-tuning walkthrough has a held-out set to predict on without requiring you to split anything yourself. It covers 6 of the 7 cell types.

Note

Both colon files are genuinely raw counts (integer-valued, sparse CSR, maximum around 350 per gene in the 1,000-cell file) and cover the full 35,461-gene transcriptome. They are a good reference if you are unsure whether your own file is prepared correctly.

Its seven batches are small — between 30 and 363 cells — which is enough for Harmony but not for Seurat’s CCA and RPCA, which need more than 30 cells per batch. See Embed.

The liver pair¶

These two demonstrate the extreme end of label transfer: liver_1shot_support.h5ad contains exactly five cells, one per cell type. That single cell per class is all the classifier gets.

>>> support.obs["cell_type"].cat.categories.tolist()
['mast cell', 'mononuclear phagocyte', 'natural killer cell',
 'plasma cell', 'cholangiocyte']
>>> support.n_obs
5

liver_1shot_query.h5ad holds 75 cells spanning the same five classes. It does carry ground-truth labels, which the prediction never reads — they are there so you can score the result afterwards:

import anndata as ad
import pandas as pd

pred = pd.read_csv(
    "results/transfer/predictions/scgpt/prototype/liver_1shot_query_predicted_labels.tsv",
    sep="\t", index_col=0,
)
truth = ad.read_h5ad("demo/liver_1shot_query.h5ad").obs["cell_type"]

accuracy = (pred["predicted_label"] == truth.reindex(pred.index)).mean()
print(f"1-shot accuracy: {accuracy:.1%}")

With one cell per class, prototype and knn (with --knn-k 1) are the classifiers that make sense; logreg fits, but on five points; mlp refuses to run because it cannot hold out a validation split.

What these datasets are good for¶

Verifying an installation. The PCA reference on colon_1000.h5ad needs no checkpoint and exercises the container mounts, so it is the fastest way to confirm a working setup.

Comparing methods cheaply. A thousand cells embeds in minutes on most models, so you can run several and compare before committing GPU hours to a real dataset.

Learning the output layout. Running each task once produces the full results/ tree, which is easier to explore than reading about it.

What they are not good for¶

They are far too small for conclusions. Metric values on 1,000 cells across 7 cell types are dominated by sampling noise, and the 5-cell reference is a stress test rather than a realistic annotation scenario. The benchmark in the accompanying paper uses 26 tissues and 548,977 cells for exactly this reason — see Benchmark design.