Contributing a method

New methods are welcome as pull requests to [https://github.com/Svvord/scFoundry](https://github.com/Svvord/scFoundry). The bar is deliberately high, because every method in the repository is implicitly claimed to run its authors’ recipe — and a method that does not would make the benchmark wrong for everyone. This page is the checklist a pull request is reviewed against.

Note

Contributions open with the first public release of scFoundry, once the accompanying paper is published. Until then the repository accepts issues but not method pull requests; the checklist below is what they will be reviewed against.

What a pull request contains

Item

Requirement

containers/<id>/Dockerfile

Required. The image the module uses, buildable from this file, every dependency pinned, tagged with a version (not latest). Maintainers rebuild it before merging.

workflows/utils/download.nf process

The official checkpoint, from the authors’ release, at a pinned revision, with a checksum where the source does not pin one.

workflows/methods/<id>.nf (+ bin/<id>/)

The module, following the contracts. Only the tasks the authors’ recipe supports.

conf/methods.json entry and task-map entries

As described in Adding a method; tests/test_registry.py passes.

Method card

In the pull request description — see below.

Demo evidence

The scfoundry embed output shape on data/demo/colon_1000.h5ad and the benchmark table it produces next to PCA’s, plus whatever you used to confirm the recipe against the authors’ own results.

Documentation

The rows listed under Document it, as a pull request to the documentation repository.

The method card

The review is a comparison of the module against the authors’ published usage, so the pull request has to make that comparison possible. Fill in every line:

### Method card: NewModel

- Paper / preprint: <link>
- Code: <repo> at commit/tag <…>; licence: <…>
- Checkpoint: <name>, from <official source URL>, revision <…>, sha256 <…>; licence: <…>
- Category: zero-shot | reference | integration
- Tasks implemented: embed, transfer[, finetune]
- Recipe source: <link to the authors' tutorial / script the module follows>
- Preprocessing performed by the module: <normalisation, gene alignment, tokenisation, …>
- Embedding: <which layer / pooling>; dimension <…>
- Defaults: model <…>, batch_size <…>[, finetune_epoch <…>, finetune_batch_size <…>]
- Deviations from the authors' recipe, and why: <none | …>
- Validation: <what you reproduced from the authors' results, and how close it was>
- Demo: colon_1000 embedding shape <…>; benchmark table attached
- Resource notes: GPU memory at the default batch size, runtime on the demo

“Deviations: none” is the expected answer. Where a deviation is unavoidable — a tutorial that only exists for one dataset, an API that has changed — say so; it will be recorded in the registry notes and on the method reference.

What maintainers do

  1. Rebuild the image from the Dockerfile and push it under the project’s registry namespace with the version tag; the module is updated to point there.

  2. Run the demo smoke tests and the unit tests.

  3. Read the module against the method card and the authors’ code.

  4. Merge, and add the method to the documentation.

Appearing in the published comparison

Merging a method does not put it in the benchmark results. Those tables are the paper’s, computed by the maintainers on the 26 Tabula Sapiens tissues, and a new row means running the method on all of them under the same protocol.

Two routes:

  • Maintainers run it. When time and GPUs allow, the maintainers embed the 26 tissues with the merged method, score them with scfoundry benchmark and scfoundry geometry, and add the rows. Such rows are indistinguishable from the original ones.

  • The contributor runs it. A contributor who has run the full protocol themselves can submit the resulting wide tables (the *_metrics_wide.csv files of benchmark and the *_geometry.csv of geometry) together with the run records. The rows are added with a community badge, which states that the numbers were supplied by the contributor under the published protocol and have not been reproduced by the maintainers. The badge is removed if and when the maintainers reproduce them.

Either way the inputs are the prepared Tabula Sapiens files described in Benchmark design, the settings are the method’s defaults, and the results are ranked with everything else — a community method is not ranked in a separate league, only labelled.

Other contributions

Bug reports and fixes, documentation corrections and reproducibility reports (a method that no longer builds, a checkpoint that moved) are welcome at any time as issues or pull requests, with the usual expectations: one change per pull request, the unit tests passing, and for pipeline changes a demo run showing the effect.

See also