CellForge
A reliability-aware trust layer between perturbation screens and experimental decisions
A Python benchmarking harness for single-cell perturbation models that validates datasets, builds unseen-perturbation splits, scores models against strong baselines, and grades every prediction by how reliable the underlying measurement is.
Python
PyTorch
GEARS
Scanpy
AnnData
Pertpy
pandas
Plotly
Streamlit
3Dmol.js
DockerA model can look strong on a random cell split while failing on unseen biology, and a prediction can only be as trustworthy as the measurement it is compared with. CellForge makes the split, the baseline, and the measurement reliability explicit before any model earns trust.
Dataset validation, deterministic unseen-perturbation splits, control-mean, training-mean, and embedding-ridge baselines, and one GEARS adapter run under the same evaluation contract.
A split-half, Spearman-Brown reliability gate classifies each perturbation as specific, shared, or unreliable before a model-vs-baseline comparison can raise trust.
Normalized, payload-hashed evidence records and conservative contradiction rules feed deterministic decision packages that a scientist reviews.
A bounded multi-connector gateway applies allowlists, concurrency limits, timeouts, and separate approval gates, and returns a SHA-256 manifest of every run.
On the Norman 2019 Perturb-seq data with 10 held-out perturbations, GEARS reached 0.33 mean Pearson against 0.60 for the training-mean baseline and lost to ridge on 9 of 10 targets. Replacing a mismatched SNR heuristic with split-half reliability changed trust for one target, CEBPA, and the run is documented as a one-epoch CPU verification, not a tuned benchmark.