Abstract:Single-cell representation learning (SCRL) from gene expression data offers a way to uncover the complex regulatory logic underlying cellular function. Inspired by large language models in natural language modeling, several single-cell pretrained models have recently been proposed that treat genes as tokens and cells as sentences. However, these models are fundamentally limited by the long-tailed nature of cell-type distributions and struggle to generalize under covariate shifts in gene expression data. While fine-tuning is often used to mitigate these issues, we observe that performance remains bounded. To address this challenge, we introduce CellRefine, a post-pretraining method that operates between the pretraining and fine-tuning stages of a single-cell foundation model. CellRefine uses a multi-faceted objective that incorporates marker-gene sets as structural priors to guide post-pretraining and refine the latent embedding manifold of cells. Across multiple computational biology tasks, empirical results show that CellRefine consistently improves downstream performance, yielding gains up to 15%.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.07938 [cs.LG] |
| (or arXiv:2605.07938v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.07938 arXiv-issued DOI via DataCite (pending registration) |
From: Sachini Weerasekara [view email]
[v1]
Fri, 8 May 2026 16:08:10 UTC (1,686 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。