























Abstract:Modeling the joint distribution of data samples and their properties allows to construct a single model for both data generation and property prediction, with synergistic benefits reaching beyond purely generative or predictive models. However, training joint models presents daunting architectural and optimization challenges. Here, we propose Hyformer, a transformer-based joint model that successfully blends the generative and predictive functionalities, using an alternating attention mechanism and a joint pre-training scheme. We show that Hyformer is simultaneously optimized for molecule generation and property prediction, while exhibiting synergistic benefits in conditional sampling, out-of-distribution property prediction and representation learning. Finally, we demonstrate the benefits of joint learning in a drug design use case of discovering novel antimicrobial~peptides.
| Comments: | 17 pages, 4 figures |
| Subjects: | Machine Learning (cs.LG); Quantitative Methods (q-bio.QM) |
| MSC classes: | 68T01 |
| ACM classes: | I.2.1 |
| Cite as: | arXiv:2504.16559 [cs.LG] |
| (or arXiv:2504.16559v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2504.16559 arXiv-issued DOI via DataCite |
|
| Journal reference: | Transactions on Machine Learning Research (TMLR), 2026 |
From: Adam Izdebski [view email]
[v1]
Wed, 23 Apr 2025 09:36:46 UTC (1,561 KB)
[v2]
Fri, 23 May 2025 09:02:50 UTC (11,413 KB)
[v3]
Fri, 8 May 2026 17:37:53 UTC (10,442 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。