Large-Scale Multi-omic Biosequence Transformers for Modeling
  Peptide-Nucleotide Interactions
The transformer architecture has revolutionized bioinformatics and driven
progress in the understanding and prediction of the properties of biomolecules.
Almost all research on large-scale biosequence transformers has focused on one
domain at a time (single-omic), usually nucleotides or peptides. These models
have seen incredible success in downstream tasks in each domain and have
achieved particularly noteworthy breakthroughs in sequences of peptides and
structural modeling. However, these single-omic models are naturally incapable
of modeling multi-omic tasks, one of the most biologically critical being
nucleotide-peptide interactions.
  We present our work training the first multi-omic nucleotide-peptide
foundation models. We show that these multi-omic models (MOMs) can learn joint
representations between various single-omic distributions that are emergently
consistent with the Central Dogma of molecular biology, despite only being
trained on unlabeled biosequences. We further demonstrate that MOMs can be
fine-tuned to achieve state-of-the-art results on peptide-nucleotide
interaction tasks, namely predicting the change in Gibbs free energy
({\Delta}G) of the binding interaction between a given oligonucleotide and
peptide, as well as the effect on this binding interaction due to mutations in
the oligonucleotide sequence ({\Delta}{\Delta}G).
  Remarkably, we show that multi-omic biosequence transformers emergently learn
useful structural information without any prior structural training, allowing
us to predict which peptide residues are most involved in the
peptide-nucleotide binding interaction. Lastly, we provide evidence that
multi-omic biosequence models are non-inferior to foundation models trained on
single-omics distributions, suggesting a more generalized or foundational
approach to building these models.
66d581cead293ffc4b2ed95e
Sully F. Chen
admin_assigned
2024-09-02T09:22:48.920Z
6328dd305dc16f128183f480
/avatars/d342cfa058fa2cceb96b4b15b3edeac4.svg
Sully Chen
user
WeiHua
66d581cead293ffc4b2ed95f
Robert J. Steele
extracted_pending
2025-03-05T06:44:01.673Z
67c7f2a3252114c63bfea097
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/75MAkb33OFBXNwkFWIY69.png
Robert Steele
user
re-bort
66d581cead293ffc4b2ed960
Beakal Lemeneh
admin_assigned
2024-09-02T09:22:11.952Z
64e14499de27e92bea8f8d05
/avatars/fd7c41828927e71c50bc7ed33831a345.svg
Lemeneh
user
Beakal
66d581cead293ffc4b2ed961
Shivanand P. Lad
66d581cead293ffc4b2ed962
Eric Oermann
Large-Scale Multi-omic Biosequence Transformers for Modeling
  Peptide-Nucleotide Interactions
The transformer architecture has revolutionized bioinformatics and driven
progress in the understanding and prediction of the properties of biomolecules.
Almost all research on large-scale biosequence transformers has focused on one
domain at a time (single-omic), usually nucleotides or peptides. These models
have seen incredible success in downstream tasks in each domain and have
achieved particularly noteworthy breakthroughs in sequences of peptides and
structural modeling. However, these single-omic models are naturally incapable
of modeling multi-omic tasks, one of the most biologically critical being
nucleotide-peptide interactions.
  We present our work training the first multi-omic nucleotide-peptide
foundation models. We show that these multi-omic models (MOMs) can learn joint
representations between various single-omic distributions that are emergently
consistent with the Central Dogma of molecular biology, despite only being
trained on unlabeled biosequences. We further demonstrate that MOMs can be
fine-tuned to achieve state-of-the-art results on peptide-nucleotide
interaction tasks, namely predicting the change in Gibbs free energy
({\Delta}G) of the binding interaction between a given oligonucleotide and
peptide, as well as the effect on this binding interaction due to mutations in
the oligonucleotide sequence ({\Delta}{\Delta}G).
  Remarkably, we show that multi-omic biosequence transformers emergently learn
useful structural information without any prior structural training, allowing
us to predict which peptide residues are most involved in the
peptide-nucleotide binding interaction. Lastly, we provide evidence that
multi-omic biosequence models are non-inferior to foundation models trained on
single-omics distributions, suggesting a more generalized or foundational
approach to building these models.
A multi-omic nucleotide-peptide foundation model learns joint representations of biosequences, captures structural information without prior structural training, and achieves state-of-the-art results in predicting peptide-nucleotide interactions and mutations impact.
transformer architecture bioinformatics multi-omic models MOMs single-omic nucleotides peptides Gibbs free energy ΔG ΔΔG Central Dogma structural information biosequence transformers