title: Large-Scale Multi-omic Biosequence Transformers for Modeling
    Peptide-Nucleotide Interactions
summary: The transformer architecture has revolutionized bioinformatics and driven
  progress in the understanding and prediction of the properties of biomolecules.
  Almost all research on large-scale biosequence transformers has focused on one
  domain at a time (single-omic), usually nucleotides or peptides. These models
  have seen incredible success in downstream tasks in each domain and have
  achieved particularly noteworthy breakthroughs in sequences of peptides and
  structural modeling. However, these single-omic models are naturally incapable
  of modeling multi-omic tasks, one of the most biologically critical being
  nucleotide-peptide interactions.
    We present our work training the first multi-omic nucleotide-peptide
  foundation models. We show that these multi-omic models (MOMs) can learn joint
  representations between various single-omic distributions that are emergently
  consistent with the Central Dogma of molecular biology, despite only being
  trained on unlabeled biosequences. We further demonstrate that MOMs can be
  fine-tuned to achieve state-of-the-art results on peptide-nucleotide
  interaction tasks, namely predicting the change in Gibbs free energy
  ({\Delta}G) of the binding interaction between a given oligonucleotide and
  peptide, as well as the effect on this binding interaction due to mutations in
  the oligonucleotide sequence ({\Delta}{\Delta}G).
    Remarkably, we show that multi-omic biosequence transformers emergently learn
  useful structural information without any prior structural training, allowing
  us to predict which peptide residues are most involved in the
  peptide-nucleotide binding interaction. Lastly, we provide evidence that
  multi-omic biosequence models are non-inferior to foundation models trained on
  single-omics distributions, suggesting a more generalized or foundational
  approach to building these models.
paper_authors._id: 66d581cead293ffc4b2ed95e
paper_authors.name: Sully F. Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T09:22:48.920Z
paper_authors._id: 6328dd305dc16f128183f480
paper_authors.avatarUrl: /avatars/d342cfa058fa2cceb96b4b15b3edeac4.svg
paper_authors.fullname: Sully Chen
paper_authors.type: user
paper_authors.user: WeiHua
paper_authors._id: 66d581cead293ffc4b2ed95f
paper_authors.name: Robert J. Steele
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-05T06:44:01.673Z
paper_authors._id: 67c7f2a3252114c63bfea097
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/75MAkb33OFBXNwkFWIY69.png
paper_authors.fullname: Robert Steele
paper_authors.type: user
paper_authors.user: re-bort
paper_authors._id: 66d581cead293ffc4b2ed960
paper_authors.name: Beakal Lemeneh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T09:22:11.952Z
paper_authors._id: 64e14499de27e92bea8f8d05
paper_authors.avatarUrl: /avatars/fd7c41828927e71c50bc7ed33831a345.svg
paper_authors.fullname: Lemeneh
paper_authors.type: user
paper_authors.user: Beakal
paper_authors._id: 66d581cead293ffc4b2ed961
paper_authors.name: Shivanand P. Lad
paper_authors._id: 66d581cead293ffc4b2ed962
paper_authors.name: Eric Oermann
paper_title: Large-Scale Multi-omic Biosequence Transformers for Modeling
    Peptide-Nucleotide Interactions
paper_summary: The transformer architecture has revolutionized bioinformatics and driven
  progress in the understanding and prediction of the properties of biomolecules.
  Almost all research on large-scale biosequence transformers has focused on one
  domain at a time (single-omic), usually nucleotides or peptides. These models
  have seen incredible success in downstream tasks in each domain and have
  achieved particularly noteworthy breakthroughs in sequences of peptides and
  structural modeling. However, these single-omic models are naturally incapable
  of modeling multi-omic tasks, one of the most biologically critical being
  nucleotide-peptide interactions.
    We present our work training the first multi-omic nucleotide-peptide
  foundation models. We show that these multi-omic models (MOMs) can learn joint
  representations between various single-omic distributions that are emergently
  consistent with the Central Dogma of molecular biology, despite only being
  trained on unlabeled biosequences. We further demonstrate that MOMs can be
  fine-tuned to achieve state-of-the-art results on peptide-nucleotide
  interaction tasks, namely predicting the change in Gibbs free energy
  ({\Delta}G) of the binding interaction between a given oligonucleotide and
  peptide, as well as the effect on this binding interaction due to mutations in
  the oligonucleotide sequence ({\Delta}{\Delta}G).
    Remarkably, we show that multi-omic biosequence transformers emergently learn
  useful structural information without any prior structural training, allowing
  us to predict which peptide residues are most involved in the
  peptide-nucleotide binding interaction. Lastly, we provide evidence that
  multi-omic biosequence models are non-inferior to foundation models trained on
  single-omics distributions, suggesting a more generalized or foundational
  approach to building these models.
paper_ai_summary: A multi-omic nucleotide-peptide foundation model learns joint representations of biosequences, captures structural information without prior structural training, and achieves state-of-the-art results in predicting peptide-nucleotide interactions and mutations impact.
paper_ai_keywords: transformer architecture bioinformatics multi-omic models MOMs single-omic nucleotides peptides Gibbs free energy ΔG ΔΔG Central Dogma structural information biosequence transformers