title: Prot2Token: A Unified Framework for Protein Modeling via Next-Token
    Prediction
summary: The diverse nature of protein prediction tasks has traditionally necessitated
  specialized models, hindering the development of broadly applicable and
  computationally efficient Protein Language Models (PLMs). In this work, we
  introduce Prot2Token, a unified framework that overcomes these challenges by
  converting a wide spectrum of protein-related predictions, from sequence-level
  properties and residue-specific attributes to complex inter-protein
  interactions, into a standardized next-token prediction format. At its core,
  Prot2Token employs an autoregressive decoder, conditioned on embeddings from
  pre-trained protein encoders and guided by learnable task tokens, to perform
  diverse predictions. This architecture uniquely facilitates multi-task
  learning, enabling a single model to master numerous tasks with improved
  efficiency. We present extensive experimental validation across a variety of
  benchmarks, demonstrating Prot2Tokens strong predictive power in different
  types of protein-prediction tasks. Key results include significant speedups
  (e.g., near 1000x over AlphaFold2 with MSA) and performance often matching or
  exceeding specialized approaches. Beyond that, we introduce an auxiliary
  self-supervised decoder pre-training approach to improve spatially sensitive
  task performance. Prot2Token thus offers a significant step towards a
  versatile, high-throughput paradigm for protein modeling, promising to
  accelerate biological discovery and the development of novel therapeutics. The
  code is available at https://github.com/mahdip72/prot2token .
paper_authors._id: 6836b933f9baa9388e75e171
paper_authors.name: Mahdi Pourmirzaei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:43:49.366Z
paper_authors._id: 62d05322dd7bdfc5e5c86298
paper_authors.avatarUrl: /avatars/62879c8a8bbdbaaf63da27a44e90bd5f.svg
paper_authors.fullname: Mahdi Pourmirzaei
paper_authors.type: user
paper_authors.user: Mahdip72
paper_authors._id: 6836b933f9baa9388e75e172
paper_authors.name: Farzaneh Esmaili
paper_authors._id: 6836b933f9baa9388e75e173
paper_authors.name: Salhuldin Alqarghuli
paper_authors._id: 6836b933f9baa9388e75e174
paper_authors.name: Mohammadreza Pourmirzaei
paper_authors._id: 6836b933f9baa9388e75e175
paper_authors.name: Ye Han
paper_authors._id: 6836b933f9baa9388e75e176
paper_authors.name: Kai Chen
paper_authors._id: 6836b933f9baa9388e75e177
paper_authors.name: Mohsen Rezaei
paper_authors._id: 6836b933f9baa9388e75e178
paper_authors.name: Duolin Wang
paper_authors._id: 6836b933f9baa9388e75e179
paper_authors.name: Dong Xu
paper_title: Prot2Token: A Unified Framework for Protein Modeling via Next-Token
    Prediction
paper_summary: The diverse nature of protein prediction tasks has traditionally necessitated
  specialized models, hindering the development of broadly applicable and
  computationally efficient Protein Language Models (PLMs). In this work, we
  introduce Prot2Token, a unified framework that overcomes these challenges by
  converting a wide spectrum of protein-related predictions, from sequence-level
  properties and residue-specific attributes to complex inter-protein
  interactions, into a standardized next-token prediction format. At its core,
  Prot2Token employs an autoregressive decoder, conditioned on embeddings from
  pre-trained protein encoders and guided by learnable task tokens, to perform
  diverse predictions. This architecture uniquely facilitates multi-task
  learning, enabling a single model to master numerous tasks with improved
  efficiency. We present extensive experimental validation across a variety of
  benchmarks, demonstrating Prot2Tokens strong predictive power in different
  types of protein-prediction tasks. Key results include significant speedups
  (e.g., near 1000x over AlphaFold2 with MSA) and performance often matching or
  exceeding specialized approaches. Beyond that, we introduce an auxiliary
  self-supervised decoder pre-training approach to improve spatially sensitive
  task performance. Prot2Token thus offers a significant step towards a
  versatile, high-throughput paradigm for protein modeling, promising to
  accelerate biological discovery and the development of novel therapeutics. The
  code is available at https://github.com/mahdip72/prot2token .
paper_ai_summary: Prot2Token unifies protein prediction tasks using an autoregressive decoder with task tokens, improving efficiency and accuracy across different benchmarks compared to specialized models.
paper_ai_keywords: Protein Language Models autoregressive decoder pre-trained protein encoders task tokens multi-task learning next-token prediction self-supervised decoder pre-training