Prot2Token: A Unified Framework for Protein Modeling via Next-Token
  Prediction
The diverse nature of protein prediction tasks has traditionally necessitated
specialized models, hindering the development of broadly applicable and
computationally efficient Protein Language Models (PLMs). In this work, we
introduce Prot2Token, a unified framework that overcomes these challenges by
converting a wide spectrum of protein-related predictions, from sequence-level
properties and residue-specific attributes to complex inter-protein
interactions, into a standardized next-token prediction format. At its core,
Prot2Token employs an autoregressive decoder, conditioned on embeddings from
pre-trained protein encoders and guided by learnable task tokens, to perform
diverse predictions. This architecture uniquely facilitates multi-task
learning, enabling a single model to master numerous tasks with improved
efficiency. We present extensive experimental validation across a variety of
benchmarks, demonstrating Prot2Tokens strong predictive power in different
types of protein-prediction tasks. Key results include significant speedups
(e.g., near 1000x over AlphaFold2 with MSA) and performance often matching or
exceeding specialized approaches. Beyond that, we introduce an auxiliary
self-supervised decoder pre-training approach to improve spatially sensitive
task performance. Prot2Token thus offers a significant step towards a
versatile, high-throughput paradigm for protein modeling, promising to
accelerate biological discovery and the development of novel therapeutics. The
code is available at https://github.com/mahdip72/prot2token .
6836b933f9baa9388e75e171
Mahdi Pourmirzaei
claimed_verified
2025-05-29T07:43:49.366Z
62d05322dd7bdfc5e5c86298
/avatars/62879c8a8bbdbaaf63da27a44e90bd5f.svg
Mahdi Pourmirzaei
user
Mahdip72
6836b933f9baa9388e75e172
Farzaneh Esmaili
6836b933f9baa9388e75e173
Salhuldin Alqarghuli
6836b933f9baa9388e75e174
Mohammadreza Pourmirzaei
6836b933f9baa9388e75e175
Ye Han
6836b933f9baa9388e75e176
Kai Chen
6836b933f9baa9388e75e177
Mohsen Rezaei
6836b933f9baa9388e75e178
Duolin Wang
6836b933f9baa9388e75e179
Dong Xu
Prot2Token: A Unified Framework for Protein Modeling via Next-Token
  Prediction
The diverse nature of protein prediction tasks has traditionally necessitated
specialized models, hindering the development of broadly applicable and
computationally efficient Protein Language Models (PLMs). In this work, we
introduce Prot2Token, a unified framework that overcomes these challenges by
converting a wide spectrum of protein-related predictions, from sequence-level
properties and residue-specific attributes to complex inter-protein
interactions, into a standardized next-token prediction format. At its core,
Prot2Token employs an autoregressive decoder, conditioned on embeddings from
pre-trained protein encoders and guided by learnable task tokens, to perform
diverse predictions. This architecture uniquely facilitates multi-task
learning, enabling a single model to master numerous tasks with improved
efficiency. We present extensive experimental validation across a variety of
benchmarks, demonstrating Prot2Tokens strong predictive power in different
types of protein-prediction tasks. Key results include significant speedups
(e.g., near 1000x over AlphaFold2 with MSA) and performance often matching or
exceeding specialized approaches. Beyond that, we introduce an auxiliary
self-supervised decoder pre-training approach to improve spatially sensitive
task performance. Prot2Token thus offers a significant step towards a
versatile, high-throughput paradigm for protein modeling, promising to
accelerate biological discovery and the development of novel therapeutics. The
code is available at https://github.com/mahdip72/prot2token .
Prot2Token unifies protein prediction tasks using an autoregressive decoder with task tokens, improving efficiency and accuracy across different benchmarks compared to specialized models.
Protein Language Models autoregressive decoder pre-trained protein encoders task tokens multi-task learning next-token prediction self-supervised decoder pre-training