DPLM-2: A Multimodal Diffusion Protein Language Model
Proteins are essential macromolecules defined by their amino acid sequences,
which determine their three-dimensional structures and, consequently, their
functions in all living organisms. Therefore, generative protein modeling
necessitates a multimodal approach to simultaneously model, understand, and
generate both sequences and structures. However, existing methods typically use
separate models for each modality, limiting their ability to capture the
intricate relationships between sequence and structure. This results in
suboptimal performance in tasks that requires joint understanding and
generation of both modalities. In this paper, we introduce DPLM-2, a multimodal
protein foundation model that extends discrete diffusion protein language model
(DPLM) to accommodate both sequences and structures. To enable structural
learning with the language model, 3D coordinates are converted to discrete
tokens using a lookup-free quantization-based tokenizer. By training on both
experimental and high-quality synthetic structures, DPLM-2 learns the joint
distribution of sequence and structure, as well as their marginals and
conditionals. We also implement an efficient warm-up strategy to exploit the
connection between large-scale evolutionary data and structural inductive
biases from pre-trained sequence-based protein language models. Empirical
evaluation shows that DPLM-2 can simultaneously generate highly compatible
amino acid sequences and their corresponding 3D structures eliminating the need
for a two-stage generation approach. Moreover, DPLM-2 demonstrates competitive
performance in various conditional generation tasks, including folding, inverse
folding, and scaffolding with multimodal motif inputs, as well as providing
structure-aware representations for predictive tasks.
6715fdfc6757313a2bca5806
Xinyou Wang
claimed_verified
2024-10-22T08:01:14.824Z
64d0938b37a8b7adaf8c8219
/avatars/d8ea6a7fe938db2280b828aaeb41dcaf.svg
wang
user
airkingbd
6715fdfc6757313a2bca5807
Zaixiang Zheng
claimed_verified
2024-10-21T12:31:01.433Z
6347faa36d7f1d10f5ecdc18
/avatars/0fea067fa7bc8c4a797268b68a868756.svg
zzheng
user
zzheng
6715fdfc6757313a2bca5808
Fei Ye
6715fdfc6757313a2bca5809
Dongyu Xue
6715fdfc6757313a2bca580a
Shujian Huang
6715fdfc6757313a2bca580b
Quanquan Gu
admin_assigned
2024-10-21T12:18:39.526Z
64c039128e2612254356bba5
/avatars/06cc76feebba0cc80ebb8f4ff86f6d9b.svg
Quanquan Gu
user
thughost
DPLM-2: A Multimodal Diffusion Protein Language Model
Proteins are essential macromolecules defined by their amino acid sequences,
which determine their three-dimensional structures and, consequently, their
functions in all living organisms. Therefore, generative protein modeling
necessitates a multimodal approach to simultaneously model, understand, and
generate both sequences and structures. However, existing methods typically use
separate models for each modality, limiting their ability to capture the
intricate relationships between sequence and structure. This results in
suboptimal performance in tasks that requires joint understanding and
generation of both modalities. In this paper, we introduce DPLM-2, a multimodal
protein foundation model that extends discrete diffusion protein language model
(DPLM) to accommodate both sequences and structures. To enable structural
learning with the language model, 3D coordinates are converted to discrete
tokens using a lookup-free quantization-based tokenizer. By training on both
experimental and high-quality synthetic structures, DPLM-2 learns the joint
distribution of sequence and structure, as well as their marginals and
conditionals. We also implement an efficient warm-up strategy to exploit the
connection between large-scale evolutionary data and structural inductive
biases from pre-trained sequence-based protein language models. Empirical
evaluation shows that DPLM-2 can simultaneously generate highly compatible
amino acid sequences and their corresponding 3D structures eliminating the need
for a two-stage generation approach. Moreover, DPLM-2 demonstrates competitive
performance in various conditional generation tasks, including folding, inverse
folding, and scaffolding with multimodal motif inputs, as well as providing
structure-aware representations for predictive tasks.
DPLM-2 is a multimodal protein foundation model that simultaneously generates amino acid sequences and their 3D structures using a discrete diffusion approach, improving performance in tasks like folding and scaffold prediction.
discrete diffusion protein language model DPLM multimodal protein foundation model 3D coordinates discrete tokens lookup-free quantization-based tokenizer joint distribution sequence-based protein language models warm-up strategy evolutionary data structural inductive biases conditional generation tasks folding inverse folding scaffolding multimodal motif inputs structure-aware representations