title: DPLM-2: A Multimodal Diffusion Protein Language Model
summary: Proteins are essential macromolecules defined by their amino acid sequences,
  which determine their three-dimensional structures and, consequently, their
  functions in all living organisms. Therefore, generative protein modeling
  necessitates a multimodal approach to simultaneously model, understand, and
  generate both sequences and structures. However, existing methods typically use
  separate models for each modality, limiting their ability to capture the
  intricate relationships between sequence and structure. This results in
  suboptimal performance in tasks that requires joint understanding and
  generation of both modalities. In this paper, we introduce DPLM-2, a multimodal
  protein foundation model that extends discrete diffusion protein language model
  (DPLM) to accommodate both sequences and structures. To enable structural
  learning with the language model, 3D coordinates are converted to discrete
  tokens using a lookup-free quantization-based tokenizer. By training on both
  experimental and high-quality synthetic structures, DPLM-2 learns the joint
  distribution of sequence and structure, as well as their marginals and
  conditionals. We also implement an efficient warm-up strategy to exploit the
  connection between large-scale evolutionary data and structural inductive
  biases from pre-trained sequence-based protein language models. Empirical
  evaluation shows that DPLM-2 can simultaneously generate highly compatible
  amino acid sequences and their corresponding 3D structures eliminating the need
  for a two-stage generation approach. Moreover, DPLM-2 demonstrates competitive
  performance in various conditional generation tasks, including folding, inverse
  folding, and scaffolding with multimodal motif inputs, as well as providing
  structure-aware representations for predictive tasks.
paper_authors._id: 6715fdfc6757313a2bca5806
paper_authors.name: Xinyou Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-22T08:01:14.824Z
paper_authors._id: 64d0938b37a8b7adaf8c8219
paper_authors.avatarUrl: /avatars/d8ea6a7fe938db2280b828aaeb41dcaf.svg
paper_authors.fullname: wang
paper_authors.type: user
paper_authors.user: airkingbd
paper_authors._id: 6715fdfc6757313a2bca5807
paper_authors.name: Zaixiang Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T12:31:01.433Z
paper_authors._id: 6347faa36d7f1d10f5ecdc18
paper_authors.avatarUrl: /avatars/0fea067fa7bc8c4a797268b68a868756.svg
paper_authors.fullname: zzheng
paper_authors.type: user
paper_authors.user: zzheng
paper_authors._id: 6715fdfc6757313a2bca5808
paper_authors.name: Fei Ye
paper_authors._id: 6715fdfc6757313a2bca5809
paper_authors.name: Dongyu Xue
paper_authors._id: 6715fdfc6757313a2bca580a
paper_authors.name: Shujian Huang
paper_authors._id: 6715fdfc6757313a2bca580b
paper_authors.name: Quanquan Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:18:39.526Z
paper_authors._id: 64c039128e2612254356bba5
paper_authors.avatarUrl: /avatars/06cc76feebba0cc80ebb8f4ff86f6d9b.svg
paper_authors.fullname: Quanquan Gu
paper_authors.type: user
paper_authors.user: thughost
paper_title: DPLM-2: A Multimodal Diffusion Protein Language Model
paper_summary: Proteins are essential macromolecules defined by their amino acid sequences,
  which determine their three-dimensional structures and, consequently, their
  functions in all living organisms. Therefore, generative protein modeling
  necessitates a multimodal approach to simultaneously model, understand, and
  generate both sequences and structures. However, existing methods typically use
  separate models for each modality, limiting their ability to capture the
  intricate relationships between sequence and structure. This results in
  suboptimal performance in tasks that requires joint understanding and
  generation of both modalities. In this paper, we introduce DPLM-2, a multimodal
  protein foundation model that extends discrete diffusion protein language model
  (DPLM) to accommodate both sequences and structures. To enable structural
  learning with the language model, 3D coordinates are converted to discrete
  tokens using a lookup-free quantization-based tokenizer. By training on both
  experimental and high-quality synthetic structures, DPLM-2 learns the joint
  distribution of sequence and structure, as well as their marginals and
  conditionals. We also implement an efficient warm-up strategy to exploit the
  connection between large-scale evolutionary data and structural inductive
  biases from pre-trained sequence-based protein language models. Empirical
  evaluation shows that DPLM-2 can simultaneously generate highly compatible
  amino acid sequences and their corresponding 3D structures eliminating the need
  for a two-stage generation approach. Moreover, DPLM-2 demonstrates competitive
  performance in various conditional generation tasks, including folding, inverse
  folding, and scaffolding with multimodal motif inputs, as well as providing
  structure-aware representations for predictive tasks.
paper_ai_summary: DPLM-2 is a multimodal protein foundation model that simultaneously generates amino acid sequences and their 3D structures using a discrete diffusion approach, improving performance in tasks like folding and scaffold prediction.
paper_ai_keywords: discrete diffusion protein language model DPLM multimodal protein foundation model 3D coordinates discrete tokens lookup-free quantization-based tokenizer joint distribution sequence-based protein language models warm-up strategy evolutionary data structural inductive biases conditional generation tasks folding inverse folding scaffolding multimodal motif inputs structure-aware representations