title: MolXPT: Wrapping Molecules with Text for Generative Pre-training
summary: Generative pre-trained Transformer (GPT) has demonstrates its great success
  in natural language processing and related techniques have been adapted into
  molecular modeling. Considering that text is the most important record for
  scientific discovery, in this paper, we propose MolXPT, a unified language
  model of text and molecules pre-trained on SMILES (a sequence representation of
  molecules) wrapped by text. Briefly, we detect the molecule names in each
  sequence and replace them to the corresponding SMILES. In this way, the SMILES
  could leverage the information from surrounding text, and vice versa. The above
  wrapped sequences, text sequences from PubMed and SMILES sequences from PubChem
  are all fed into a language model for pre-training. Experimental results
  demonstrate that MolXPT outperforms strong baselines of molecular property
  prediction on MoleculeNet, performs comparably to the best model in
  text-molecule translation while using less than half of its parameters, and
  enables zero-shot molecular generation without finetuning.
paper_authors._id: 6466faedcdea48159aa5af04
paper_authors.name: Zequn Liu
paper_authors._id: 6466faedcdea48159aa5af05
paper_authors.name: Wei Zhang
paper_authors._id: 6466faedcdea48159aa5af06
paper_authors.name: Yingce Xia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T19:04:39.990Z
paper_authors._id: 644a2eddd9a3ae834104c654
paper_authors.avatarUrl: /avatars/a52dbbd6ef1dca5ba9a94d0f564204de.svg
paper_authors.fullname: Yingce Xia
paper_authors.type: user
paper_authors.user: xyc1207
paper_authors._id: 6466faedcdea48159aa5af07
paper_authors.name: Lijun Wu
paper_authors._id: 6466faedcdea48159aa5af08
paper_authors.name: Shufang Xie
paper_authors._id: 6466faedcdea48159aa5af09
paper_authors.name: Tao Qin
paper_authors._id: 6466faedcdea48159aa5af0a
paper_authors.name: Ming Zhang
paper_authors._id: 6466faedcdea48159aa5af0b
paper_authors.name: Tie-Yan Liu
paper_title: MolXPT: Wrapping Molecules with Text for Generative Pre-training
paper_summary: Generative pre-trained Transformer (GPT) has demonstrates its great success
  in natural language processing and related techniques have been adapted into
  molecular modeling. Considering that text is the most important record for
  scientific discovery, in this paper, we propose MolXPT, a unified language
  model of text and molecules pre-trained on SMILES (a sequence representation of
  molecules) wrapped by text. Briefly, we detect the molecule names in each
  sequence and replace them to the corresponding SMILES. In this way, the SMILES
  could leverage the information from surrounding text, and vice versa. The above
  wrapped sequences, text sequences from PubMed and SMILES sequences from PubChem
  are all fed into a language model for pre-training. Experimental results
  demonstrate that MolXPT outperforms strong baselines of molecular property
  prediction on MoleculeNet, performs comparably to the best model in
  text-molecule translation while using less than half of its parameters, and
  enables zero-shot molecular generation without finetuning.
paper_ai_summary: MolXPT, a unified text and molecular language model pre-trained on wrapped SMILES and text data, outperforms baselines in molecular property prediction, matches top models in text-molecule translation with fewer parameters, and supports zero-shot molecular generation.
paper_ai_keywords: Generative pre-trained Transformer MolXPT SMILES language model PubMed PubChem molecular property prediction MoleculeNet text-molecule translation zero-shot molecular generation