title: A Large Encoder-Decoder Family of Foundation Models For Chemical
    Language
summary: Large-scale pre-training methodologies for chemical language models represent
  a breakthrough in cheminformatics. These methods excel in tasks such as
  property prediction and molecule generation by learning contextualized
  representations of input tokens through self-supervised learning on large
  unlabeled corpora. Typically, this involves pre-training on unlabeled data
  followed by fine-tuning on specific tasks, reducing dependence on annotated
  datasets and broadening chemical language representation understanding. This
  paper introduces a large encoder-decoder chemical foundation models pre-trained
  on a curated dataset of 91 million SMILES samples sourced from PubChem, which
  is equivalent to 4 billion of molecular tokens. The proposed foundation model
  supports different complex tasks, including quantum property prediction, and
  offer flexibility with two main variants (289M and 8times289M). Our
  experiments across multiple benchmark datasets validate the capacity of the
  proposed model in providing state-of-the-art results for different tasks. We
  also provide a preliminary assessment of the compositionality of the embedding
  space as a prerequisite for the reasoning tasks. We demonstrate that the
  produced latent space is separable compared to the state-of-the-art with
  few-shot learning capabilities.
paper_authors._id: 66aa32f0a7081716091aab99
paper_authors.name: Eduardo Soares
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-31T12:49:53.838Z
paper_authors._id: 66a2816e5cdb6fc163826d8a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66a2816e5cdb6fc163826d8a/RNLmcR_eVyT0OfYfb-LMM.jpeg
paper_authors.fullname: Eduardo Soares
paper_authors.type: user
paper_authors.user: eduardosoares99
paper_authors._id: 66aa32f0a7081716091aab9a
paper_authors.name: Victor Shirasuna
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-08-30T13:33:00.087Z
paper_authors._id: 66d1c9a6b003aeb6d16679b5
paper_authors.avatarUrl: /avatars/9139af48fc764505e62a12769727d02c.svg
paper_authors.fullname: Victor Yukio Shirasuna
paper_authors.type: user
paper_authors.user: vshirasuna
paper_authors._id: 66aa32f0a7081716091aab9b
paper_authors.name: Emilio Vital Brazil
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-02T07:34:56.956Z
paper_authors._id: 63a202bb931d2c13ad7c721e
paper_authors.avatarUrl: /avatars/a85f844d374c6f60782607e7dfca378b.svg
paper_authors.fullname: Emilio Ashton Vital Brazil
paper_authors.type: user
paper_authors.user: emiliobrazil
paper_authors._id: 66aa32f0a7081716091aab9c
paper_authors.name: Renato Cerqueira
paper_authors._id: 66aa32f0a7081716091aab9d
paper_authors.name: Dmitry Zubarev
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-31T12:49:53.838Z
paper_authors._id: 635868fc70c0738527a8f0b8
paper_authors.avatarUrl: /avatars/9d633c23661b97ea4122439a2622c8a8.svg
paper_authors.fullname: Dmitry Zubarev
paper_authors.type: user
paper_authors.user: dzubarev
paper_authors._id: 66aa32f0a7081716091aab9e
paper_authors.name: Kristin Schmidt
paper_title: A Large Encoder-Decoder Family of Foundation Models For Chemical
    Language
paper_summary: Large-scale pre-training methodologies for chemical language models represent
  a breakthrough in cheminformatics. These methods excel in tasks such as
  property prediction and molecule generation by learning contextualized
  representations of input tokens through self-supervised learning on large
  unlabeled corpora. Typically, this involves pre-training on unlabeled data
  followed by fine-tuning on specific tasks, reducing dependence on annotated
  datasets and broadening chemical language representation understanding. This
  paper introduces a large encoder-decoder chemical foundation models pre-trained
  on a curated dataset of 91 million SMILES samples sourced from PubChem, which
  is equivalent to 4 billion of molecular tokens. The proposed foundation model
  supports different complex tasks, including quantum property prediction, and
  offer flexibility with two main variants (289M and 8times289M). Our
  experiments across multiple benchmark datasets validate the capacity of the
  proposed model in providing state-of-the-art results for different tasks. We
  also provide a preliminary assessment of the compositionality of the embedding
  space as a prerequisite for the reasoning tasks. We demonstrate that the
  produced latent space is separable compared to the state-of-the-art with
  few-shot learning capabilities.
paper_ai_summary: A large-scale pre-trained encoder-decoder chemical language model achieves state-of-the-art performance across various tasks by leveraging a vast dataset of molecular structures, exhibiting strong few-shot learning capabilities.
paper_ai_keywords: chemical language models large-scale pre-training self-supervised learning SMILES encoder-decoder model quantum property prediction few-shot learning latent space embedding space