title: LLaMo: Large Language Model-based Molecular Graph Assistant
summary: Large Language Models (LLMs) have demonstrated remarkable generalization and
  instruction-following capabilities with instruction tuning. The advancements in
  LLMs and instruction tuning have led to the development of Large
  Vision-Language Models (LVLMs). However, the competency of the LLMs and
  instruction tuning have been less explored in the molecular domain. Thus, we
  propose LLaMo: Large Language Model-based Molecular graph assistant, which is
  an end-to-end trained large molecular graph-language model. To bridge the
  discrepancy between the language and graph modalities, we present the
  multi-level graph projector that transforms graph representations into graph
  tokens by abstracting the output representations of each GNN layer and motif
  representations with the cross-attention mechanism. We also introduce
  machine-generated molecular graph instruction data to instruction-tune the
  large molecular graph-language model for general-purpose molecule and language
  understanding. Our extensive experiments demonstrate that LLaMo shows the best
  performance on diverse tasks, such as molecular description generation,
  property prediction, and IUPAC name prediction. The code of LLaMo is available
  at https://github.com/mlvlab/LLaMo.
paper_authors._id: 672aed3f44b2453edfdbeafd
paper_authors.name: Jinyoung Park
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:37:04.989Z
paper_authors._id: 64b6eae88ba7d6c922c0434a
paper_authors.avatarUrl: /avatars/6adac8242106ab12abeaa3584346c0cd.svg
paper_authors.fullname: Jinyoung Park
paper_authors.type: user
paper_authors.user: jinypark
paper_authors._id: 672aed3f44b2453edfdbeafe
paper_authors.name: Minseong Bae
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-02T10:19:29.383Z
paper_authors._id: 64549b4cc13cdb83f1100b94
paper_authors.avatarUrl: /avatars/4e809bed509e6ce2dd5532730af81a12.svg
paper_authors.fullname: Minseong Bae
paper_authors.type: user
paper_authors.user: KyleBae1017
paper_authors._id: 672aed3f44b2453edfdbeaff
paper_authors.name: Dohwan Ko
paper_authors._id: 672aed3f44b2453edfdbeb00
paper_authors.name: Hyunwoo J. Kim
paper_title: LLaMo: Large Language Model-based Molecular Graph Assistant
paper_summary: Large Language Models (LLMs) have demonstrated remarkable generalization and
  instruction-following capabilities with instruction tuning. The advancements in
  LLMs and instruction tuning have led to the development of Large
  Vision-Language Models (LVLMs). However, the competency of the LLMs and
  instruction tuning have been less explored in the molecular domain. Thus, we
  propose LLaMo: Large Language Model-based Molecular graph assistant, which is
  an end-to-end trained large molecular graph-language model. To bridge the
  discrepancy between the language and graph modalities, we present the
  multi-level graph projector that transforms graph representations into graph
  tokens by abstracting the output representations of each GNN layer and motif
  representations with the cross-attention mechanism. We also introduce
  machine-generated molecular graph instruction data to instruction-tune the
  large molecular graph-language model for general-purpose molecule and language
  understanding. Our extensive experiments demonstrate that LLaMo shows the best
  performance on diverse tasks, such as molecular description generation,
  property prediction, and IUPAC name prediction. The code of LLaMo is available
  at https://github.com/mlvlab/LLaMo.
paper_ai_keywords: LLaMo Large Language Model-based Molecular graph assistant Large Vision-Language Models (LVLMs) molecular graph-language model multi-level graph projector graph representations graph tokens output representations GNN layer motif representations cross-attention mechanism machine-generated molecular graph instruction data instruction-tune molecular description generation property prediction IUPAC name prediction