title: Design Proteins Using Large Language Models: Enhancements and
    Comparative Analyses
summary: Pre-trained LLMs have demonstrated substantial capabilities across a range of
  conventional natural language processing (NLP) tasks, such as summarization and
  entity recognition. In this paper, we explore the application of LLMs in the
  generation of high-quality protein sequences. Specifically, we adopt a suite of
  pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and
  gemma-7B4, to produce valid protein sequences. All of these models are publicly
  available.5 Unlike previous work in this field, our approach utilizes a
  relatively small dataset comprising 42,000 distinct human protein sequences. We
  retrain these models to process protein-related data, ensuring the generation
  of biologically feasible protein structures. Our findings demonstrate that even
  with limited data, the adapted models exhibit efficiency comparable to
  established protein-focused models such as ProGen varieties, ProtGPT2, and
  ProLLaMA, which were trained on millions of protein sequences. To validate and
  quantify the performance of our models, we conduct comparative analyses
  employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore,
  we commit to making the trained versions of all four models publicly available,
  fostering greater transparency and collaboration in the field of computational
  biology.
paper_authors._id: 66bc876cae04ebb2b914cb51
paper_authors.name: Kamyar Zeinalipour
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T12:56:37.662Z
paper_authors._id: 64a6e42e2d2c7a251072037f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64a6e42e2d2c7a251072037f/fn5YLrT2OHPMzzs6acZMj.png
paper_authors.fullname: Kamyar Zeinalipour
paper_authors.type: user
paper_authors.user: Kamyar-zeinalipour
paper_authors._id: 66bc876cae04ebb2b914cb52
paper_authors.name: Neda Jamshidi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-15T10:11:34.574Z
paper_authors._id: 65a1445fb85ee9dd15cd2cd8
paper_authors.avatarUrl: /avatars/c91bfd37e55cbdea5158c9bf1b922967.svg
paper_authors.fullname: Neda Jamshidi
paper_authors.type: user
paper_authors.user: Neda-Jamshidi
paper_authors._id: 66bc876cae04ebb2b914cb53
paper_authors.name: Monica Bianchini
paper_authors._id: 66bc876cae04ebb2b914cb54
paper_authors.name: Marco Maggini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T11:11:30.091Z
paper_authors._id: 653e274b78214ec5d5e0e4a3
paper_authors.avatarUrl: /avatars/700ed89b53c1733010d0eb0fb40eccfa.svg
paper_authors.fullname: Marco Maggini
paper_authors.type: user
paper_authors.user: magginim
paper_authors._id: 66bc876cae04ebb2b914cb55
paper_authors.name: Marco Gori
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T11:11:39.646Z
paper_authors._id: 6551cebb19c62ea90f689692
paper_authors.avatarUrl: /avatars/cf10668a1d47c50b54fdd848f63b9c5d.svg
paper_authors.fullname: Marco Gori
paper_authors.type: user
paper_authors.user: marcogori
paper_title: Design Proteins Using Large Language Models: Enhancements and
    Comparative Analyses
paper_summary: Pre-trained LLMs have demonstrated substantial capabilities across a range of
  conventional natural language processing (NLP) tasks, such as summarization and
  entity recognition. In this paper, we explore the application of LLMs in the
  generation of high-quality protein sequences. Specifically, we adopt a suite of
  pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and
  gemma-7B4, to produce valid protein sequences. All of these models are publicly
  available.5 Unlike previous work in this field, our approach utilizes a
  relatively small dataset comprising 42,000 distinct human protein sequences. We
  retrain these models to process protein-related data, ensuring the generation
  of biologically feasible protein structures. Our findings demonstrate that even
  with limited data, the adapted models exhibit efficiency comparable to
  established protein-focused models such as ProGen varieties, ProtGPT2, and
  ProLLaMA, which were trained on millions of protein sequences. To validate and
  quantify the performance of our models, we conduct comparative analyses
  employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore,
  we commit to making the trained versions of all four models publicly available,
  fostering greater transparency and collaboration in the field of computational
  biology.
paper_ai_keywords: LLMs pre-trained LLMs protein sequences Mistral-7B1 Llama-2-7B2 Llama-3-8B3 gemma-7B4 retrain biologically feasible protein structures ProGen ProtGPT2 ProLLaMA pLDDT RMSD TM-score REU