Design Proteins Using Large Language Models: Enhancements and
  Comparative Analyses
Pre-trained LLMs have demonstrated substantial capabilities across a range of
conventional natural language processing (NLP) tasks, such as summarization and
entity recognition. In this paper, we explore the application of LLMs in the
generation of high-quality protein sequences. Specifically, we adopt a suite of
pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and
gemma-7B4, to produce valid protein sequences. All of these models are publicly
available.5 Unlike previous work in this field, our approach utilizes a
relatively small dataset comprising 42,000 distinct human protein sequences. We
retrain these models to process protein-related data, ensuring the generation
of biologically feasible protein structures. Our findings demonstrate that even
with limited data, the adapted models exhibit efficiency comparable to
established protein-focused models such as ProGen varieties, ProtGPT2, and
ProLLaMA, which were trained on millions of protein sequences. To validate and
quantify the performance of our models, we conduct comparative analyses
employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore,
we commit to making the trained versions of all four models publicly available,
fostering greater transparency and collaboration in the field of computational
biology.
66bc876cae04ebb2b914cb51
Kamyar Zeinalipour
admin_assigned
2024-08-14T12:56:37.662Z
64a6e42e2d2c7a251072037f
https://cdn-avatars.huggingface.co/v1/production/uploads/64a6e42e2d2c7a251072037f/fn5YLrT2OHPMzzs6acZMj.png
Kamyar Zeinalipour
user
Kamyar-zeinalipour
66bc876cae04ebb2b914cb52
Neda Jamshidi
claimed_verified
2024-08-15T10:11:34.574Z
65a1445fb85ee9dd15cd2cd8
/avatars/c91bfd37e55cbdea5158c9bf1b922967.svg
Neda Jamshidi
user
Neda-Jamshidi
66bc876cae04ebb2b914cb53
Monica Bianchini
66bc876cae04ebb2b914cb54
Marco Maggini
admin_assigned
2024-08-14T11:11:30.091Z
653e274b78214ec5d5e0e4a3
/avatars/700ed89b53c1733010d0eb0fb40eccfa.svg
Marco Maggini
user
magginim
66bc876cae04ebb2b914cb55
Marco Gori
admin_assigned
2024-08-14T11:11:39.646Z
6551cebb19c62ea90f689692
/avatars/cf10668a1d47c50b54fdd848f63b9c5d.svg
Marco Gori
user
marcogori
Design Proteins Using Large Language Models: Enhancements and
  Comparative Analyses
Pre-trained LLMs have demonstrated substantial capabilities across a range of
conventional natural language processing (NLP) tasks, such as summarization and
entity recognition. In this paper, we explore the application of LLMs in the
generation of high-quality protein sequences. Specifically, we adopt a suite of
pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and
gemma-7B4, to produce valid protein sequences. All of these models are publicly
available.5 Unlike previous work in this field, our approach utilizes a
relatively small dataset comprising 42,000 distinct human protein sequences. We
retrain these models to process protein-related data, ensuring the generation
of biologically feasible protein structures. Our findings demonstrate that even
with limited data, the adapted models exhibit efficiency comparable to
established protein-focused models such as ProGen varieties, ProtGPT2, and
ProLLaMA, which were trained on millions of protein sequences. To validate and
quantify the performance of our models, we conduct comparative analyses
employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore,
we commit to making the trained versions of all four models publicly available,
fostering greater transparency and collaboration in the field of computational
biology.
Pre-trained LLMs can generate high-quality protein sequences with limited data, achieving performance comparable to specialized protein-focused models.
LLMs Mistral-7B Llama-2-7B Llama-3-8B gemma-7B protein sequences pLDDT RMSD TM-score REU