FoldToken: Learning Protein Language via Vector Quantization and Beyond
<|reference_start|>FoldToken: Learning Protein Language via Vector Quantization and Beyond: Is there a foreign language describing protein sequences and structures simultaneously? Protein structures, represented by continuous 3D points, have long posed a challenge due to the contrasting modeling paradigms of discrete sequences. We introduce \textbf{FoldTokenizer} to represent protein sequence-structure as discrete symbols. This innovative approach involves projecting residue types and structures into a discrete space, guided by a reconstruction loss for information preservation. We refer to the learned discrete symbols as \textbf{FoldToken}, and the sequence of FoldTokens serves as a new protein language, transforming the protein sequence-structure into a unified modality. We apply the created protein language on general backbone inpainting and antibody design tasks, building the first GPT-style model (\textbf{FoldGPT}) for sequence-structure co-generation with promising results. Key to our success is the substantial enhancement of the vector quantization module, Soft Conditional Vector Quantization (\textbf{SoftCVQ}).<|reference_end|>
@article{gao2024foldtoken:,
    title={FoldToken: Learning Protein Language via Vector Quantization and Beyond},
    author={Zhangyang Gao, Cheng Tan, Jue Wang, Yufei Huang, Lirong Wu, Stan Z. Li},
    journal={arXiv preprint arXiv:2403.09673},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.09673},
    primaryClass={q-bio.BM cs.AI cs.LG}
}