Large language models generate functional protein sequences across families
Some notable technical information: It is a 1.2bln param model, trained on raw sequences only, can generate full length functional proteins of about 150-200 residues (approx lysozyme size). The generated proteins are very different to native ones (30-40% similarity).<p>The interesting thing about this model is that it also exhibit emergent capabilities. It was trained only on raw sequences but somehow managed to capture information about functionality and solubility of the folded proteins, and then implemented that in the generated sequences.<p>Amino acid sequences are just a bunch of jumbled words if you compared them to English. It usually has to go through folding to form proper "sentences" with meanings. I guess you can compare this to "grammar". This is probably the model managed to learn protein grammar purely by brute force. Now if only we can get a model in the range of 100bln parameters...