Update README.md

README.md

@@ -4,7 +4,7 @@ license: other
 
 # AIDO.Protein-RAG-16B 
 
-AIDO.RAGProtein-16B is a multimodal protein language model that integrates Multiple Sequence Alignment (MSA) and structural data, building upon the [AIDO.Protein-16B](https://huggingface.co/genbio-ai/AIDO.Protein-16B) foundation. The training process comprises three main stages:
+AIDO.Protein-RAG-16B is a multimodal protein language model that integrates Multiple Sequence Alignment (MSA) and structural data, building upon the [AIDO.Protein-16B](https://huggingface.co/genbio-ai/AIDO.Protein-16B) foundation. The training process comprises three main stages:
 
 1. 2D RoPE encoding fine-tuning
 2. Initial training on 100 billion tokens from UniRef50/UniClust30 MSA data
@@ -12,7 +12,7 @@ AIDO.RAGProtein-16B is a multimodal protein language model that integrates Multi
 
 ## Model Architecture Details
 
-AIDO.RAGProtein-16B employs a transformer encoder-only architecture featuring sparse Mixture-of-Experts (MoE) layers that replace dense MLP layers in each transformer block. Utilizing single amino acid tokenization and optimized through masked language modeling (MLM), the model activates 2 experts per token via top-2 routing mechanisms.
+AIDO.Protein-RAG-16B employs a transformer encoder-only architecture featuring sparse Mixture-of-Experts (MoE) layers that replace dense MLP layers in each transformer block. Utilizing single amino acid tokenization and optimized through masked language modeling (MLM), the model activates 2 experts per token via top-2 routing mechanisms.
 
 <center><img src="proteinmoe_architecture.png" alt="An Overview of AIDO.Protein" style="width:70%; height:auto;" /></center>
 
@@ -29,9 +29,9 @@ More architecture details are shown below:
 | Vocab Size              |  44   |
 | Context Length          | 2048  |
 
-## Pre-training of AIDO.RAGProtein-16B
+## Pre-training of AIDO.Protein-RAG-16B
 
-Here we briefly introduce the details of pre-training of AIDO.RAGProtein-16B. Mainly divided into three stages: (1) 1D -> 2D RoPE encoding finetuning; (2) UniRef50/Uniclust30 MSA finetuning; (3) AlphaFold Database MSA & Structure tokens finetuning.
+Here we briefly introduce the details of pre-training of AIDO.Protein-RAG-16B. Mainly divided into three stages: (1) 1D -> 2D RoPE encoding finetuning; (2) UniRef50/Uniclust30 MSA finetuning; (3) AlphaFold Database MSA & Structure tokens finetuning.
 
 ### Data
 
@@ -202,7 +202,7 @@ print(logits.shape)
 
 # Citation
 
-Please cite AIDO.RAGProtein-16B using the following BibTex code:
+Please cite AIDO.Protein-RAG-16B using the following BibTex code:
 
 ```
 @inproceedings{sun_mixture_2024,