Upload folder using huggingface_hub

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 inverse_folding.png filter=lfs diff=lfs merge=lfs -text
+proteinmoe_architecture.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -66,8 +66,8 @@ We fine-tuned a pretrained masked language model using MSA data by concatenating
 | Sequence length             |          2048           |                 12800                  |                   12800                    |
 | Per Device Micro Batch Size |            1            |                   1                    |                     1                      |
 | Precision                   |     Mixed FP32-FP16     |            Mixed FP32-FP16             |              Mixed FP32-FP16               |
-| 1st Stage LR                |       [5e-6,5e-5]       |              [1e-6, 1e-5]              |                    1e-5                    |
-| 1st Stage Num Tokens        |       10 billion        |              100 billion               |                 80 billion                 |
+| LR                |       [5e-6,5e-5]       |              [1e-6, 1e-5]              |                    1e-5                    |
+| Num Tokens        |       10 billion        |              100 billion               |                 80 billion                 |
 
 ### Tokenization