Update README.md
Browse files
README.md
CHANGED
@@ -50,14 +50,17 @@ Cephalo provides a robust framework for multimodal interaction and understanding
|
|
50 |
- [lamm-mit/Cephalo-Idefics-2-vision-12b-alpha](https://huggingface.co/lamm-mit/Cephalo-Idefics-2-vision-12b-alpha)
|
51 |
- Merged model, 32+16=48 layers, checkpoint after first epoch. Trained on GPT-4o distilled image-text data from Wikipedia and scientific papers (dataset derivived from both Idefics-2 and GPT-4o distillation of the paper corpus).
|
52 |
|
|
|
|
|
|
|
|
|
53 |
### Mixture-of-Expert models
|
54 |
|
55 |
- [lamm-mit/Cephalo-Phi-3-MoE-vision-128k-3x4b-beta](https://huggingface.co/lamm-mit/Cephalo-Phi-3-MoE-vision-128k-3x4b-beta)
|
56 |
- Mixture-of-expert model based on several smaller Cephalo-Phi-3 models.
|
57 |
|
58 |
-
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/
|
59 |
|
60 |
-
The image shows a summary of model merging approach, constructing larger models from smaller pre-trained building blocks. a, Fine-tuning the base model. b, Constructing the larger, merged model by combining the whole or parts of smaller models. c, Fine-tuning the integrated hybrid, merged, model.
|
61 |
|
62 |
## Etymology and inspiration behind the name Cephalo"
|
63 |
|
|
|
50 |
- [lamm-mit/Cephalo-Idefics-2-vision-12b-alpha](https://huggingface.co/lamm-mit/Cephalo-Idefics-2-vision-12b-alpha)
|
51 |
- Merged model, 32+16=48 layers, checkpoint after first epoch. Trained on GPT-4o distilled image-text data from Wikipedia and scientific papers (dataset derivived from both Idefics-2 and GPT-4o distillation of the paper corpus).
|
52 |
|
53 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/3Nfhn3f3FyK7Zgdg9GKJQ.png)
|
54 |
+
|
55 |
+
The image shows a summary of model merging approach, constructing larger models from smaller pre-trained building blocks. a, Fine-tuning the base model. b, Constructing the larger, merged model by combining the whole or parts of smaller models. c, Fine-tuning the integrated hybrid, merged, model.
|
56 |
+
|
57 |
### Mixture-of-Expert models
|
58 |
|
59 |
- [lamm-mit/Cephalo-Phi-3-MoE-vision-128k-3x4b-beta](https://huggingface.co/lamm-mit/Cephalo-Phi-3-MoE-vision-128k-3x4b-beta)
|
60 |
- Mixture-of-expert model based on several smaller Cephalo-Phi-3 models.
|
61 |
|
62 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/NK9KNOxmnVtn_PzwJtKPR.png)
|
63 |
|
|
|
64 |
|
65 |
## Etymology and inspiration behind the name Cephalo"
|
66 |
|