Add pipeline tag, link to paper
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,28 +1,31 @@
|
|
| 1 |
---
|
|
|
|
| 2 |
license: other
|
| 3 |
license_name: bagel-derived
|
| 4 |
license_link: https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
gated: manual
|
| 6 |
-
extra_gated_heading:
|
| 7 |
-
extra_gated_prompt:
|
| 8 |
-
|
| 9 |
-
MODUS 15-modality dataset. By requesting access you agree to use them for
|
| 10 |
research purposes only, consistent with the upstream BAGEL license.
|
| 11 |
extra_gated_fields:
|
| 12 |
Name: text
|
| 13 |
Affiliation: text
|
| 14 |
I will use these weights for research purposes only: checkbox
|
| 15 |
-
extra_gated_button_content:
|
| 16 |
-
tags:
|
| 17 |
-
- multimodal
|
| 18 |
-
- any-to-any
|
| 19 |
-
- modus
|
| 20 |
-
library_name: safetensors
|
| 21 |
---
|
| 22 |
|
| 23 |
# MODUS
|
| 24 |
|
| 25 |
-
MODUS for any-to-any generation (15 aligned modalities).
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
**Inference config (important):**
|
| 28 |
- modality config: `conf/modalities/instruction_16mod_stage2.yaml`
|
|
@@ -32,4 +35,4 @@ MODUS for any-to-any generation (15 aligned modalities).
|
|
| 32 |
- `model.safetensors` β trained weights (bf16)
|
| 33 |
- `ae.safetensors` β VAE (image decode)
|
| 34 |
- `config.json` / `llm_config.json` / `vit_config.json` β architecture config
|
| 35 |
-
- `vocab.json` / `merges.txt` / `tokenizer_config.json` β tokenizer
|
|
|
|
| 1 |
---
|
| 2 |
+
library_name: safetensors
|
| 3 |
license: other
|
| 4 |
license_name: bagel-derived
|
| 5 |
license_link: https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT
|
| 6 |
+
pipeline_tag: any-to-any
|
| 7 |
+
tags:
|
| 8 |
+
- multimodal
|
| 9 |
+
- any-to-any
|
| 10 |
+
- modus
|
| 11 |
gated: manual
|
| 12 |
+
extra_gated_heading: Access MODUS 16-modality weights
|
| 13 |
+
extra_gated_prompt: These weights are derived from ByteDance BAGEL-7B-MoT and fine-tuned
|
| 14 |
+
on the MODUS 15-modality dataset. By requesting access you agree to use them for
|
|
|
|
| 15 |
research purposes only, consistent with the upstream BAGEL license.
|
| 16 |
extra_gated_fields:
|
| 17 |
Name: text
|
| 18 |
Affiliation: text
|
| 19 |
I will use these weights for research purposes only: checkbox
|
| 20 |
+
extra_gated_button_content: Request access
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
---
|
| 22 |
|
| 23 |
# MODUS
|
| 24 |
|
| 25 |
+
MODUS for any-to-any generation (15 aligned modalities) as presented in [MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities](https://huggingface.co/papers/2607.25948).
|
| 26 |
+
|
| 27 |
+
Project page: https://modus-multimodal.epfl.ch/
|
| 28 |
+
Code: https://github.com/EPFL-VILAB/Modus
|
| 29 |
|
| 30 |
**Inference config (important):**
|
| 31 |
- modality config: `conf/modalities/instruction_16mod_stage2.yaml`
|
|
|
|
| 35 |
- `model.safetensors` β trained weights (bf16)
|
| 36 |
- `ae.safetensors` β VAE (image decode)
|
| 37 |
- `config.json` / `llm_config.json` / `vit_config.json` β architecture config
|
| 38 |
+
- `vocab.json` / `merges.txt` / `tokenizer_config.json` β tokenizer
|