Update README.md
Browse files
README.md
CHANGED
@@ -1,11 +1,11 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
datasets:
|
4 |
-
-
|
5 |
-
-
|
6 |
-
-
|
7 |
- DataProvenanceInitiative/Commercially-Verified-Licenses
|
8 |
-
-
|
9 |
language:
|
10 |
- afr
|
11 |
- amh
|
@@ -121,19 +121,19 @@ metrics:
|
|
121 |
|
122 |
> The Aya model is a massively multilingual generative language model that follows instructions in 101 languages.
|
123 |
> Aya outperforms [mT0](https://huggingface.co/bigscience/mt0-xxl) and [BLOOMZ](https://huggingface.co/bigscience/bloomz) a wide variety of automatic and human evaluations despite covering double the number of languages.
|
124 |
-
> The Aya model is trained using [xP3x](https://huggingface.co/datasets/
|
125 |
> We release the checkpoints under a Apache-2.0 license to further our mission of multilingual technologies empowering a
|
126 |
> multilingual world.
|
127 |
|
128 |
-
- **Developed by:** [Cohere
|
129 |
- **Model type:** a Transformer style autoregressive massively multilingual language model.
|
130 |
- **Paper**: [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://arxiv.org/abs/2402.07827)
|
131 |
-
- **Point of Contact**: Cohere
|
132 |
- **Languages**: Refer to the list of languages in the `language` section of this model card.
|
133 |
- **License**: Apache-2.0
|
134 |
-
- **Model**: [Aya-101](https://huggingface.co/
|
135 |
- **Model Size**: 13 billion parameters
|
136 |
-
- **Datasets**: [xP3x](https://huggingface.co/datasets/
|
137 |
|
138 |
## Use
|
139 |
|
@@ -141,7 +141,7 @@ metrics:
|
|
141 |
# pip install -q transformers
|
142 |
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
|
143 |
|
144 |
-
checkpoint = "
|
145 |
|
146 |
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
147 |
aya_model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)
|
@@ -174,9 +174,9 @@ print(tokenizer.decode(hin_outputs[0]))
|
|
174 |
|
175 |
The Aya model is trained on the following datasets:
|
176 |
|
177 |
-
- [xP3x](https://huggingface.co/datasets/
|
178 |
-
- [Aya Dataset](https://huggingface.co/datasets/
|
179 |
-
- [Aya Collection](https://huggingface.co/datasets/
|
180 |
- [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses)
|
181 |
- ShareGPT-Command
|
182 |
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
datasets:
|
4 |
+
- CohereLabs/xP3x
|
5 |
+
- CohereLabs/aya_dataset
|
6 |
+
- CohereLabs/aya_collection
|
7 |
- DataProvenanceInitiative/Commercially-Verified-Licenses
|
8 |
+
- CohereLabs/aya_evaluation_suite
|
9 |
language:
|
10 |
- afr
|
11 |
- amh
|
|
|
121 |
|
122 |
> The Aya model is a massively multilingual generative language model that follows instructions in 101 languages.
|
123 |
> Aya outperforms [mT0](https://huggingface.co/bigscience/mt0-xxl) and [BLOOMZ](https://huggingface.co/bigscience/bloomz) a wide variety of automatic and human evaluations despite covering double the number of languages.
|
124 |
+
> The Aya model is trained using [xP3x](https://huggingface.co/datasets/CohereLabs/xP3x), [Aya Dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset), [Aya Collection](https://huggingface.co/datasets/CohereForAI/aya_collection), a subset of [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses) and ShareGPT-Command.
|
125 |
> We release the checkpoints under a Apache-2.0 license to further our mission of multilingual technologies empowering a
|
126 |
> multilingual world.
|
127 |
|
128 |
+
- **Developed by:** [Cohere Labs](https://cohere.for.ai)
|
129 |
- **Model type:** a Transformer style autoregressive massively multilingual language model.
|
130 |
- **Paper**: [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://arxiv.org/abs/2402.07827)
|
131 |
+
- **Point of Contact**: [Cohere Labs](https://cohere.for.ai)
|
132 |
- **Languages**: Refer to the list of languages in the `language` section of this model card.
|
133 |
- **License**: Apache-2.0
|
134 |
+
- **Model**: [Aya-101](https://huggingface.co/CohereLabs/aya-101)
|
135 |
- **Model Size**: 13 billion parameters
|
136 |
+
- **Datasets**: [xP3x](https://huggingface.co/datasets/CohereLabs/xP3x), [Aya Dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset), [Aya Collection](https://huggingface.co/datasets/CohereLabs/aya_collection), [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses), ShareGPT-Command.
|
137 |
|
138 |
## Use
|
139 |
|
|
|
141 |
# pip install -q transformers
|
142 |
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
|
143 |
|
144 |
+
checkpoint = "CohereLabs/aya-101"
|
145 |
|
146 |
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
147 |
aya_model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)
|
|
|
174 |
|
175 |
The Aya model is trained on the following datasets:
|
176 |
|
177 |
+
- [xP3x](https://huggingface.co/datasets/CohereLabs/xP3x)
|
178 |
+
- [Aya Dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset)
|
179 |
+
- [Aya Collection](https://huggingface.co/datasets/CohereLabs/aya_collection)
|
180 |
- [DataProvenance collection](https://huggingface.co/datasets/DataProvenanceInitiative/Commercially-Verified-Licenses)
|
181 |
- ShareGPT-Command
|
182 |
|