transformers safetensors llama text-generation text-generation-inference unsloth trl sft tango en es dataset:spanish-ir/messirve dataset:tatakof/messi_mod-v0.0.2 arxiv:2409.05994 arxiv:2410.01257 license:llama3.1 autotrain_compatible endpoints_compatible region:us
---
base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
datasets:
- spanish-ir/messirve
- tatakof/messi_mod-v0.0.2
language:
- en
- es
library_name: transformers
license: llama3.1
pipeline_tag: text-generation
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
- tango
---

# Uploaded  model

- **Developed by:** sandbox-ai
- **License:** apache-2.0
- **Finetuned from model :** unsloth/meta-llama-3.1-8b-bnb-4bit


## Evaluation Metrics
|Task                                                                                                    |Name               |Description                                                            |Language|Metric        |Task type                                 |
|--------------------------------------------------------------------------------------------------------|-------------------|-----------------------------------------------------------------------|--------|--------------|------------------------------------------|
|[AQuAS](https://huggingface.co/datasets/IIC/AQuAS)                                                      |AQuAS              |Abstractive Question-Answering in Spanish                              |ES      |sas_encoder   |Abstractive QA                            |
|[ARC_ca](https://huggingface.co/datasets/projecte-aina/arc_ca)                                          |ARC_ca             |Grade-school level science questions in Catalan                        |CA      |acc           |Multi choice QA                           |
|[BEC2016eu](https://huggingface.co/datasets/orai-nlp/basqueGLUE)                                        |BEC2016eu          |Basque Election Campaign 2016 Opinion Dataset                          |EU      |f1            |Sentiment Analysis                        |
|[Belebele Glg](https://huggingface.co/datasets/facebook/belebele)                                       |Belebele Glg       |Reading Comprehension in Galician                                      |GL      |acc           |Reading Comprehension                     |
|[BertaQA](https://huggingface.co/datasets/HiTZ/BertaQA)                                                 |BertaQA            |Trivia dataset with global and local questions about the Basque Country|EU      |acc           |Multi choice QA                           |
|[BHTCv2](https://huggingface.co/datasets/orai-nlp/basqueGLUE)                                           |BHTCv2             |Topic Classification of News Headlines in Basque                       |EU      |f1            |Classification, Topic Classification      |
|[caBREU](https://huggingface.co/datasets/projecte-aina/caBreu)                                          |caBREU             |Article Summarization in Catalan                                       |CA      |bleu          |Summarization                             |
|[CatalanQA](https://huggingface.co/datasets/projecte-aina/catalanqa)                                    |CatalanQA          |Extractive QA in Catalan                                               |CA      |f1            |Extractive QA                             |
|[CatCoLA](https://huggingface.co/datasets/nbel/CatCoLA)                                                 |CatCoLA            |Linguistic Acceptability in Catalan                                    |CA      |mcc           |Linguistic Acceptability                  |
|[ClinDiagnosES](https://huggingface.co/datasets/LenguajeNaturalAI/ClinDiagnosES)                        |ClinDiagnosES      |Diagnosis of clinical cases in Spanish                                 |ES      |sas_encoder   |Open QA                                   |
|[ClinTreatES](https://huggingface.co/datasets/LenguajeNaturalAI/ClinTreatES)                            |ClinTreatES        |Treatment for clinical cases in Spanish                                |ES      |sas_encoder   |Open QA                                   |
|[COPA_ca](https://huggingface.co/datasets/projecte-aina/COPA-ca)                                        |COPA_ca            |Choice Of Plausible Alternatives in Catalan                            |CA      |acc           |Reasoning                                 |
|[CoQCat](https://huggingface.co/datasets/projecte-aina/CoQCat)                                          |CoQCat             |Conversational Question Answering in Catalan                           |CA      |f1            |Extractive QA                             |
|[Crows Pairs Spanish](https://huggingface.co/datasets/multilingual-crows-pairs/multilingual-crows-pairs)|Crows Pairs Spanish|Bias evaluation using stereotypes                                      |ES      |pct_stereotype|Bias Detection                            |
|[EpecKorrefBin](https://huggingface.co/datasets/orai-nlp/basqueGLUE)                                    |EpecKorrefBin      |Coreference resolution in Basque                                       |EU      |acc           |Coreference Resolution, Textual Entailment|
|[EsCoLA](https://huggingface.co/datasets/nbel/EsCoLA)                                                   |EsCoLA             |Spanish Corpus of Linguistic Acceptability                             |ES      |mcc           |Linguistic Acceptability                  |
|[EusExams](https://huggingface.co/datasets/HiTZ/EusExams)                                               |EusExams           |Public Service examinations questions in Basque                        |EU      |acc           |Multi choice QA                           |
|[EusProficiency](https://huggingface.co/datasets/HiTZ/EusProficiency)                                   |EusProficiency     |C1-level proficiency questions in Basque                               |EU      |acc           |Multi choice QA                           |
|[EusReading](https://huggingface.co/datasets/HiTZ/EusReading)                                           |EusReading         |EGA exams reading comprehension in Basque                              |EU      |acc           |Multi choice QA                           |
|[EusTrivia](https://huggingface.co/datasets/HiTZ/EusTrivia)                                             |EusTrivia          |Trivia questions in Basque                                             |EU      |acc           |Multi choice QA                           |
|[Fake News ES](https://huggingface.co/datasets/mariagrandury/fake_news_corpus_spanish)                  |Fake News ES       |Fake News Detection in Spanish                                         |ES      |acc           |Classification                            |
|[GalCoLA](https://huggingface.co/datasets/proxectonos/galcola)                                          |GalCoLA            |Galician Corpus of Linguistic Acceptability                            |GL      |mcc           |Linguistic Acceptability                  |
|[HumorQA](https://huggingface.co/datasets/LenguajeNaturalAI/HumorQA)                                    |HumorQA            |White humour joke classification                                       |ES      |acc           |Classification                            |
|[MGSM_ca](https://huggingface.co/datasets/projecte-aina/mgsm_ca)                                        |MGSM_ca            |Grade-school math problems in Catalan                                  |CA      |exact_match   |Math Reasoning                            |
|[MGSM_es](https://huggingface.co/datasets/juletxara/mgsm)                                               |MGSM_es            |Grade-school math problems in Spanish                                  |ES      |exact_match   |Math Reasoning                            |
|[MGSM_eu](https://huggingface.co/datasets/HiTZ/MGSM-eu)                                                 |MGSM_eu            |Grade-school math problems in Basque                                   |EU      |exact_match   |Math Reasoning                            |
|[MGSM_gl](https://huggingface.co/datasets/proxectonos/mgsm_gl)                                          |MGSM_gl            |Grade-school math problems in Galician                                 |GL      |exact_match   |Math Reasoning                            |
|[NoticIA](https://huggingface.co/datasets/Iker/NoticIA)                                                 |NoticIA            |A Clickbait Article Summarization Dataset in Spanish                   |ES      |rouge1        |Summarization                             |
|[OffendES](https://huggingface.co/datasets/SINAI/OffendES)                                              |OffendES           |Clasificación de comentarios ofensivos en español                      |ES      |acc           |Classification                            |
|[OpenBookQA_ca](https://huggingface.co/datasets/projecte-aina/openbookqa_ca)                            |OpenBookQA_ca      |Multi-step reasoning QA in Catalan                                     |CA      |acc           |Reasoning                                 |
|[OpenBookQA_gl](https://huggingface.co/datasets/proxectonos/openbookqa_gl)                              |OpenBookQA_gl      |Multi-step reasoning QA in Galician                                    |GL      |acc           |Reasoning                                 |
|[Parafraseja](https://huggingface.co/datasets/projecte-aina/Parafraseja)                                |Parafraseja        |Paraphrase identification in Catalan                                   |CA      |acc           |Paraphrasing                              |
|[ParafrasesGL](https://huggingface.co/datasets/proxectonos/parafrases_gl)                               |ParafrasesGL       |Paraphrase identification in Galician                                  |GL      |acc           |Paraphrasing                              |
|[PAWS_ca](https://huggingface.co/datasets/projecte-aina/PAWS-ca)                                        |PAWS_ca            |Paraphrase Adversaries from Word Scrambling in Catalan                 |CA      |acc           |Paraphrasing                              |
|[PAWS-X_es](https://huggingface.co/datasets/google-research-datasets/paws-x)                            |PAWS-X_es          |Paraphrase Adversaries from Word Scrambling in Spanish                 |ES      |acc           |Paraphrasing                              |
|[PAWS_gl](https://huggingface.co/datasets/proxectonos/PAWS-gl)                                          |PAWS_gl            |Paraphrase Adversaries from Word Scrambling in Galician                |GL      |acc           |Paraphrasing                              |
|[PIQA_ca](https://huggingface.co/datasets/projecte-aina/piqa_ca)                                        |PIQA_ca            |Physical Interaction QA in Catalan                                     |CA      |acc           |Reasoning                                 |
|[QNLIeu](https://huggingface.co/datasets/orai-nlp/basqueGLUE)                                           |QNLIeu             |Textual Entailment in Basque                                           |EU      |acc           |NLI, Textual Entailment                   |
|[RagQuAS](https://huggingface.co/datasets/IIC/RagQuAS)                                                  |RagQuAS            |Retrieval-Augmented-Generation and Question-Answering in Spanish       |ES      |sas_encoder   |Abstractive QA                            |
|[SIQA_ca](https://huggingface.co/datasets/projecte-aina/siqa_ca)                                        |SIQA_ca            |Social Interaction QA in Catalan                                       |CA      |acc           |Reasoning                                 |
|[SpaLawEx](https://huggingface.co/datasets/LenguajeNaturalAI/examenes_abogacia)                         |SpaLawEx           |Spanish Law School Access Exams                                        |ES      |acc           |Multi choice QA                           |
|[SummarizationGL](https://huggingface.co/datasets/proxectonos/summarization_gl)                         |SummarizationGL    |Abstractive Summarization in Galician                                  |GL      |bleu          |Summarization                             |
|[TE-ca](https://huggingface.co/datasets/projecte-aina/teca)                                             |TE-ca              |Textual Entailment in Catalan                                          |CA      |acc           |Textual Entailment                        |
|[TELEIA](https://huggingface.co/datasets/gonzmart/teleia)                                               |TELEIA             |Test de Español como Lengua Extranjera para Inteligencia Artificial    |ES      |acc           |Multi choice QA                           |
|[VaxxStance](https://huggingface.co/datasets/orai-nlp/basqueGLUE)                                       |VaxxStance         |Stance detection on the Antivaxxers movement                           |EU      |f1            |Sentiment Analysis, Stance Detection      |
|[WiCeu](https://huggingface.co/datasets/orai-nlp/basqueGLUE)                                            |WiCeu              |Word sense disambiguation in Basque                                    |EU      |acc           |Textual Entailment                        |
|[WNLI_ca](https://huggingface.co/datasets/projecte-aina/wnli-ca)                                        |WNLI_ca            |Winograd-schema-type dataset in Catalan                                |CA      |acc           |NLI, Textual Entailment                   |
|[WNLI ES](huggingface.co/datasets/PlanTL-GOB-ES/wnli-es)                                                |WNLI ES            |Winograd-schema-type dataset in Spanish                                |ES      |acc           |NLI, Textual Entailment                   |
|[XCOPA_eu](https://huggingface.co/datasets/HiTZ/XCOPA-eu)                                               |XCOPA_eu           |Choice Of Plausible Alternatives in Basque                             |EU      |acc           |Reasoning                                 |
|[XNLI_ca](https://huggingface.co/datasets/projecte-aina/xnli-ca)                                        |XNLI_ca            |Cross-lingual Natural Language Inference in Catalan                    |CA      |acc           |NLI, Textual Entailment                   |
|[XNLI_es](https://huggingface.co/datasets/facebook/xnli)                                                |XNLI_es            |Cross-lingual Natural Language Inference in Spanish                    |ES      |acc           |NLI                                       |
|[XNLI_eu](https://huggingface.co/datasets/HiTZ/xnli-eu)                                                 |XNLI_eu            |Cross-lingual Natural Language Inference in Basque                     |EU      |acc           |NLI, Textual Entailment                   |
|[XQuAD_ca](https://huggingface.co/datasets/projecte-aina/xquad-ca)                                      |XQuAD_ca           |Cross-lingual Question Answering Dataset in Catalan                    |CA      |f1            |Extractive QA                             |
|[XQuAD_es](https://huggingface.co/datasets/google/xquad)                                                |XQuAD_es           |Cross-lingual Question Answering Dataset in Spanish                    |ES      |f1            |Extractive QA                             |
|[xStoryCloze_ca](https://huggingface.co/datasets/projecte-aina/xstorycloze_ca)                          |xStoryCloze_ca     |Narrative completion in Catalan                                        |CA      |acc           |Reasoning                                 |
|[xStoryCloze_es](https://huggingface.co/datasets/juletxara/xstory_cloze)                                |xStoryCloze_es     |Narrative completion in Spanish                                        |ES      |acc           |Reasoning                                 |
|[xStoryCloze_eu](https://huggingface.co/datasets/juletxara/xstory_cloze)                                |xStoryCloze_eu     |Narrative completion in Basque                                         |EU      |acc           |Reasoning                                 |

         
## Usage:

You can use the model using HuggingFace Transformers library with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 150GB of free disk space to accomodate the download.

This code has been tested on Transformers v4.44.0, torch v2.4.0 and 2 A100 80GB GPUs, but any setup that supports ```meta-llama/Llama-3.1-70B-Instruct``` should support this model as well. If you run into problems, you can consider doing ```pip install -U transformers```.
```python
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sandbox-ai/Llama-3.1-Tango-8b-f16")
```


## References(s):

* TODO

## Model Architecture: 
**Architecture Type:** Transformer <br>
**Network Architecture:** Llama 3.1 <br>

## Input:
**Input Type(s):** Text <br>
**Input Format:** String <br>
**Input Parameters:** One Dimensional (1D) <br>
**Other Properties Related to Input:** Max of 128k tokens<br>

## Output:
**Output Type(s):** Text <br>
**Output Format:** String <br>
**Output Parameters:** One Dimensional (1D) <br>
**Other Properties Related to Output:**  Max of 4k tokens <br>



# Training & Evaluation: 
 - TODO

# Dataset:

**MessIRve: A Large-Scale Spanish Information Retrieval Dataset** <br>
* [spanish/-ir/messirve](https://huggingface.co/datasets/spanish-ir/messirve) <br>
** messi_mod-v0.0.2
[tatakof/messi_mod-v0.0.2](https://huggingface.co/datasets/tatakof/messi_mod-v0.0.2)

## Citation

```bibtex
@article{valentini2024messirve,
      title={MessIRve: A Large-Scale Spanish Information Retrieval Dataset}, 
      author={Francisco Valentini and Viviana Cotik and Damián Furman and Ivan Bercovich and Edgar Altszyler and Juan Manuel Pérez},
      year={2024},
      eprint={2409.05994},
      journal={arxiv:2409.05994},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2409.05994}, 
}

@misc{wang2024helpsteer2preferencecomplementingratingspreferences,
      title={HelpSteer2-Preference: Complementing Ratings with Preferences}, 
      author={Zhilin Wang and Alexander Bukharin and Olivier Delalleau and Daniel Egert and Gerald Shen and Jiaqi Zeng and Oleksii Kuchaiev and Yi Dong},
      year={2024},
      eprint={2410.01257},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2410.01257}, 
}

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
