gguf text2text-generation llama-cpp gguf-my-repo af am ar az be bg bn ca ceb co cs cy da de el en eo es et eu fa fi fil fr fy ga gd gl gu ha haw hi hmn ht hu hy ig is it iw ja jv ka kk km kn ko ku ky la lb lo lt lv mg mi mk ml mn mr ms mt my ne nl no ny pa pl ps pt ro ru sd si sk sl sm sn so sq sr st su sv sw ta te tg th tr uk und ur uz vi xh yi yo zh zu dataset:bigscience/xP3mt dataset:mc4 base_model:bigscience/mt0-xxl-mt base_model:quantized:bigscience/mt0-xxl-mt license:apache-2.0 model-index endpoints_compatible region:us
---
base_model: bigscience/mt0-xxl-mt
datasets:
- bigscience/xP3mt
- mc4
language:
- af
- am
- ar
- az
- be
- bg
- bn
- ca
- ceb
- co
- cs
- cy
- da
- de
- el
- en
- eo
- es
- et
- eu
- fa
- fi
- fil
- fr
- fy
- ga
- gd
- gl
- gu
- ha
- haw
- hi
- hmn
- ht
- hu
- hy
- ig
- is
- it
- iw
- ja
- jv
- ka
- kk
- km
- kn
- ko
- ku
- ky
- la
- lb
- lo
- lt
- lv
- mg
- mi
- mk
- ml
- mn
- mr
- ms
- mt
- my
- ne
- nl
- 'no'
- ny
- pa
- pl
- ps
- pt
- ro
- ru
- sd
- si
- sk
- sl
- sm
- sn
- so
- sq
- sr
- st
- su
- sv
- sw
- ta
- te
- tg
- th
- tr
- uk
- und
- ur
- uz
- vi
- xh
- yi
- yo
- zh
- zu
license: apache-2.0
pipeline_tag: text2text-generation
tags:
- text2text-generation
- llama-cpp
- gguf-my-repo
widget:
- text: Life is beautiful! Translate to Mongolian.
  example_title: mn-en translation
- text: Le mot japonais «憂鬱» veut dire quoi en Odia?
  example_title: jp-or-fr translation
- text: Stell mir eine schwierige Quiz Frage bei der es um Astronomie geht. Bitte
    stell die Frage auf Norwegisch.
  example_title: de-nb quiz
- text: 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。Would you rate the previous
    review as positive, neutral or negative?
  example_title: zh-en sentiment
- text: 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评？
  example_title: zh-zh sentiment
- text: Suggest at least five related search terms to "Mạng neural nhân tạo".
  example_title: vi-en query
- text: Proposez au moins cinq mots clés concernant «Réseau de neurones artificiels».
  example_title: fr-fr query
- text: Explain in a sentence in Telugu what is backpropagation in neural networks.
  example_title: te-en qa
- text: Why is the sky blue?
  example_title: en-en qa
- text: 'Write a fairy tale about a troll saving a princess from a dangerous dragon.
    The fairy tale is a masterpiece that has achieved praise worldwide and its moral
    is "Heroes Come in All Shapes and Sizes". Story (in Spanish):'
  example_title: es-en fable
- text: 'Write a fable about wood elves living in a forest that is suddenly invaded
    by ogres. The fable is a masterpiece that has achieved praise worldwide and its
    moral is "Violence is the last refuge of the incompetent". Fable (in Hindi):'
  example_title: hi-en fable
model-index:
- name: mt0-xxl-mt
  results:
  - task:
      type: Coreference resolution
    dataset:
      name: Winogrande XL (xl)
      type: winogrande
      config: xl
      split: validation
      revision: a80f460359d1e9a67c006011c94de42a8759430c
    metrics:
    - type: Accuracy
      value: 62.67
  - task:
      type: Coreference resolution
    dataset:
      name: XWinograd (en)
      type: Muennighoff/xwinograd
      config: en
      split: test
      revision: 9dd5ea5505fad86b7bedad667955577815300cee
    metrics:
    - type: Accuracy
      value: 83.31
  - task:
      type: Coreference resolution
    dataset:
      name: XWinograd (fr)
      type: Muennighoff/xwinograd
      config: fr
      split: test
      revision: 9dd5ea5505fad86b7bedad667955577815300cee
    metrics:
    - type: Accuracy
      value: 78.31
  - task:
      type: Coreference resolution
    dataset:
      name: XWinograd (jp)
      type: Muennighoff/xwinograd
      config: jp
      split: test
      revision: 9dd5ea5505fad86b7bedad667955577815300cee
    metrics:
    - type: Accuracy
      value: 80.19
  - task:
      type: Coreference resolution
    dataset:
      name: XWinograd (pt)
      type: Muennighoff/xwinograd
      config: pt
      split: test
      revision: 9dd5ea5505fad86b7bedad667955577815300cee
    metrics:
    - type: Accuracy
      value: 80.99
  - task:
      type: Coreference resolution
    dataset:
      name: XWinograd (ru)
      type: Muennighoff/xwinograd
      config: ru
      split: test
      revision: 9dd5ea5505fad86b7bedad667955577815300cee
    metrics:
    - type: Accuracy
      value: 79.05
  - task:
      type: Coreference resolution
    dataset:
      name: XWinograd (zh)
      type: Muennighoff/xwinograd
      config: zh
      split: test
      revision: 9dd5ea5505fad86b7bedad667955577815300cee
    metrics:
    - type: Accuracy
      value: 82.34
  - task:
      type: Natural language inference
    dataset:
      name: ANLI (r1)
      type: anli
      config: r1
      split: validation
      revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
    metrics:
    - type: Accuracy
      value: 49.5
  - task:
      type: Natural language inference
    dataset:
      name: ANLI (r2)
      type: anli
      config: r2
      split: validation
      revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
    metrics:
    - type: Accuracy
      value: 42
  - task:
      type: Natural language inference
    dataset:
      name: ANLI (r3)
      type: anli
      config: r3
      split: validation
      revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
    metrics:
    - type: Accuracy
      value: 48.17
  - task:
      type: Natural language inference
    dataset:
      name: SuperGLUE (cb)
      type: super_glue
      config: cb
      split: validation
      revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
    metrics:
    - type: Accuracy
      value: 87.5
  - task:
      type: Natural language inference
    dataset:
      name: SuperGLUE (rte)
      type: super_glue
      config: rte
      split: validation
      revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
    metrics:
    - type: Accuracy
      value: 84.84
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (ar)
      type: xnli
      config: ar
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 58.03
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (bg)
      type: xnli
      config: bg
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 59.92
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (de)
      type: xnli
      config: de
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 60.16
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (el)
      type: xnli
      config: el
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 59.2
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (en)
      type: xnli
      config: en
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 62.25
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (es)
      type: xnli
      config: es
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 60.92
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (fr)
      type: xnli
      config: fr
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 59.88
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (hi)
      type: xnli
      config: hi
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 57.47
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (ru)
      type: xnli
      config: ru
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 58.67
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (sw)
      type: xnli
      config: sw
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 56.79
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (th)
      type: xnli
      config: th
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 58.03
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (tr)
      type: xnli
      config: tr
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 57.67
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (ur)
      type: xnli
      config: ur
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 55.98
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (vi)
      type: xnli
      config: vi
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 58.92
  - task:
      type: Natural language inference
    dataset:
      name: XNLI (zh)
      type: xnli
      config: zh
      split: validation
      revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
    metrics:
    - type: Accuracy
      value: 58.71
  - task:
      type: Sentence completion
    dataset:
      name: StoryCloze (2016)
      type: story_cloze
      config: '2016'
      split: validation
      revision: e724c6f8cdf7c7a2fb229d862226e15b023ee4db
    metrics:
    - type: Accuracy
      value: 94.66
  - task:
      type: Sentence completion
    dataset:
      name: SuperGLUE (copa)
      type: super_glue
      config: copa
      split: validation
      revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
    metrics:
    - type: Accuracy
      value: 88
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (et)
      type: xcopa
      config: et
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 81
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (ht)
      type: xcopa
      config: ht
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 79
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (id)
      type: xcopa
      config: id
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 90
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (it)
      type: xcopa
      config: it
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 88
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (qu)
      type: xcopa
      config: qu
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 56
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (sw)
      type: xcopa
      config: sw
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 81
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (ta)
      type: xcopa
      config: ta
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 81
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (th)
      type: xcopa
      config: th
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 76
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (tr)
      type: xcopa
      config: tr
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 76
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (vi)
      type: xcopa
      config: vi
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 85
  - task:
      type: Sentence completion
    dataset:
      name: XCOPA (zh)
      type: xcopa
      config: zh
      split: validation
      revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
    metrics:
    - type: Accuracy
      value: 87
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (ar)
      type: Muennighoff/xstory_cloze
      config: ar
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 91
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (es)
      type: Muennighoff/xstory_cloze
      config: es
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 93.38
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (eu)
      type: Muennighoff/xstory_cloze
      config: eu
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 91.13
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (hi)
      type: Muennighoff/xstory_cloze
      config: hi
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 90.73
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (id)
      type: Muennighoff/xstory_cloze
      config: id
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 93.05
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (my)
      type: Muennighoff/xstory_cloze
      config: my
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 86.7
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (ru)
      type: Muennighoff/xstory_cloze
      config: ru
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 91.66
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (sw)
      type: Muennighoff/xstory_cloze
      config: sw
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 89.61
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (te)
      type: Muennighoff/xstory_cloze
      config: te
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 90.4
  - task:
      type: Sentence completion
    dataset:
      name: XStoryCloze (zh)
      type: Muennighoff/xstory_cloze
      config: zh
      split: validation
      revision: 8bb76e594b68147f1a430e86829d07189622b90d
    metrics:
    - type: Accuracy
      value: 93.05
---

# Markobes/mt0-xxl-mt-Q4_K_M-GGUF
This model was converted to GGUF format from [`bigscience/mt0-xxl-mt`](https://huggingface.co/bigscience/mt0-xxl-mt) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/bigscience/mt0-xxl-mt) for more details on the model.

## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -c 2048
```
