| base_model: mistralai/Mistral-7B-Instruct-v0.2 | |
| license: apache-2.0 | |
| library_name: transformers | |
| tags: | |
| - 4-bit | |
| - AWQ | |
| - text-generation | |
| - autotrain_compatible | |
| - endpoints_compatible | |
| - finetuned | |
| pipeline_tag: text-generation | |
| inference: false | |
| quantized_by: Suparious | |
| # mistralai/Mistral-7B-Instruct-v0.2 AWQ | |
| - Model creator: [mistralai](https://huggingface.co/mistralai) | |
| - Original model: [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) | |
| ## Model Summary | |
| The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2. | |
| Mistral-7B-v0.2 has the following changes compared to Mistral-7B-v0.1 | |
| - 32k context window (vs 8k context in v0.1) | |
| - Rope-theta = 1e6 | |
| - No Sliding-Window Attention | |
| For full details of this model please read our [paper](https://arxiv.org/abs/2310.06825) and [release blog post](https://mistral.ai/news/la-plateforme/). | |
| ## Instruction format | |
| In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id. | |
| E.g. | |
| ``` | |
| text = "<s>[INST] What is your favourite condiment? [/INST]" | |
| "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> " | |
| "[INST] Do you have mayonnaise recipes? [/INST]" | |
| ``` | |
| This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method. | |