safetensors gemma2 ar dataset:Slim205/total_data_baraka_ift base_model:google/gemma-2-9b-it base_model:finetune:google/gemma-2-9b-it license:mit region:us
---
base_model:
- google/gemma-2-9b-it
datasets:
- Slim205/total_data_baraka_ift
language:
- ar
license: mit
---
![Alt text](photo.png)

# Motivation : 

The goal of the project was to adapt large language models for the Arabic language and create a new state-of-the-art Arabic LLM. Due to the scarcity of Arabic instruction fine-tuning data, not many LLMs have been trained specifically in Arabic, which is surprising given the large number of Arabic speakers.  
Our final model was trained on a high-quality instruction fine-tuning (IFT) dataset, generated synthetically and then evaluated using the Hugging Face Arabic leaderboard.

# Training : 
This model is the 9B version. It was trained for a week on 4 A100 GPUs using LoRA with a rank of 128, a learning rate of 1e-4, and a cosine learning rate schedule.

# Evaluation :
| Metric               | Slim205/Barka-9b-it |
|----------------------|---------------------|
| Average              | 61.71               |
| ACVA                 | 73.68               |
| AlGhafa              | 54.42               |
| MMLU                 | 52.52               |
| EXAMS                | 52.51               |
| ARC Challenge        | 59.14               |
| ARC Easy             | 59.69               |
| BOOLQ                | 86.41               |
| COPA                 | 58.89               |
| HELLAWSWAG           | 38.04               |
| OPENBOOK QA          | 56.16               |
| PIQA                 | 72.01               |
| RACE                 | 48.71               |
| SCIQ                 | 66.43               |
| TOXIGEN              | 85.35               |

Please refer to https://github.com/Slim205/Arabicllm/ for more details.