MaxLSB commited on
Commit
93e24e2
·
verified ·
1 Parent(s): 550854f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -52
README.md CHANGED
@@ -13,62 +13,16 @@ pipeline_tag: text-generation
13
 
14
  ![Kurakura AI Logo](media/logo_kurakura.png)
15
 
16
- ---
17
-
18
- # Luth-1.7B-Instruct
19
 
20
  **Luth-1.7B-Instruct** is a French fine-tuned version of [Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B), trained on the [Luth-SFT](https://huggingface.co/datasets/kurakurai/luth-sft) dataset. The model has drastically improved its French capabilities in instruction following, math, and general knowledge. Additionally, its English capabilities have remained stable and have even increased in some areas.
21
 
22
- Our Evaluation, training and data scripts are available on [GitHub](https://github.com/kurakurai/Luth), along with the [Blog](https://huggingface.co/blog/MaxLSB/luth) we wrote.
23
-
24
- ## Model Details
25
-
26
- Luth was trained using full fine-tuning on the Luth-SFT dataset with [Axolotl](https://github.com/axolotl-ai-cloud/axolotl). The resulting model was then merged with the base Qwen3-1.7B model. This process successfully retained the model's English capabilities while improving its performance on most selected benchmarks in both French and English.
27
-
28
- ## Benchmark Results
29
-
30
- We used LightEval for evaluation, with custom tasks for the French benchmarks. The models were evaluated with a `temperature=0`.
31
-
32
- ### Evaluation Visualizations
33
-
34
- **French Evaluation:**
35
-
36
- ![French Evaluation](media/french_evaluation.png)
37
-
38
- **English Evaluation:**
39
-
40
- ![English Evaluation](media/english_evaluation.png)
41
-
42
- ### French Benchmark Scores
43
-
44
- | Benchmark | Qwen3-1.7B | SmolLM2-1.7B-Instruct | Qwen2.5-1.5B-Instruct | Luth-1.7B-Instruct |
45
- |-------------------|------------------|-----------------------|-----------------------|----------------------|
46
- | ifeval-fr | 54.53 | 31.24 | 32.90 | <u>57.67</u> |
47
- | gpqa-diamond-fr | 26.90 | 21.83 | 28.93 | <u>38.58</u> |
48
- | mmlu-fr | 28.46 | 33.73 | 46.25 | <u>49.66</u> |
49
- | math-500-fr | 60.80 | 11.20 | 32.20 | <u>64.00</u> |
50
- | arc-chall-fr | 33.28 | 28.57 | 32.68 | <u>35.16</u> |
51
- | hellaswag-fr | 24.86 | <u>49.58</u> | 34.34 | 31.93 |
52
-
53
- ### English Benchmark Scores
54
 
55
- | Benchmark | Qwen3-1.7B | SmolLM2-1.7B-Instruct | Qwen2.5-1.5B-Instruct | Luth-1.7B-Instruct |
56
- |-------------------|------------------|-----------------------|-----------------------|----------------------|
57
- | ifeval-en | <u>68.39</u> | 48.24 | 39.93 | 65.80 |
58
- | gpqa-diamond-en | <u>31.82</u> | 24.75 | 30.30 | 31.82 |
59
- | mmlu-en | 52.74 | 50.27 | 59.81 | <u>60.19</u> |
60
- | math-500-en | 69.20 | 22.40 | 56.00 | <u>70.00</u> |
61
- | arc-chall-en | 36.09 | 42.32 | 41.04 | <u>42.24</u> |
62
- | hellaswag-en | 46.96 | <u>66.94</u> | 64.48 | 58.55 |
63
 
64
- ## Citation
65
 
66
- ```bibtex
67
- @misc{luth2025kurakurai,
68
- title = {Luth-1.7B-Instruct},
69
- author = {Kurakura AI Team},
70
- year = {2025},
71
- howpublished = {\url{https://huggingface.co/kurakurai/Luth-0.6B}},
72
- note = {Qwen3-1.7B fine-tuned on French datasets}
73
- }
74
  ```
 
 
 
13
 
14
  ![Kurakura AI Logo](media/logo_kurakura.png)
15
 
16
+ # Luth-1.7B-Instruct-GGUF
 
 
17
 
18
  **Luth-1.7B-Instruct** is a French fine-tuned version of [Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B), trained on the [Luth-SFT](https://huggingface.co/datasets/kurakurai/luth-sft) dataset. The model has drastically improved its French capabilities in instruction following, math, and general knowledge. Additionally, its English capabilities have remained stable and have even increased in some areas.
19
 
20
+ Find more details in the original model card: https://huggingface.co/kurakurai/Luth-1.7B-Instruct
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
+ ## 🏃 How to run Luth
 
 
 
 
 
 
 
23
 
24
+ Example usage with [llama.cpp](https://github.com/ggml-org/llama.cpp):
25
 
 
 
 
 
 
 
 
 
26
  ```
27
+ llama-cli -hf kurakurai/Luth-1.7B-Instruct-GGUF
28
+ ```