Update README.md

README.md

@@ -12,30 +12,45 @@ base_model:
 ---
 
 # Kiwi-1.0-0.7B-32k-Instruct
+## Instruction-Tuned Model
+
+* **Developed by**: [EmpirischTech](https://empirischtech.at)/[ChaperoneAI](https://chaperoneai.net)
+* **Backbone Model**: [Kiwi-1.0-0.7B-32k](https://huggingface.co/empirischtech/Kiwi-1.0-0.7B-32k)
+* **Parameters**: 700m
+* **Context Window**: 32k
+* **Language(s)**: English
+* **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
+* **License**: Creative Common Attribute 4.0 (CCA-4.0)
+* **Contact**: For questions and comments about the model, please email [contact-us](https://chaperoneai.net/contact)
+
+## Main Message
+
+We present our initial results validating depth up-scaling—a method that combines depthwise scaling with continued pretraining. Unlike other LLM up-scaling approaches that rely on mixture-of-experts, DUS requires no complex modifications for efficient training and inference, making it a simple yet effective strategy for scaling high-performance LLMs from smaller models.
+
+In our approach, we carefully selected the dense layers from Qwen2.5-0.5B to construct our model. Notably, while Qwen2.5-0.5B was trained on *18 trillion* tokens, our model was trained on only *5 billion* tokens—over three orders of magnitude fewer—yet it achieves comparable performance.
+
+**Note**: Please note that this model has not yet been instruction-tuned; instruction-tuning is an area of ongoing development.
+
+## Evaluation Results
+
+
 
 ### Harness Evaluation
 
 - The performance evaluation is based on the tasks being evaluated on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
-The model is evaluated on three benchmark datasets, which include `ARC-Challenge`, `HellaSwag`, `MMLU` and `IFEval`.
+The model is evaluated on three benchmark datasets, which include `ARC-Challenge`, `HellaSwag`, `MMLU`, `IFEval` and `GPQA`.
 The library used is [lm-evaluation-harness repository](https://github.com/EleutherAI/lm-evaluation-harness)
 
 
-#### Main Results
-| Model |  ARC | HellaSwag | MMLU-PRO | IFEval |  GPQA (Diamond) |
-|------------------------|----------|--------|------|--------|--------|
-| **Qwen2.5-05B** | **33.45** | **52.37** | **14.03** | **37.53**| **12.27**(Zero-shot CoT)|
-| **Kiwi-1.0-0.7B-32k-Instruct** | **32.34** | **48.59** | **12.89** | **27.1** | **17.17**(Zero-shot CoT) |
-
-
-#### Main Results
+### Main Results
 | Metric           | **Qwen2.5-05B-Instruct**             | **Kiwi-1.0-0.7B-32k-Instruct**  |
 |-----------------|:-------------------:|:--------------------------:|
 | ARC             | 33.45                 | 32.34                      |
 | HellaSwag       | 52.37                 | 48.59                      |
 | MMLU-PRO        | 14.03                 | 12.89                      |
 | IFEval          | 37.53                 | 27.1                       |
-| GPQA (Diamond)\(Zero-shot CoT)  | 12.27  | 17.17  |
-| Average  | 29,93 | 27,27     |
+| GPQA (Diamond)<br>(Zero-shot CoT)  | 12.27  | 17.17  |
+| **Average**  | **29,93** | **27,27**     |