Update README.md

README.md

@@ -17,6 +17,8 @@ quantized_by: brittlewis12
 
 # Phi 3 Mini 4K Instruct GGUF
 
+***Updated with Microsoft’s [latest model changes](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/commit/4f818b18e097c9ae8f93a29a57027cad54b75304) as of July 21, 2024***
+
 **Original model**: [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
 
 **Model creator**: [Microsoft](https://huggingface.co/microsoft)
@@ -31,7 +33,7 @@ Learn more on Microsoft’s [Model page](https://azure.microsoft.com/en-us/blog/
 
 GGUF is a file format for representing AI models. It is the third version of the format, 
 introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. 
-Converted with llama.cpp build 2721 (revision [28103f4](https://github.com/ggerganov/llama.cpp/commit/28103f4832e301a9c84d44ff0df9d75d46ab6c76)),
+Converted with llama.cpp build 3432 (revision [45f2c19](https://github.com/ggerganov/llama.cpp/commit/45f2c19cc57286eead7b232ce8028273a817aa4d)),
 using [autogguf](https://github.com/brittlewis12/autogguf).
 
 ### Prompt template
@@ -63,6 +65,24 @@ using [autogguf](https://github.com/brittlewis12/autogguf).
 
 ## Original Model Evaluation
 
+Comparison of July update vs original April release:
+
+| Benchmarks | Original | June 2024 Update |
+|------------|----------|------------------|
+| Instruction Extra Hard | 5.7 | 6.0 |
+| Instruction Hard | 4.9 | 5.1 |
+| Instructions Challenge | 24.6 | 42.3 |
+| JSON Structure Output | 11.5 | 52.3 |
+| XML Structure Output | 14.4 | 49.8 |
+| GPQA	| 23.7	| 30.6 |
+| MMLU	| 68.8	| 70.9 |
+| **Average**	| **21.9**	| **36.7** |
+
+
+---
+
+### Original April release
+
 > As is now standard, we use few-shot prompts to evaluate the models, at temperature 0. 
 > The prompts and number of shots are part of a Microsoft internal tool to evaluate language models, and in particular we did no optimization to the pipeline for Phi-3.
 > More specifically, we do not change prompts, pick different few-shot examples, change prompt format, or do any other form of optimization for the model.