Model Description
This is a model using the llama2 architecture and only 30 million parameters. It is based off of this model and was finetuned on approximately 85 million tokens of instruct data from the first 20000 rows of the openhermes 2.5 dataset with a low learning rate of 2e-6 and context length of 512.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 4.32 |
| IFEval (0-Shot) | 17.06 |
| BBH (3-Shot) | 2.48 |
| MATH Lvl 5 (4-Shot) | 0.00 |
| GPQA (0-shot) | 1.34 |
| MuSR (0-shot) | 3.18 |
| MMLU-PRO (5-shot) | 1.85 |
- Downloads last month
- 144
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.
Model tree for cpayne1303/cp2024-instruct
Base model
cpayne1303/cp2024Dataset used to train cpayne1303/cp2024-instruct
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard17.060
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard2.480
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard0.000
- acc_norm on GPQA (0-shot)Open LLM Leaderboard1.340
- acc_norm on MuSR (0-shot)Open LLM Leaderboard3.180
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard1.850