File size: 1,328 Bytes
19e3ec1 fa3fa06 19e3ec1 df791c8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
---
license: apache-2.0
base_model: swiss-ai/Apertus-70B-Instruct-2509
base_model_relation: quantized
quantized_by: turboderp
tags:
- exl3
---
EXL3 quants of [Apertus-70B-Instruct-2509](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509)
[2.00 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/2.0bpw)
[2.50 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/2.5bpw)
[3.00 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/3.0bpw)
[3.50 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/3.5bpw)
[4.00 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/4.0bpw)
[5.00 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/5.0bpw)
[6.00 bits per weight](https://huggingface.co/turboderp/Apertus-70B-Instruct-2509-exl3/tree/6.0bpw)
Quick sanity checks:
. | MMLU | 95% CI
----------|--------------|------------
2.0 bpw | 58.90% | +/- 1.50%
2.5 bpw | 64.20% | +/- 1.46%
3.0 bpw | 67.00% | +/- 1.43%
3.5 bpw | 67.70% | +/- 1.43%
4.0 bpw | 69.40% | +/- 1.40%
5.0 bpw | 70.30% | +/- 1.39%
6.0 bpw | 69.60% | +/- 1.40% |