GGUFs for Ernie4.5 MoE 21B-A3B Quantized with imatrix from Bartowski's Qwen data (https://gist.github.com/bartowski1182/f003237f2e8612278a6d01622af1cb6f)

Base template not supported, run with --jinja

Downloads last month
1,822
GGUF
Model size
21.8B params
Architecture
ernie4_5-moe
Hardware compatibility
Log In to view the estimation

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ilintar/ERNIE-4.5-21B-A3B-PT-gguf

Quantized
(17)
this model