Quantization Config
#2
by
chriswritescode
- opened
AWQModifier(
ignore=["lm_head", "re:.*mlp.gate$", "re:.*mlp.shared_expert_gate$"],
What method was applied? was the gate kept in bf16 ?
no idea, I did not make this quant. mirrored from model scope.