Machine Learning, Transformers, Data Science, Python, Hugging Face.
Face docs. Parameters are self-explained, 4-bit quantization, C4 dataset, and the tokenizer to use during quantization. The other two parameters take the default values: group_size: The group size to use for quantization. Recommended value is 128 and -1 uses per-column quantization desc_act: