Large Language Models, Artificial Intelligence, Data Science, Machine Learning, Editors Pick.
done, we save the tokenizer and the model in a safetensors format. # Quantize with GPTQ model.quantize( examples_ids, batch_size=1, use_triton=True, ) # Save model and tokenizer model.save_quantized(out_dir, use_safetensors=True) tokenizer.save_pretrained(out_dir) As per usual, the model and