Nicemleme, Quantization Experiment, Bitsandbytesconfig, Model Quantization, Üretken Yapay Zeka.
return_tensors="pt").to("cuda") #outputs = model.generate(inputs, max_new_tokens=640, pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.1, top_k=50, top_p=0.95) outputs = model.generate(inputs, max_length=512, pad_token_id=tokenizer.eos_token_id) answer =