Large Language Models, Quantization, Machine Learning, Data Science, Hands On Tutorials.
top-k sampling. def generate_text(model, input_text, max_length=50): input_ids = tokenizer.encode(input_text, return_tensors='pt').to(device) output = model.generate(inputs=input_ids, max_length=max_length, do_sample=True, top_k=30, pad_token_id=tokenizer.eos_token_id,