AXCXEPT
/

EZO-Qwen2.5-32B-Instruct

Text Generation

Model card Files Files and versions Community

AXCXEPT commited on Sep 23

Commit

abe333b

•

1 Parent(s): 63b5d57

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -57,7 +57,8 @@ text = tokenizer.apply_chat_template(
     tokenize=False,
     add_generation_prompt=True
 )
-model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
 generated_ids = model.generate(
     **model_inputs,

     tokenize=False,
     add_generation_prompt=True
 )
+model_inputs = tokenizer([text], return_tensors="pt")
+#if you don't use "load_in_4bit", you should do "model_inputs = tokenizer([text], return_tensors="pt").to(model.device)"
 generated_ids = model.generate(
     **model_inputs,