Update README.md

README.md

@@ -24,6 +24,54 @@ This model is a fine-tuned version of the **Qwen2.5-0.5B-Instruct** model, speci
 
 ## Model Details
 
+## How to Get Started with the Model
+
+Use the code below to load and use the model:
+
+```python
+from unsloth import FastLanguageModel
+from vllm import SamplingParams
+import torch
+
+# Load the Model & Tokenizer
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "AdamLucek/Qwen2.5-3B-Instruct-GRPO-2K-GSM8K",
+    max_seq_length = 2048,
+    load_in_4bit = True,
+    fast_inference = True,
+    gpu_memory_utilization = 0.7,
+)
+
+# Prep the Message
+PROMPT = "How many r's are in the word strawberry?"
+
+SYSTEM_PROMPT = """
+Respond in the following format:
+<reasoning>
+...
+</reasoning>
+<answer>
+...
+</answer>
+"""
+
+text = tokenizer.apply_chat_template([
+    {"role" : "system", "content" : SYSTEM_PROMPT},
+    {"role" : "user", "content" : PROMPT},
+], tokenize = False, add_generation_prompt = True)
+
+# Generate a response
+sampling_params = SamplingParams(
+    temperature = 0.8,
+    top_p = 0.95,
+    max_tokens = 1024,
+)
+output = model.fast_generate(
+    text,
+    sampling_params = sampling_params,
+)[0].outputs[0].text
+```
+
 ### Model Description
 
 - **Developed by:** [Your Name or Organization]
@@ -70,21 +118,3 @@ Users should:
 - Fine-tune the model further for domain-specific tasks.
 - Be aware of potential biases and limitations in reasoning capabilities.
 
-## How to Get Started with the Model
-
-Use the code below to load and use the model:
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model_name = "[Your Model Name on Hugging Face Hub]"
-model = AutoModelForCausalLM.from_pretrained(model_name)
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-
-# Example input
-input_text = "There are 15 apples in a basket. If 3 are removed, how many apples are left?"
-inputs = tokenizer(input_text, return_tensors="pt")
-
-# Generate output
-outputs = model.generate(**inputs)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))