Update README.md

README.md

@@ -14,6 +14,43 @@ Llama-3.2-SFT-Vision-Arena is a chat assistant trained by fine-tuning Llama-3.2-
 - Repository: https://github.com/lm-sys/FastChat
 - Paper: https://arxiv.org/abs/2412.08687
 
+### Sample Inference Code
+```
+import requests
+import torch
+from PIL import Image
+from transformers import MllamaForConditionalGeneration, AutoProcessor
+
+model_id = "lmarena-ai/llama-3.2-sft-vision-arena"
+
+model = MllamaForConditionalGeneration.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+processor = AutoProcessor.from_pretrained(model_id)
+
+url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
+image = Image.open(requests.get(url, stream=True).raw)
+
+messages = [
+    {"role": "user", "content": [
+        {"type": "image"},
+        {"type": "text", "text": "Write a haiku about this image: "}
+    ]}
+]
+input_text = processor.tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+inputs = processor(
+    image,
+    input_text,
+    add_special_tokens=False,
+    return_tensors="pt"
+).to(model.device)
+
+output = model.generate(**inputs, max_new_tokens=30)
+print(processor.decode(output[0]))
+```
+
 ### Uses
 The primary use of Llama-3.2-SFT-Vision-Arena is research on vision language models and chatbots. The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.