ljvmiranda921 commited on
Commit
7b79e31
·
verified ·
1 Parent(s): c72e5a2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md CHANGED
@@ -36,6 +36,40 @@ evaluation rubrics, and a score along with the corresponding reasoning.
36
  - **Repository:** https://github.com/rubricreward/r3
37
  - **Paper:** https://arxiv.org/abs/2505.13388
38
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  ## License and use
40
 
41
  R3 is licensed under the Apache 2.0 license.
 
36
  - **Repository:** https://github.com/rubricreward/r3
37
  - **Paper:** https://arxiv.org/abs/2505.13388
38
 
39
+ ## Using the Model
40
+
41
+
42
+ ```python
43
+ from transformers import AutoTokenizer
44
+ from vllm import LLM, SamplingParams
45
+
46
+ model_path = "rubricreward/R3-Qwen3-4B-LoRA-4k"
47
+ tokenizer = AutoTokenizer.from_pretrained(model_path)
48
+ sampling_params = SamplingParams(temperature=0.6, top_p=0.95, max_tokens=8192, min_p=0, top_k=20)
49
+
50
+ llm = LLM(
51
+ model=model_path,
52
+ dtype="bfloat16",
53
+ max_model_len=10000,
54
+ tensor_parallel_size=2,
55
+ gpu_memory_utilization=0.9,
56
+ enforce_eager=True,
57
+ )
58
+
59
+ messages: list[dict[str, str]] = [
60
+ {'content': "Evaluate the response based on the given task, input, response, and evaluation rubric. Provide a fair and detailed assessment following the rubric...", 'role': 'user'}
61
+ ]
62
+
63
+ list_text = tokenizer.apply_chat_template(
64
+ messages,
65
+ tokenize=False,
66
+ add_generation_prompt=True,
67
+ enable_thinking=True # Switch between thinking and non-thinking modes.
68
+ )
69
+
70
+ outputs = llm.generate(list_text, sampling_params)
71
+ ```
72
+
73
  ## License and use
74
 
75
  R3 is licensed under the Apache 2.0 license.