Update README.md

README.md

@@ -6,4 +6,71 @@ language:
 - en
 base_model:
 - Qwen/Qwen2.5-7B-Instruct
----
+---
+
+# PairJudge RM
+
+
+**PairJudge RM** is a pairwise judge reward model designed to enhance Best-of-N sampling for mathematical reasoning tasks. Instead of assigning arbitrary absolute scores to candidate solutions, PairJudge RM compares them in pairs using chain-of-thought (CoT) reasoning and selects the best answer via a knockout tournament strategy.
+
+- Paper: https://arxiv.org/abs/2501.13007
+- Code: https://github.com/THU-KEG/PairJudgeRM
+- Dataset: https://huggingface.co/datasets/THU-KEG/PairJudge-432K
+
+
+## Overview
+
+- **Pairwise Judgment:** Evaluates two candidate solutions simultaneously to determine which is more correct.
+- **Chain-of-Thought Reasoning:** Leverages CoT to transparently verify each step of the candidate solutions.
+
+## Model Architecture & Training
+
+PairJudge RM is built by fine-tuning a pre-trained language model (e.g., Qwen-2.5-7B-Instruct) on the PAIRJUDGE-432K dataset. Key training details include:
+
+- **Optimizer:** Adam
+- **Learning Rate:** 1×10⁻⁵
+- **Batch Size:** 128
+- **Epochs:** 8
+
+## Usage
+
+Below is an example of how to use PairJudge RM for evaluating candidate solutions:
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# template file is avaliable in [https://github.com/THU-KEG/PairwiseRM/blob/main/prompt/compare_0_ex.md]
+TEMPLATE = open("prompts/compare_0_ex.md", "r").read()
+
+# Load the tokenizer and model from Hugging Face
+tokenizer = AutoTokenizer.from_pretrained("THU-KEG/PairJudgeRM")
+model = AutoModelForCausalLM.from_pretrained("THU-KEG/PairJudgeRM")
+
+# Example math problem and candidate solutions
+question = "If one equilateral triangle in a regular hexagon has a perimeter of 21 inches, what is the hexagon’s perimeter?"
+response_a = "Each side is 7 inches; hexagon perimeter is 42 inches."
+response_b = "The triangle's perimeter is 21 inches; hexagon perimeter is 126 inches."
+
+# Construct the input prompt for pairwise judgment
+input_text = template.format(question=question, response_a=response_a, response_b=response_b)
+inputs = tokenizer(input_text, return_tensors="pt")
+
+# Generate the judgment with a chain-of-thought explanation
+outputs = model.generate(**inputs, max_new_tokens=2048)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+
+## Citation
+If you find our work useful, please consider citing our paper:
+
+```
+@article{liu2025PairJudge,
+  title={PairJudge RM: Perform Best-of-N Sampling with Knockout Tournament},
+  author={Liu, Yantao and Yao, Zijun and Min, Rui and Cao, Yixin and Hou, Lei and Li, Juanzi},
+  journal={arXiv preprint arXiv:2501.13007},
+  year={2025},
+  note={in progress work},
+  url={https://doi.org/10.48550/arXiv.2501.13007}
+}
+```