Update README.md
Browse files
README.md
CHANGED
|
@@ -26,22 +26,7 @@ Jan-v1 leverages the newly released [Qwen3-4B-thinking](https://huggingface.co/Q
|
|
| 26 |
### Question Answering (SimpleQA)
|
| 27 |
For question-answering, Jan-v1 shows a significant performance gain from model scaling, achieving 91.2% accuracy.
|
| 28 |
|
| 29 |
-
|
| 30 |
-
| :--- | :--- |
|
| 31 |
-
| **Jan-v1 (Ours)** | **91.1%** |
|
| 32 |
-
| Qwen3-4B-thinking-2507 | 86.5% |
|
| 33 |
-
| Jan-nano-128k-MCP (YaRN 130k) | 83.2% |
|
| 34 |
-
| Jan-nano-MCP | 80.7% |
|
| 35 |
-
| Jan-nano-MCP (YaRN 130k) | 79.7% |
|
| 36 |
-
| Lucy (YaRN 130k) | 78.3% |
|
| 37 |
-
| DeepSeek-V3-MCP | 78.2% |
|
| 38 |
-
| ChatGPT-4.5 | 62.5% |
|
| 39 |
-
| Baseline-MCP | 59.2% |
|
| 40 |
-
| Gemini-2.5-Pro | 52.9% |
|
| 41 |
-
| Claude-3.7-Sonnet | 50% |
|
| 42 |
-
| o3 | 49.4% |
|
| 43 |
-
| Grok-3 | 44.6% |
|
| 44 |
-
| o1 | 42.6% |
|
| 45 |
|
| 46 |
*The 91.2% SimpleQA accuracy represents a significant milestone in factual question answering for models of this scale, demonstrating the effectiveness of our scaling and fine-tuning approach.*
|
| 47 |
|
|
|
|
| 26 |
### Question Answering (SimpleQA)
|
| 27 |
For question-answering, Jan-v1 shows a significant performance gain from model scaling, achieving 91.2% accuracy.
|
| 28 |
|
| 29 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
*The 91.2% SimpleQA accuracy represents a significant milestone in factual question answering for models of this scale, demonstrating the effectiveness of our scaling and fine-tuning approach.*
|
| 32 |
|