II-Vietnam/Qwen3-4B-Code-CI-RL-V2

II-Search-CIR 4B

Inspired by the success of our II-Researcher approach, which applies tools with augmented reasoning on top of the Deep-seek-R1 model, II-Search-4B-CIR introduces Code-Integrated Reasoning (CIR), a more powerful and flexible method for tool interaction with the reasoning process.

Model Description

Code Integrated Reasoning

We instruct the model to generate code blocks enclosed between <start_code>\n```python and \n```<end_code> , within which it can invoke a set of predefined functions.

These functions act as interfaces to external resources, similar to the tool call paradigm but offering greater flexibility and control. This approach enables the model to not only retrieve external information but also process, filter, and reason over it programmatically within the code itself.

In our setup, we provide two predefined functions:

• web_search(query: str, num_result: int)
• web_visit(url: str)

Training Methodology

In our early experiments, we found that even large models such as Qwen/Qwen3-235B-A22B or Deep-seek-R1 could not produce the code format efficiently. Sometimes, models would not use any code blocks at all, instead relying on their internal knowledge base to answer the query. To address this issue, we first curated a dataset and performed SFT fine-tuning on the Qwen/Qwen3-4B model. Following this, we further optimized the SFT model by training DAPO on a hard-reasoning dataset to boost performance.

For SFT stage we using the hyperparameters:

• Max Length: 26000.
• Batch Size: 128.
• Learning-Rate: 1e-5.
• Number Of Epoch: 4.

For RL stage we setup training with:

• Max prompt length: 3000 tokens.
• Max response length: 16384 tokens.
• Max Total length: 32768 tokens.
• Max Observation Length: 3000 tokens per observation.
• Masking Observation Tokens: True
• Max n.o code blocks: 32.
• Clip ratios: Low 0.2, High 0.3.
• Batch sizes: Train prompt 128, Generation prompt 128, Mini-batch 16.
• Responses per prompt: 16.
• Temperature: 1.0, Top-p: 1.0, Top-k: -1 (vLLM rollout).
• Learning rate: 1e-6, Warmup steps: 20.
• Loss aggregation: Token-mean.
• Gradient clipping: 1.0.

We describe more detail of our training methodology in our II-Search-4B blog post

Evaluation Results

We compare our model with other small-sized open-source models, including Qwen3-4B (the model on which we are based) and other models that also specialize in information-seeking tasks. Qwen3-4B, Jan-4B, WebSailor-3B. We also reported the benchmarking results on Google Frames dataset from 2 latest MoE models Qwen3-30B-A3B-Instruct-2507 and Qwen3-30B-A3B-Thinking-2507 on this task. The search API was SerpDev, Google Gemini Pro 2.5 was used to extract and judge the answers (Using the proper judge prompt from the each benchmarking dataset’s author).

Benchmark	Qwen3-4B	Jan-4B	WebSailor-3B	II-Search-4B	II-Search-CIR-4B
OpenAI/SimpleQA	76.8	80.1	81.8	91.8	91.8
Google/Frames	30.7	24.8	34.0	67.5	72.93
Seal_0	6.31	2.7	1.8	22.5	27.03

Note: Our MCP ensure that we didn't go to any url come from the huggingface when we evaluate the II-Search-CIR-4B model.

V. How To Use

Our model can be utilized in the same manner as Qwen or Deepseek-R1-Distill models.

For instance, you can easily start a service using vLLM:

vllm serve 

You can also easily start a service using SGLang:

python -m sglang.launch_server --model 

VI. Usage Guidelines

• Recommended Sampling Parameters: temperature = 0.6, top_p = 0.95
• TODO: add more

VIII. Citation

@misc{2025II-Medical-8B,
      title={II-Search-4B: Search Reasoning Model}, 
      author={Intelligent Internet},
      year={2025}
}