Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+figures/wechat.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,179 @@
----
-license: mit
----
+---
+license: mit
+
+pipeline_tag: image-text-to-text
+tags:
+- multimodal
+- vision-language
+- chat
+library_name: transformers
+language:
+- en
+- zh
+---
+# dots.vlm1
+
+<p align="center">
+    <img src="figures/logo.png" width="300"/>
+<p>
+<p align="center">
+    &nbsp&nbsp🤗 <a href="https://huggingface.co/rednote-hilab/dots.vlm1">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp 📄 <a href="https://github.com/rednote-hilab/dots.vlm1/blob/main/assets/blog.md">Blog</a> &nbsp&nbsp 
+<br>
+🖥️ <a href="https://huggingface.co/spaces/rednote-hilab/dots-vlm1-demo">Demo</a>&nbsp&nbsp | &nbsp&nbsp💬 <a href="https://raw.githubusercontent.com/rednote-hilab/dots.vlm1/master/assets/wechat.png">WeChat (微信)</a>&nbsp&nbsp | &nbsp&nbsp📕 <a href="https://www.xiaohongshu.com/user/profile/683ffe42000000001d021a4c">rednote</a>&nbsp&nbsp
+</p>
+
+Visit our Hugging Face (click links above) or check out our [live demo](https://huggingface.co/spaces/rednote-hilab/dots-vlm1-demo) to try dots.vlm1! Enjoy!
+
+## Introduction
+
+We are excited to introduce **dots.vlm1**, the first vision-language model in the dots model family. Built upon a 1.2 billion-parameter vision encoder and the DeepSeek V3 large language model (LLM), **dots.vlm1** demonstrates strong multimodal understanding and reasoning capabilities.
+
+Through large-scale pretraining and carefully tuned post-training, **dots.vlm1 achieves near state-of-the-art performance in both visual perception and reasoning**, setting a new performance ceiling for open-source vision-language models—while still maintaining competitive capabilities in pure-text tasks.
+
+## Model Summary
+
+**This repo contains the instruction-tuned `dots.vlm1` model** which has the following features:
+
+- Type: A multimodal vision-language model with 1.2B vision encoder and DeepSeek V3 LLM
+- Training Stages: Vision encoder pretraining, VLM pretraining, and supervised fine-tuning (SFT)
+- Architecture: NaViT vision encoder + MLP adapter + DeepSeek V3 MoE language model
+- Vision Encoder: 1.2B parameters, 42 transformer layers with RMSNorm, SwiGLU, and 2D RoPE
+- Supported Languages: English, Chinese
+- Context Length: 65,536 tokens
+- License: MIT
+
+**Model Highlights**:
+- **NaViT Vision Encoder**: Trained entirely from scratch rather than fine-tuning an existing vision backbone. It natively supports dynamic resolution and incorporates pure visual supervision in addition to traditional text supervision, thereby enhancing the upper bound of perceptual capacity. Beyond image captioning datasets, a large amount of structured image data was introduced during pretraining to improve the model's perceptual capabilities—particularly for tasks such as OCR.
+- **Multimodal Training Data**: In addition to conventional approaches, dots.vlm1 leverages a wide range of synthetic data strategies to cover diverse image types (e.g., tables, charts, documents, graphics) and descriptions (e.g., alt text, dense captions, grounding annotations). Furthermore, a strong multimodal model was used to rewrite web page data with interleaved text and images, significantly improving the quality of the training corpus.
+
+## Example Usage
+
+### Environment Setup
+
+You have two options to set up the environment:
+
+#### Option 1: Using Base Image + Manual Installation
+```bash
+# Use the base SGLang image
+docker run -it --gpus all lmsysorg/sglang:v0.4.9.post1-cu126
+
+# Clone and install our custom SGLang branch
+# IMPORTANT: Only our specific SGLang version supports dots.vlm1 models
+# We have submitted a PR to the main SGLang repository (currently under review):
+# https://github.com/sgl-project/sglang/pull/8778
+git clone --branch dots.vlm1.v1 https://github.com/rednote-hilab/sglang sglang
+pip install -e sglang/python
+```
+
+#### Option 2: Using Pre-built Image (Recommended)
+```bash
+# Use our pre-built image with dots.vlm1 support
+docker run -it --gpus all rednotehilab/dots.vlm1_sglang:v0.4.9.post1-cu126
+```
+
+### Multi-Node Deployment
+
+Our model supports distributed deployment across multiple machines. Here's how to set up a 2-node cluster:
+
+**Prerequisites:**
+- Model: `rednote-hilab/dots.vlm1.inst`
+- Node 1 IP: `10.0.0.1` (master node)
+- Node 2 IP: `10.0.0.2` (worker node)
+
+#### Node 1 (Master - rank 0):
+```bash
+export HF_MODEL_PATH="rednote-hilab/dots.vlm1.inst"
+
+python3 -m sglang.launch_server \
+    --model-path $HF_MODEL_PATH \
+    --tp 16 \
+    --dist-init-addr 10.0.0.1:23456 \
+    --nnodes 2 \
+    --node-rank 0 \
+    --trust-remote-code \
+    --host 0.0.0.0 \
+    --port 15553 \
+    --context-length 65536 \
+    --max-running-requests 64 \
+    --disable-radix-cache \
+    --mem-fraction-static 0.8 \
+    --chunked-prefill-size -1 \
+    --chat-template dots-vlm \
+    --cuda-graph-max-bs 64 \
+    --quantization fp8
+```
+
+#### Node 2 (Worker - rank 1):
+```bash
+export HF_MODEL_PATH="rednote-hilab/dots.vlm1.inst"
+
+python3 -m sglang.launch_server \
+    --model-path $HF_MODEL_PATH \
+    --tp 16 \
+    --dist-init-addr 10.0.0.1:23456 \
+    --nnodes 2 \
+    --node-rank 1 \
+    --trust-remote-code \
+    --host 0.0.0.0 \
+    --port 15553 \
+    --context-length 65536 \
+    --max-running-requests 64 \
+    --disable-radix-cache \
+    --mem-fraction-static 0.8 \
+    --chunked-prefill-size -1 \
+    --chat-template dots-vlm \
+    --cuda-graph-max-bs 64 \
+    --quantization fp8
+```
+
+### Configuration Parameters
+
+Key parameters explanation:
+- `--tp 16`: Tensor parallelism across 16 GPUs per node
+- `--nnodes 2`: Total number of nodes in the cluster
+- `--node-rank`: Node identifier (0 for master, 1+ for workers)
+- `--context-length 65536`: Maximum context length
+- `--quantization fp8`: Use FP8 quantization for efficiency
+- `--chat-template dots-vlm`: Use custom chat template for dots.vlm model
+
+### API Usage
+
+Once the server is launched, you can access the model through OpenAI-compatible API:
+
+```bash
+curl -X POST http://localhost:15553/v1/chat/completions \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "model",
+        "messages": [
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "text",
+                        "text": "Please briefly describe this image"
+                    },
+                    {
+                        "type": "image_url",
+                        "image_url": {
+                            "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
+                        }
+                    }
+                ]
+            }
+        ],
+        "temperature": 0.1,
+        "top_p": 0.9,
+        "max_tokens": 55000
+    }'
+```
+
+## Performance
+
+On major visual benchmarks, dots.vlm1 has achieved overall performance comparable to leading models such as **Gemini 2.5 Pro** and **Seed-VL1.5 thinking**. In particular, it demonstrates strong visual-text understanding and reasoning capabilities on datasets like **MMMU**, **MathVision**, and **OCR Reasoning**.
+
+For typical text-based reasoning tasks (e.g., **AIME**, **GPQA**, **LiveCodeBench**), **dots.vlm1** performs roughly on par with **DeepSeek-R1-0528**, showing competitive general capability in mathematics and coding.
+
+Overall, **dots.vlm1** approaches state-of-the-art levels in multimodal visual understanding and achieves mainstream performance in text reasoning.
+
+Detailed evaluation results are available in our [blog](https://github.com/rednote-hilab/dots.vlm1/blob/main/assets/blog.md).