add deploy using vLLM
Browse files
README.md
CHANGED
@@ -140,4 +140,9 @@ print(f'User: {question}\nAssistant: {response}')
|
|
140 |
#question = "Câu hỏi khác ......"
|
141 |
#response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=history, return_history=True)
|
142 |
#print(f'User: {question}\nAssistant: {response}')
|
|
|
|
|
|
|
|
|
|
|
143 |
```
|
|
|
140 |
#question = "Câu hỏi khác ......"
|
141 |
#response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=history, return_history=True)
|
142 |
#print(f'User: {question}\nAssistant: {response}')
|
143 |
+
```
|
144 |
+
|
145 |
+
Deploying the fine-tuned OCR model as a server:
|
146 |
+
```bash
|
147 |
+
!vllm serve TienAnh/Finetune_OCR_1B --port 8000 --dtype=auto
|
148 |
```
|