metadata
license: bsd-3-clause
language:
- en
- zh
base_model:
- HuggingFaceTB/SmolVLM2-500M-Video-Instruct
pipeline_tag: visual-question-answering
tags:
- HuggingFaceTB
- SmolVLM2-500M-Video-Instruct
SmolVLM2-500M-Video-Instruct-Int8
This version of SmolVLM2-500M-Video-Instruct has been converted to run on the Axera NPU using w8a16 quantization.
Compatible with Pulsar2 version: 4.0
Convert tools links:
For those who are interested in model conversion, you can try to export axmodel through the original repo:
Support Platform
- AX650
How to use
Download all files from this repository to the device.
Using AX650 Board
ai@ai-bj ~/yongqiang/SmolVLM2-500M-Video-Instruct $ tree -L 1
.
├── assets
├── embeds
├── infer_axmodel.py
├── README.md
├── smolvlm2_axmodel
├── smolvlm2_tokenizer
└── vit_mdoel
5 directories, 2 files
Inference with AX650 Host, such as M4N-Dock(爱芯派Pro) or AX650N DEMO Board
Multimodal Understanding
input image
input text:
Can you describe this image?
log information:
ai@ai-bj ~/yongqiang/SmolVLM2-500M-Video-Instruct $ python3 infer_axmodel.py
input prompt: Can you describe this image?
answer >> The image captures a close-up view of a pink flower, prominently featuring a bumblebee. The bumblebee, with its black and yellow stripes, is in the center of the frame, its body slightly tilted to the left. The flower, with its petals fully spread, is the main subject of the image. The background is blurred, drawing focus to the flower and the bumblebee. The blurred background suggests a garden or a field, providing a sense of depth to the image. The^@ colors in the image are vibrant, with the pink of the flower contrasting against the green of the leaves and the brown of the stems. The image does not provide enough detail to confidently identify the specific location or landmark referred to as "sa_16743".