File size: 1,485 Bytes
c26b9c4 98d3f30 c26b9c4 722c750 c26b9c4 98d3f30 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
---
inference: false
library_name: transformers
---
# VW-LMM Model Card
This repo contains the weights of VW-LMM-Vicuna-pif-7b proposed in paper "Multi-modal Auto-regressive Modeling via Visual Words"
The term "pif" in the model name stands for "<strong>p</strong>seudo <strong>i</strong>mage <strong>f</strong>eatures". This model is capable of accepting pseudo-image features constructed using the VM_head and word embeddings of the model as a substitute for real image inputs.
For specific usage and chat templates, please refer to our project repo https://github.com/pengts/VW-LMM
## Model details
**Model type:**
VW-LMM is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data.
It is an auto-regressive language model, based on the transformer architecture.
**paper:**
https://arxiv.org/abs/2403.07720
**code:**
https://github.com/pengts/VW-LMM
## License
Llama 2 is licensed under the LLAMA 2 Community License,
Copyright (c) Meta Platforms, Inc. All Rights Reserved.
## Citation
If you find our paper and code useful in your research, please consider giving a star :star: and citation :pencil:.
```BibTeX
@misc{peng2024multimodal,
title={Multi-modal Auto-regressive Modeling via Visual Words},
author={Tianshuo Peng and Zuchao Li and Lefei Zhang and Hai Zhao and Ping Wang and Bo Du},
year={2024},
eprint={2403.07720},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
``` |