transformers safetensors spec_vision text-generation spec-vision vision-language-model image-text-to-text conversational custom_code multilingual license:mit autotrain_compatible region:us
---
language:
- multilingual
license: mit
pipeline_tag: image-text-to-text
tags:
- spec-vision
- vision-language-model
- transformers
---

# Model Summary

Spec-Vision-V1 is a lightweight, state-of-the-art open multimodal model built on datasets that include synthetic data and filtered publicly available sources, with a focus on high-quality, reasoning-dense data in both text and vision. The model belongs to the SpecVision family and supports a 128K context length (in tokens). It has undergone a rigorous enhancement process, incorporating supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures.

# 🚀 Model Overview  

**Spec-Vision-V1** is built for **deep integration of visual and textual data**, enabling it to understand and process images in combination with natural language. The model has been trained on a diverse dataset containing images with associated captions, descriptions, and contextual information.  

### ✨ Key Features

- **🖼️ Multimodal Processing**: Seamlessly combines image and text inputs.
- **⚡ Transformer-Based Architecture**: High efficiency in vision-language understanding.
- **📝 Optimized for VQA & Captioning**: Excels in answering visual questions and generating descriptions.
- **📥 Pre-trained Model**: Available for inference and fine-tuning.

---

## 📌 Installation

To use Spec-Vision-V1, install the required dependencies:

```bash
pip install transformers torch torchvision pillow
```

---

## 🔥 Usage

### 📥 Load the Model

```python
from transformers import AutoModelForCausalLM, AutoProcessor
from PIL import Image
import torch

# Load the model and processor
model_name = "Spec-Vision-V1"
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)

# Load an example image
image = Image.open("example.jpg")

# Input text prompt
text = "Describe the image in detail."

# Process inputs
inputs = processor(images=image, text=text, return_tensors="pt")

# Generate output
with torch.no_grad():
    outputs = model(**inputs)

# Print the generated text
print(outputs)
```

---

## 📊 Model Specifications

| Attribute        | Description                                  |
|-----------------|----------------------------------------------|
| **Model Name**  | Spec-Vision-V1                               |
| **Architecture** | Transformer-based Vision-Language Model    |
| **Pretrained**  | ✅ Yes                                      |
| **Dataset**     | Trained on diverse image-text pairs        |
| **Framework**   | PyTorch & Hugging Face Transformers        |

---

## 🎯 Applications

| Task                     | Description                                                   |
|--------------------------|--------------------------------------------------------------|
| **🖼️ Image Captioning**    | Generates detailed descriptions for input images.         |
| **🧐 Visual Question Answering** | Answers questions about images.                  |
| **🔎 Image-Text Matching**  | Determines the relevance of an image to a given text.  |
| **🌍 Scene Understanding**  | Extracts insights from complex visual data.              |

---

## BLINK Benchmark

A benchmark with 14 visual tasks that humans can solve very quickly but are still hard for current multimodal LLMs.

| Benchmark                | Spec-Vision-V1 | LlaVA-Interleave-Qwen-7B | InternVL-2-4B | InternVL-2-8B | Gemini-1.5-Flash | GPT-4o-mini | Claude-3.5-Sonnet | Gemini-1.5-Pro | GPT-4o |
|--------------------------|--------------|--------------------------|---------------|---------------|------------------|-------------|-------------------|----------------|--------|
| Art Style               | 87.2         | 62.4                     | 55.6          | 52.1          | 64.1             | 70.1        | 59.8              | 70.9           | 73.3   |
| Counting                | 54.2         | 56.7                     | 54.2          | 66.7          | 51.7             | 55.0        | 59.2              | 65.0           | 65.0   |
| Forensic Detection      | 92.4         | 31.1                     | 40.9          | 34.1          | 54.5             | 38.6        | 67.4              | 60.6           | 75.8   |
| Functional Correspondence | 29.2       | 34.6                     | 24.6          | 24.6          | 33.1             | 26.9        | 33.8              | 31.5           | 43.8   |
| IQ Test                 | 25.3         | 26.7                     | 26.0          | 30.7          | 25.3             | 29.3        | 26.0              | 34.0           | 19.3   |
| Jigsaw                  | 68.0         | 86.0                     | 55.3          | 52.7          | 71.3             | 72.7        | 57.3              | 68.0           | 67.3   |
| Multi-View Reasoning    | 54.1         | 44.4                     | 48.9          | 42.9          | 48.9             | 48.1        | 55.6              | 49.6           | 46.6   |
| Object Localization     | 49.2         | 54.9                     | 53.3          | 54.1          | 44.3             | 57.4        | 62.3              | 65.6           | 68.0   |
| Relative Depth          | 69.4         | 77.4                     | 63.7          | 67.7          | 57.3             | 58.1        | 71.8              | 76.6           | 71.0   |
| Relative Reflectance    | 37.3         | 34.3                     | 32.8          | 38.8          | 32.8             | 27.6        | 36.6              | 38.8           | 40.3   |
| Semantic Correspondence | 36.7         | 31.7                     | 31.7          | 22.3          | 32.4             | 31.7        | 45.3              | 48.9           | 54.0   |
| Spatial Relation       | 65.7         | 75.5                     | 78.3          | 78.3          | 55.9             | 81.1        | 60.1              | 79.0           | 84.6   |
| Visual Correspondence  | 53.5         | 40.7                     | 34.9          | 33.1          | 29.7             | 52.9        | 72.1              | 81.4           | 86.0   |
| Visual Similarity      | 83.0         | 91.9                     | 48.1          | 45.2          | 47.4             | 77.8        | 84.4              | 81.5           | 88.1   |
| **Overall**            | **57.0**     | **53.1**                 | **45.9**      | **45.4**      | **45.8**         | **51.9**    | **56.5**          | **61.0**       | **63.2** |

---

## Video-MME Benchmark

A benchmark that comprehensively assesses the capabilities of multimodal LLMs in processing video data, covering a wide range of visual domains, temporal durations, and data modalities.

| Benchmark               | Spec-Vision-V1 | LlaVA-Interleave-Qwen-7B | InternVL-2-4B | InternVL-2-8B | Gemini-1.5-Flash | GPT-4o-mini | Claude-3.5-Sonnet | Gemini-1.5-Pro | GPT-4o |
|-------------------------|--------------|--------------------------|---------------|---------------|------------------|-------------|-------------------|----------------|--------|
| Short (<2min)          | 60.8         | 62.3                     | 60.7          | 61.7          | 72.2             | 70.1        | 66.3              | 73.3           | 77.7   |
| Medium (4-15min)       | 47.7         | 47.1                     | 46.4          | 49.6          | 62.7             | 59.6        | 54.7              | 61.2           | 68.0   |
| Long (30-60min)        | 43.8         | 41.2                     | 42.6          | 46.6          | 52.1             | 53.9        | 46.6              | 53.2           | 59.6   |
| **Overall**            | **50.8**     | **50.2**                 | **49.9**      | **52.6**      | **62.3**         | **61.2**    | **55.9**          | **62.6**       | **68.4** |


---

## 🏗️ Model Training Details

| Parameter             | Value                          |
|----------------------|--------------------------------|
| **Batch Size**      | 16                             |
| **Optimizer**       | AdamW                          |
| **Learning Rate**   | 5e-5                           |
| **Training Steps**  | 100k                           |
| **Loss Function**   | CrossEntropyLoss               |
| **Framework**       | PyTorch & Transformers         |

---

## 📜 License

**Spec-Vision-V1** is released under the **MIT**.

---

## 📖 Citation

If you use **Spec-Vision-V1** in your research or application, please cite:

```bibtex
@article{SpecVision2025,
  title={Spec-Vision-V1: A Vision-Language Transformer Model},
  author={SVECTOR},
  year={2025},
  journal={SVECTOR Research}
}
```

---

## 📬 Contact

For support or inquiries, reach out to **SVECTOR**:

- **🌐 Website**: [svector.co.in](https://www.svector.co.in)
- **📧 Email**: [Research@svector.co.in](Research@svector.co.in)
- **✨ GitHub**: [SVECTOR GitHub](https://github.com/SVECTOR-CORPORATION)