---
library_name: transformers
license: apache-2.0
base_model: google/siglip2-base-patch16-224
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: siglip2-finetuned-marathi-sign-language
  results: []
datasets:
- VinayHajare/Marathi-Sign-Language
language:
- mr
pipeline_tag: image-classification
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# siglip2-finetuned-marathi-sign-language

This model is a fine-tuned version of [google/siglip2-base-patch16-224](https://huggingface.co/google/siglip2-base-patch16-224) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0006
- Model Preparation Time: 0.0057
- Accuracy: 0.9997

## Model description

Marathi-Sign-Language-Detection is a vision-language model fine-tuned from google/siglip2-base-patch16-224 for multi-class image classification. It is trained to recognize Marathi sign language hand gestures and map them to corresponding Devanagari characters using the SiglipForImageClassification architecture.

## Training and evaluation data

```java
Classification Report:
                precision    recall  f1-score   support

           अ     1.0000    1.0000    1.0000       404
           आ     1.0000    1.0000    1.0000       409
           इ     1.0000    1.0000    1.0000       440
           ई     0.9866    1.0000    0.9932       441
           उ     1.0000    1.0000    1.0000       479
           ऊ     1.0000    1.0000    1.0000       428
           ए     1.0000    1.0000    1.0000       457
           ऐ     1.0000    1.0000    1.0000       436
           ओ     1.0000    1.0000    1.0000       430
           औ     1.0000    1.0000    1.0000       408
           क     1.0000    1.0000    1.0000       433
           क्ष     1.0000    1.0000    1.0000       480
           ख     1.0000    1.0000    1.0000       456
           ग     1.0000    1.0000    1.0000       444
           घ     1.0000    1.0000    1.0000       480
           च     1.0000    1.0000    1.0000       463
           छ     1.0000    1.0000    1.0000       468
           ज     1.0000    1.0000    1.0000       480
           ज्ञ     1.0000    1.0000    1.0000       480
           झ     1.0000    1.0000    1.0000       480
           ट     1.0000    1.0000    1.0000       480
           ठ     1.0000    1.0000    1.0000       480
           ड     1.0000    1.0000    1.0000       480
           ढ     1.0000    1.0000    1.0000       480
           ण     1.0000    1.0000    1.0000       480
           त     1.0000    1.0000    1.0000       480
           थ     1.0000    1.0000    1.0000       480
           द     1.0000    0.9875    0.9937       480
           ध     1.0000    1.0000    1.0000       480
           न     1.0000    1.0000    1.0000       480
           प     1.0000    1.0000    1.0000       480
           फ     1.0000    1.0000    1.0000       480
           ब     1.0000    1.0000    1.0000       480
           भ     1.0000    1.0000    1.0000       480
           म     1.0000    1.0000    1.0000       480
           य     1.0000    1.0000    1.0000       480
           र     1.0000    1.0000    1.0000       484
           ल     1.0000    1.0000    1.0000       480
           ळ     1.0000    1.0000    1.0000       480
           व     1.0000    1.0000    1.0000       480
           श     1.0000    1.0000    1.0000       480
           स     1.0000    1.0000    1.0000       480
           ह     1.0000    1.0000    1.0000       480

    accuracy                         0.9997     20040
   macro avg     0.9997    0.9997    0.9997     20040
weighted avg     0.9997    0.9997    0.9997     20040
```
---

## Install Dependencies

```bash
pip install -q transformers torch pillow gradio
```

---

## Inference Code

```python
import gradio as gr
from transformers import AutoImageProcessor, SiglipForImageClassification
from PIL import Image
import torch

# Load model and processor
model_name = "VinayHajare/siglip2-finetuned-marathi-sign-language"
model = SiglipForImageClassification.from_pretrained(model_name)
processor = AutoImageProcessor.from_pretrained(model_name)

# Marathi label mapping
id2label = {
    "0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ",
    "6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष",
    "12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज",
    "18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ",
    "24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न",
    "30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य",
    "36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह"
}

def classify_marathi_sign(image):
    image = Image.fromarray(image).convert("RGB")
    inputs = processor(images=image, return_tensors="pt")

    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits
        probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist()

    prediction = {
        id2label[str(i)]: round(probs[i], 3) for i in range(len(probs))
    }

    return prediction

# Gradio Interface
iface = gr.Interface(
    fn=classify_marathi_sign,
    inputs=gr.Image(type="numpy"),
    outputs=gr.Label(num_top_classes=5, label="Marathi Sign Classification"),
    title="Marathi-Sign-Language-Detection",
    description="Upload an image of a Marathi sign language hand gesture to identify the corresponding character."
)

if __name__ == "__main__":
    iface.launch()
```

---
## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-06
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- num_epochs: 6

### Training results

| Training Loss | Epoch | Step | Validation Loss | Model Preparation Time | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:----------------------:|:--------:|
| 1.4439        | 1.0   | 940  | 0.0090          | 0.0057                 | 0.9980   |
| 0.0052        | 2.0   | 1880 | 0.0035          | 0.0057                 | 0.9993   |
| 0.0031        | 3.0   | 2820 | 0.0016          | 0.0057                 | 0.9997   |
| 0.001         | 4.0   | 3760 | 0.0010          | 0.0057                 | 0.9997   |
| 0.0007        | 5.0   | 4700 | 0.0013          | 0.0057                 | 0.9997   |
| 0.0005        | 6.0   | 5640 | 0.0006          | 0.0057                 | 0.9997   |


### Framework versions

- Transformers 4.52.0.dev0
- Pytorch 2.6.0+cu124
- Datasets 3.5.1
- Tokenizers 0.21.1