README.md

---
title: SAM-Grounding-DINO
emoji: 🎭
colorFrom: indigo
colorTo: purple
sdk: gradio
app_file: app.py
---
# 🎭 SAM 2.1 + Grounding DINO Interactive Segmentation

A web application combining Meta's SAM 2.1 and Grounding DINO for both text-based and point-based image segmentation to enable creating and downloading a desired mask.

## ✨ Features

- **🔍 Text-Based Segmentation**: Type what you want to segment (e.g., "snoopy", "person", "car")
- **📍 Point-Based Segmentation**: Click on objects for precise manual control
- **🎭 Multiple Mask Generation**: Generate 1-5 masks and browse through them
- **🤖 SAM 2.1 + Grounding DINO**: Powered by Meta's SAM 2.1 and IDEA Research's Grounding DINO
- **📱 Smart Auto-Detection**: Automatically chooses between text and point modes
- **💾 Multiple Export Formats**: Download masks as PNG, JPG, or PyTorch tensors
- **🖼️ High-Resolution Display**: View images and masks in full detail
- **⚡ Real-Time Processing**: Fast inference with GPU acceleration

## 🚀 Quick Start

### Installation

1. Clone or download the repository
2. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```

### Running the App

```bash
streamlit run streamlit_sam_app.py
```

The app will open in your browser at `http://localhost:8501`

## 🎯 How to Use

### 1. Upload an Image
- Click "📷 Upload an image" to select an image file
- Supported formats: JPG, JPEG, PNG, BMP

### 2. Add Points
Choose between **Positive** (include) or **Negative** (exclude) point mode:

#### Quick Presets:
- **🎯 Center**: Add point at image center
- **↖️ Top-Left**: Add point at top-left quarter
- **↗️ Top-Right**: Add point at top-right quarter
- **🎲 Random**: Add random point anywhere

#### Manual Input:
- Enter X,Y coordinates manually
- Points are validated against image boundaries

### 3. Generate Segmentation Mask
- Click "🎯 Generate Segmentation Mask"
- Adjust the mask threshold in the sidebar (0.0-1.0)
- Wait for SAM 2.0 to process (may take 10-30 seconds)

### 4. View Results
- **Original Image with Points**: Shows your input selections
- **Generated Segmentation Mask**: Red overlay on original image
- **Binary Mask Preview**: Black/white mask for download
- **Statistics**: Pixel counts and coverage percentage

### 5. Download Results
- **📥 Download Mask (PNG)**: Binary mask file
- **📥 Download Overlay (PNG)**: Mask overlaid on original
- **📥 Download Data (JSON)**: Complete metadata and statistics

## 🎛️ Advanced Controls

### Sidebar Options:
- **Point Mode**: Switch between Positive/Negative points
- **Mask Threshold**: Control mask sensitivity (lower = larger masks)
- **Clear Points**: Remove all points at once

### Point Management:
- View all current points with coordinates
- Delete individual points with 🗑️ buttons
- Real-time count of positive/negative points

## 🔧 Technical Details

### SAM 2.0 Model
- Uses `facebook/sam2-hiera-small` by default
- Automatically downloads model weights on first run
- Runs on GPU if available, CPU otherwise

### Dependencies
- `streamlit`: Web interface
- `torch`: PyTorch for model inference
- `transformers`: Hugging Face model loading
- `PIL`: Image processing
- `matplotlib`: Visualization
- `numpy`: Numerical operations
- `opencv-python`: Image processing utilities

### System Requirements
- Python 3.8+
- 4GB+ RAM recommended
- GPU recommended for faster processing

## 🐛 Troubleshooting

### Common Issues:

1. **Model Download Fails**:
   - Check internet connection
   - Ensure Hugging Face access (may require token for some models)

2. **CUDA Out of Memory**:
   - Try smaller model size
   - Reduce image resolution
   - Use CPU mode: set `CUDA_VISIBLE_DEVICES=""`

3. **Slow Processing**:
   - Use GPU if available
   - Try `sam2-hiera-tiny` model for faster inference

4. **Import Errors**:
   - Ensure all dependencies are installed: `pip install -r requirements.txt`

## 📁 File Structure

```
SAM/
├── streamlit_sam_app.py    # Main application
├── fixed_sam_interface.py  # Original Gradio version
├── requirements.txt        # Dependencies
└── README.md              # This file
```

## 🎨 Interface Screenshots

The app features a clean, modern interface with:
- Full-width image display
- Intuitive sidebar controls
- Real-time point visualization
- Side-by-side result comparison
- Comprehensive download options

## 🤝 Contributing

Feel free to submit issues, feature requests, or pull requests!

## 📄 License

This project uses Meta's SAM 2.0 model. Please refer to Meta's license terms for the model weights.

## 🙏 Acknowledgments

- Meta AI for the incredible SAM 2.0 model
- Streamlit for the amazing web app framework
- Hugging Face for model hosting
- The open-source community for all the dependencies