harpreetsahota/car-dd-segmentation-yolov11

Car Damage Segmentation Model (YOLOv11-seg)

This model detects and segments six types of vehicle damage: cracks, dents, glass shatter, broken lamps, scratches, and flat tires. Fine-tuned on the CarDD dataset, it provides precise damage localization and segmentation to support vehicle inspection and repair estimation.

Model Details

• Architecture: YOLOv11-seg
• Base Model: yolo11x-seg.pt
• Task: Instance Segmentation
• Domain: Automotive Damage Assessment
• Framework: Ultralytics YOLOv11

Performance Metrics

Class-Specific Performance (Mask Segmentation)

       Class     Images  Instances      Box(P          R      mAP50  mAP50-95)     Mask(P          R      mAP50  mAP50-95)
         all        571       1247      0.824       0.75      0.799      0.599      0.827      0.749      0.792      0.576
       crack         92        152      0.657       0.48      0.545      0.322      0.665      0.483      0.518      0.214
        dent        249        366      0.706      0.571      0.633      0.377      0.697       0.56      0.612      0.344
glass shatter         89         91       0.98      0.989      0.994      0.728      0.981      0.989      0.994      0.784
 lamp broken        103        103       0.91      0.893      0.964      0.791      0.921      0.902      0.967      0.808
     scratch        281        482      0.728      0.614      0.676      0.421      0.734      0.613       0.68      0.368
   tire flat         50         53      0.962      0.951      0.982      0.951      0.962      0.949      0.982      0.941

Training Recipe

from ultralytics import YOLO

model = YOLO('yolo11x-seg.pt')

model.train(
    data='path/to/dataset.yaml',
    epochs=300,
    batch=16,
    imgsz=1024,
    dropout=0.1,
    multi_scale=True,
    val=True,
    patience=25,
    optimizer='AdamW',
    cls=0.3,              # Reduced to prioritize boundary precision over classification
    dfl=1.7,              # Increased for better damage boundary delineation
    lr0=0.0001,
    lrf=0.01,
    warmup_epochs=10,     # Extended warmup for complex damage patterns
    mosaic=1.0,
    mixup=0.1,
    copy_paste=0.1,
    hsv_h=0.015,
    hsv_s=0.7,
    hsv_v=0.4,
    cos_lr=True,
    amp=True,
    overlap_mask=True     # Better segmentation for irregular damage shapes
)

Training Recipe for Car Damage Segmentation

• imgsz=1024: Higher resolution critical for detecting small damage features like fine cracks and scratches

• optimizer='AdamW': Provides better convergence for fine-grained segmentation tasks compared to SGD

• dropout=0.1: Light regularization prevents overfitting while preserving ability to detect subtle damage features

• multi_scale=True: Enhances model robustness to different damage sizes and viewing distances

• warmup_epochs=10: Extended warmup period stabilizes early training on complex damage patterns

• overlap_mask=True: Improves segmentation quality for irregular damage shapes with complex boundaries

cls=0.3

• Reduced classification weight prioritizes boundary precision over class distinction

• Similar visual appearance between damage types (scratches vs. cracks)

• Prevents overfitting to specific damage classes

• Improves localization and segmentation accuracy

dfl=1.7

• Increased DFL for better segmentation of irregular damage boundaries

• Addresses complex boundaries of car damage shapes

• Improves bounding box precision

• Effective for elongated damages like scratches

Dataset

The model was trained on the CarDD (Car Damage Detection) dataset with:

• 571 images
• 1247 total damage instances
• Class distribution: scratches (482), dents (366), cracks (152), broken lamps (103), glass shatter (91), and flat tires (53)

Usage

Download weight: !wget https://huggingface.co/harpreetsahota/car-dd-segmentation-yolov11/resolve/main/best.pt -O yolov11-seg-cardd.pt

from ultralytics import YOLO
import os
import gdown

# Load the model
model = YOLO('best.pt')

# Apply the model to the test dataset
# FiftyOne automatically handles batching and processing
test_dataset.apply_model(
    model,
    label_field="yolo_predictions",  # Field name to store predictions
    confidence_thresh=0.25,          # Minimum confidence threshold
    batch_size=8                     # Adjust based on your GPU memory
)

# Define the class list to match your model's classes
classes = ["crack", "dent", "glass shatter", "lamp broken", "scratch", "tire flat"]

# Evaluate detections against ground truth
eval_results = test_dataset.evaluate_detections(
    "yolo_predictions",              # Field containing model predictions  
    gt_field="segmentations",        # Field containing ground truth
    eval_key="model_eval",           # Key to store evaluation results
    use_masks=True,                  # Use pixel masks for IoU calculation
    compute_mAP=True,                # Compute mean Average Precision
    method="coco",                   # COCO evaluation protocol
    classes=classes,                 # Class list
    iou=0.50                         # IoU threshold
)

# Print evaluation results
print(eval_results)

# Visualize evaluation results in the FiftyOne App
from fiftyone import ViewField as F

# Create a view showing false positives with high confidence
high_conf_fp_view = test_dataset.filter_labels(
    "yolo_predictions", 
    (F("confidence") > 0.7) & (F("eval") == "fp")
)

# Sort by false positive count
sorted_view = high_conf_fp_view.sort_by(
    F("yolo_predictions.detections").length(), reverse=True
)

# Launch the app with this view
session = fo.launch_app(sorted_view)

Limitations

• Lower recall for crack detection (36.8%)
• Performance varies across damage types
• Small or fine damages may be missed
• Best results with well-lit, clear images of vehicles

License

Since this model was trained using Ultralytics, it falls under the AGPL-3.0 License.