dev-bjoern/smollm3-int4-ov

SmolLM3 INT4 OpenVINO

🚀 Optimized for Edge Deployment

This is an INT4 quantized version of SmolLM3-3B using OpenVINO, designed for efficient inference on edge devices and CPUs.

Model Overview

• Base Model: SmolLM3-3B (3B parameters)
• Quantization: INT4 via OpenVINO
• Size Reduction: ~75% smaller than original
• Target Hardware: CPUs, Intel GPUs, NPUs
• Use Cases: Local inference, edge deployment, resource-constrained environments

🔧 Technical Details

Quantization Process

# Quantized using OpenVINO NNCF
# INT4 symmetric quantization