Hi @vladsb94 , I checked out yolov5 and applied the code you provided (gist:d863f53c8809198b3e0a4fd2af1563a7 · GitHub 9).  I’m not sure how you got things to work without additional changes.  This model uses the SiLU activation which does not have an int8 kernel.  To make this model quantizeable to int8, there are a couple of options:

add an int8 kernel for SiLU (we would happily accept a PR)
add a custom quantizeable module for SiLU (it can just do dequant → SiLU → quant)
make yolov5 symbolically traceable and use FX graph mode quantization