I wrote an extensive IoU notebook for a talk on the JIT linked from https://lernapparat.de/pytorch-jit-android/ 929 . The usual way is to do “class agnostic” IoU and a standard classification loss (eg cross entropy), so multiclass happens only in the second.
I always recommend the SSD lecture from https://fast.ai/ 278 's second course.
Best regards
Thomas