You are interpolating values using the bilinear approach and rounding afterwards, which might change the values. I’m not familiar with your use case, but as previously described, the expected target values are in [0, nb_classes-1] unless you use ignore_index for a specific index value.
Your current code crashes, because the target values are not in this range.