Thanks for the shapes. Could you check, if pred might be empty after the
pred = pred[lbl != ignore]

operation?
Based on your description I guess you might run into a scenario, where one device only gets ingore indices, and this indexing pred again might yield this error.
You could check the pred.size(0) for a 0 after the first indexing.