Thanks for the code. I’m unfortunately unsure how to reproduce the “working” behavior without the permutation, as the code doesn’t provide any switch and removing the op yields shapes mismatches in the batchnorm layer and after fixing it an activation which is too small for the model.