In that case try to compare the intermediate activations between both approaches to narrow down where the difference might be coming from. If you are using if conditions, check them of course as well to see which code path is taken in the forward method.