I don’t know what you are using exactly, but check if some functional API calls are performed in the forward pass and might differ between both approaches.
If that’s not the case, check if you’ve forgotten to remove/replace a pooling layer, which might change the shape etc.