After receiving a memory error I ran my training loop using CUDA_LAUNCH_BLOCKING=1 and now receive this rather esoteric error: max_unpooling3d_forward_kernel failed with error code 0. What does error code 0 stand for? Also, this happens after 3 successful forward and backward passes through the network.
As I changed a number of things about my training loop from the last working version, without touching the actual network, having an hint where to look for a bug would really help.