hooks revealed that the further the gradients were propagated the larger they got (pretty rapidly) until they overflowed.
Interestingly, replacing .std() with .var() fixes the problem and the network will train properly. Any ideas on why that is?