I’m training an RL network, which is pretty unstable anyway. But I’ve fixed all the weight initializations, exploration probabilities etc, using random seeds.  Sometimes the weights of the network will drop to zero in a single step. It’s not very easy to repeat; it happens systematically, but on different episodes each time; even with the stochasticity’s fixed. It’s probably something stupid I’m doing somewhere, but just wondering if anyone else has encountered something similar?
(there are no nans, just… weights drop to exactly zero, in a single optimization step   I mean, it’ll take several thousand steps, without being zero. then Bam! all zeros )