I see ok. Do you have any debugging tips as to why the gradients hit 0? At the start of the training for some initializations, I see some gradients. But that diminishes pretty quickly. Seems to happen if my weight_transform layer outputs from a non-linearity like Tanh() or Sigmoid()