I see okay, so it means I should be doing something like:
output = model(inputs)

loss = loss_function(output, labels)
model.weight_transform.register_hook()
loss.backward()

print(model.weight_transform.grad)

Hmm okay, I’ll have a look to see if a better initialization can help get gradients. How can I tell if at the last stage of the output, the gradients are likely going to be 0? Is there any way for this? One explanation could be if the input is 0 right?