This is not a question about the usage of pytorch, but a question about training problem, I am sorry but I want to get some advice from this forum.
I am training a model with my custom loss function, I think there is no error about the loss function, but when I check the training gradients, I found some thing odd, the message of gradients is:
norm: 0.000244	min: -0.000002	max: 0.000001	median: -0.000000	mean: 0.000000	std: 0.000001
And when I amplify the loss weight to 1000, the message is:
norm: 0.395154	min: -0.003929	max: 0.005707	median: 0.000015	mean: -0.000000	std: 0.000852
The average loss value in training dataset up to 24.0944.
Can you give me some advice about what’s wrong in my model? Thanks for your help.