Hi,
I guess that the issue is caused by Pytorch framework, because the phenomenon doesn’t happen in TensorFlow.
Finally, I force to set value, which less than 1.0e-6, to 0.0 and the performance do not affect.
I hope that this is work to you. Thanks.