I just tried a toy example and got the same result, so I guess it’s the correct behavior of kl_div…
image962×646 48.9 KB
I’m still a little confused about how the mean of unreduced result is guaranteed to be non-negative, though I know KL divergence should be non-negative. Any explanation is appreciated!