I’ve noticed that the pytorch implementation of KL divergence yells different results from the tensorflow implementation. The results differ significantly (0.20, and 0.14) and I was curios what could be the reason. Below you can find a small example. Any help will be more than appreciated.
import tensorflow as tf
import numpy as np
import torch
from torch.distributions.kl import kl_divergence
tf.enable_eager_execution()

preds = np.array([1.9417487e-03, 9.9999997e-10, 5.8252434e-03, 9.9999997e-10, 3.8834962e-03,
 8.1553400e-02, 3.6893204e-01, 5.2427185e-01, 7.7669914e-03, 5.8252434e-03,
 9.9999997e-10]).astype('float32')

labels = np.array([1.0362695e-02, 9.9999997e-10, 9.9999997e-10, 9.9999997e-10, 3.1088084e-02,
 9.0673573e-02, 3.4974092e-01, 5.1036268e-01, 2.5906744e-03, 9.9999997e-10,
 5.1813480e-03]).astype('float32')

preds_tf = tf.distributions.Categorical(probs=tf.convert_to_tensor(preds))
labels_tf = tf.distributions.Categorical(probs=tf.convert_to_tensor(labels))
tf_res = tf.distributions.kl_divergence(preds_tf, labels_tf)

preds_torch = torch.distributions.Categorical(probs=torch.from_numpy(preds))
labels_torch = torch.distributions.Categorical(probs=torch.from_numpy(labels))
torch_res = kl_divergence(preds_torch, labels_torch)

print(tf_res.numpy(), torch_res.item())