I see a small numerical difference when finding the log probability with a Categorical distribution, but I can’t see why.
First, find the logits and sample an action:
logits = policy(state).log_softmax(-1)
m = torch.distributions.Categorical(logits=logits)
action = m.sample()

Now get the log probability of that action in two ways:
logProb = logits.squeeze()[action]

logProb = m.log_prob(action)

The two methods produce slightly different values after the 6th decimal place.
Shouldn’t these be exactly the same?
I’m not seeing the reason for the difference.