Oops! Good catch!
Turns out that in the implementation, they calculate kl divergence (the cross entropy is calculated by using tensor ops to accomodate for this).
Thanks Richard!
Sorry for the confusion