For the record, proofs that the KL divergence is non-negative assume that both distributions are normalised with respect to the same measure. If however the variables with respect to which the distributions are computed differ (for example, you compare distributions over two layers with different dimensions), then it is possible to obtain a negative result. There’s nothing pathological about this per se, it just means you need to renormalise your target distribution.
Unsure if this is what’s causing your problem, but it’s worth keeping this technical caveat in mind, since it can arise in some applications.