Hi
We are making a project about quantisation in a neural network layer and need to calculate the entropy of these quantised values. An example of the data that the entropy should get as input is: x = [1/8,1/8,4/8,2/8]. Currently we have the following function for entropy
def Ixxhat(bitdepth,xhat):
hist = torch.histc(xhat, 2**bitdepth, 0, 1)/(xhat.shape[0]*xhat.shape[1])
xhatProbs = torch.tensor([hist[i] * torch.log2(hist[i]) if hist[i] > 0 else 0 for i in range(len(hist))])
return - torch.sum(xhatProbs), hist

We read that as long as we use the torch functions it should work, but can someone verify this for this function and/or design an entropy function we could use instead 
Thanks in advance