Hi @ptrblck! How would you suggest to proceed if we want to backprop through the MI calculation? Say, we want to learn a mapping from X (e.g. a given hidden layer) to Z (e.g. a compressed version of X) so that the mutual info between X and Z is maximal. How would you suggest to proceed?
I have come across this paper (https://arxiv.org/pdf/1801.04062.pdf 2) which builds a neural estimator of the MI which can be backpropagated through, but I was wondering if there was a simpler way. Thanks