I know this is an old conversation but I am developing a full fledged package for information theory of deep learning in PyTorch which have a lot of information bottleneck functionalities including HSIC bottleneck sigma networks (yes ! that train without backprop). Currently library is in testing phase but there are notebooks available, thought that can be useful.
Link to the repo:

  
      
      GitHub
  
  
    

spino17/PyGlow 8

This package is an attempt to implement Keras like API functionalities on PyTorch backend with functionalities supporting information theoretic methods which are relevant for understanding neural n...


  
  
    
    
  
  


Link to the documentation:
https://pyglow.github.io/ 9