On my list of things I’d like to see or do is an implementation of
Alemi et al.: Deep Variational Information Bottleneck 17
or
Shwartz-Ziv and Tishby: Opening the Black Box of Deep Neural Networks via Information 12
Also, I think the Bayesian neural network chapters in the textbook
MacKay: Information Theory, Inference, and Learning Algorithms 9
might be inspiration for some things you could do. Though for this, Soumith’s comment regarding numpy vs. torch may apply as well.
Naturally, I’d be very interested in what you do. 
Best regards
Thomas