Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs Paper • 2309.07311 • Published Sep 13, 2023 • 4
Opening the Black Box of Deep Neural Networks via Information Paper • 1703.00810 • Published Mar 2, 2017
To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review Paper • 2304.09355 • Published Apr 19, 2023 • 5