Machine Learning, Artificial Intelligence, Technology, Google Brain, Neural Networks.
the infinite width limit, we nevertheless find excellent empirical agreement between the predictions of the original network and those of the linearized version even for finite practically-sized networks. This agreement is robust across different architectures, optimization methods, and loss