Machine Learning, Deep Learning, Artificial Intelligence, Regularization, Deep Learning Book.
representational sparsity, where many of the activation values of the units are zero. In the figure below, h is a representation of x, which is sparse. Representational sparsity is obtained similarly to the way parameter sparsity is obtained, by placing a penalty on the representation h instead of