Distilling knowledge from neural networks to build smaller and faster models
Could a researcher chime in: is it possible that smaller models have a higher potential for mapping activations to abstractions that are present in our own language understanding and if so, could we use these makings to make our models more transparent and explainable?