Machine Learning, Neural Networks.
have attention. DB: What are some of the key issues around attention? IS: Differentiable attention is computationally expensive because it requires accessing your entire input at each step of the model’s operation. And this is fine when the input is a sentence that’s only, say, 100 words, but it’s