****Multi-head Attention****


### **Multiple heads lets the model consider multiple words simultaneously.**
Because we use the softmax function in attention, it amplifies the highest value while squashing the lower ones. As a result, each head tends to focus on a single element.
Multiple heads let us attend to several words. **It also provides redundancy**, where if any single head fails, we have the other attention heads to rely on.
입력 벡터를 여러 개의 헤드로 분할하여 각각 어텐션을 수행하고 결과를 결합하는 방식으로 동작



> [Some Intuition on Attention and the Transformer](https://eugeneyan.com/writing/attention/)




