I am trying to understand why the following simple code module does act as an attention layer? Most of what I am finding in blogs/papers is quite more complex. Is there a mathematical or intuitive way of understanding the following as an attention layer?
  self.layer_attend1 =  nn.Sequential(nn.Conv2d(layers[0], layers[0], stride=2, padding=1, kernel_size=3),
                                            nn.AdaptiveAvgPool2d(1),
                                            nn.Softmax(1))