About basic understanding of attention mechanism and model weights In the image domain, for a given image, suppose that we want to understand if there is a bird on that image (0 and 1 label = no bird and bird), attention mechanism helps to pay more attention to the birds and like blurring (giving less attention) the rest of the image. However, the thing that I do not understand, isn't this the same logic as the weights we learned in a deep model? The system learns what should be given more importance in order to learn the labels for the input. I can't understand what is the difference between the weight logic.

There are many different "kinds" of attention application.
I think Learn to Pay Attention has a interesting approach.
The article below provides a nice explanation of it. A little further down.
https://towardsdatascience.com/learn-to-pay-attention-trainable-visual-attention-in-cnns-87e2869f89f1
Even when ignoring the math it should become clear from that example that here attention cannot simply be replaced by the strengthening/weakening of weights.
Their attention is based on the compatibility score.

For example, if there’s a cat in the image, we assume that the whole cat is described by the global feature vector g, and furthermore, we expect that a particularly “cat-like” patch (e.g. a patch over the cat’s face) will produce local features l that produce a high compatibility score when combined with g

g is the regular feature vector obtained from the CNN. The patches are extracted from earlier layers in the CNN. So patches that "correlate" with the output are attended more than less correlated patches.
