Typically CNNs have decreasing spatial resolution, so the typical thing would be to use some of the resolution levels as hierarchy levels. The next thing is how to formulate the attention. The classic K. Xu et al.: Show, attend and tell 8 uses “positional” attention masks while  Lu et al.: Knowing when to look 5 have a query-based attention.
It would be interesting to hear about your results and experiences.
Best regards
Thomas