Visual attention mechanisms have been widely used in image captioning models. In this paper, to better link the image structure with the generated text, we replace the traditional softmax attention mechanism by two alternative sparsity-promoting transformations: sparsemax and Total-Variation Sparse Attention (TVMAX). With sparsemax, we obtain sparse attention weights, selecting relevant features. In order to promote sparsity and encourage fusing of the related adjacent spatial locations, we propose TVMAX. By selecting relevant groups of features, the TV-MAX transformation improves interpretability. We present results in the Microsoft COCO and Flickr30k datasets, obtaining gains in comparison to softmax. TV-MAX outperforms the other compared attention mechanisms in terms of humanrated caption quality and attention relevance.
This paper proposes two sparsifying methods of computing attention weights, dubbed sparsemax and TVmax, which appear to slightly improve objective and subjective image captioning scores.    The sparsifying projections are posed as optimization problems, and algorithms for their computation, along with formula for their gradients are given.  Proof of the optimality of these algorithms relies significantly on prior work, so could not be checked deeply without bringing in additional sources.  
It is not clear that the motivation for these sparsifying objectives is sound.   The conventional softmax approach to attention weights should be capable of producing attention weights near zero, which would be effectively sparse, especially if the pre-activations, z_i, in equation (1), are allowed to have a large enough range.   It's not clear why weights should need to be zero exactly in the ignored regions, since being near zero should be sufficient to contribute almost nothing to the subsequent weighted sum.    So it is also not clear why the strict sparsity itself, as opposed to the effective sparsity of the softmax, should explain the differences in Figure 1, and in the results.  In particular it is unclear why the strict sparsity should prevent repetition; when looking at the weight distributions in the two cases, a more likely story seems to be that the weight distributions don't repeat as much from one word to the next in the second case, but there is no clear reason to attribute this to sparsity.   The pictures of the attention weights are lacking a color scale, so it is impossible to see how close to zero it comes in the unattended regions, although the gray color values chosen for these regions might be misleading. 
The TVmax approach, in addition to sparsity, also constrains the non-zero region to be contiguous.   To the extent that this improves performance, this presumably introduces an inductive bias that matches the data.   It is unclear why this fails to produce better objective scores than sparsemax, while producing better human ratings.   In any case it is not clear why this should necessarily be a good inductive bias for all images, although it is plausible that it helps in some cases.  
In many neural network problems, what makes a difference has more to do with the optimizability of the gradients, than the specific activations per se, and that might be the case here too, although the paper does not analyze this aspect of the proposed models.   
Overall the paper is flawed by the lack of clarity in the motivation for the proposed methods, and the lack of retrospective analysis and understanding of why the proposed methods should improve results.