Stacked hourglass network has become an important model for Human pose estimation. The estimation of human body posture depends on the global information of the keypoints type and the local information of the keypoints location. The consistent processing of inputs and constraints makes it difficult to form differentiated and determined collaboration mechanisms for each stacked hourglass network. In this paper, we propose a Multi-Scale Stacked Hourglass (MSSH) network to highlight the differentiation capabilities of each Hourglass network for human pose estimation. The pre-processing network forms feature maps of different scales, and dispatch them to various locations of the stack hourglass network, where the small-scale features reach the front of stacked hourglass network, and large-scale features reach the rear of stacked hourglass network. And a new loss function is proposed for multi-scale stacked hourglass network. Different keypoints have different weight coefficients of loss function at different scales, and the keypoints weight coefficients are dynamically adjusted from the top-level hourglass network to the bottom-level hourglass network. Experimental results show that the proposed method is competitive with respect to the comparison algorithm on MPII and LSP datasets.
In this paper a method for pose estimation is proposed, which is based on the well known neural model “stacked hourglass networks”. The novelty in the proposed paper is a multi-scale formulation, which creates multiple scales from the input image and feeds them into different hourglass modules. The different scales are weighted differently, where the weights for a given scale depend on the error obtained on previous scales.
Important issues:
The novelty seems to be not sufficient to me, as multi-scale solutions are not new in computer vision and have been applied a lot in pose estimation as well, be it deep neural models or older techniques. In particular in deep models, multi-scale techniques have been proposed extensively for resolution preserving image to image mappings (which is done here), beginning with quite “old” techniques (in deep learning time scales) going back to 2012 (Farabet, C., Couprie, C., Najman, L., LeCun, Y., 2012. Scene parsing with multiscale feature learning, purity trees, and optimal covers. In: ICML.), or formulations which integrate scales and layers, starting with hyper-columns in 2015 (Hypercolumns for object segmentation and fine-grained localization, CVPR 2015) and many more recent variants.
In 2018, multi-scale formulations are now standard techniques in computer vision with deep networks. I am not sure how the proposed method makes a difference. Also, I am wondering whether there shouldn’t be some parameter sharing between the models of the different scales, as is often done in the literature now to reduce model capacity.
The way the multi-scale Since different resolutions are created is particular. Why not just simply subsample the input images to different resolutions? Why are these trained layers needed as a preprocessing?
Weighting different scales is not fundamentally new. We also don’t know whether it improves performance, it is not part of the ablation study.
The method has been compared to several methods, but which are not state of the art anymore. Most papers are from 2016, the field advanced quickly. The performance gains of the multi-scale formulation are pretty low, and overally, the method is not state of the art on the targeted benchmark. Obviously I do not want to say that a paper needs to be state of the art on a benchmark to get published, even at a top-level conference, far from it. However, if the methodological and theoretical contribution of the paper is rather minor, than the performance and evaluation should be flawless.
Minor remarks:
The writing of this paper is somewhat fuzzy, using non-standard technical language, which I could not decipher and which seems to me somewhat misleading. Examples are:
- “the stacked hourglass network theoretically increases the stacked depth”: how does theory tell us anything about depth in this context?
- “difficult to form differentiated and determined collaboration mechanisms for each stacked hourglass”: I don’t understand what this means, simply. This phrase is also repeated several times in the paper in different places
- “information loss caused by the functional consistency of hourglass networks”: I don’t understand what functional consistency means here, and why it leads to information loss.
The description of Algorithm 1 does not seem to be necessary to me. It basically follows from the equations 4-6, which are executed in order and per layer.