Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations
We present a method for estimating articulated human pose from a single static image based on a graphical model with novel pairwise relations that make adaptive use of local image measurements. More precisely, we specify a graphical model for human pose which exploits the fact the local image measurements can be used both to detect parts (or joints) and also to predict the spatial relationships between them (Image Dependent Pairwise Relations). These spatial relationships are represented by a mixture model. We use Deep Convolutional Neural Networks (DCNNs) to learn conditional probabilities for the presence of parts and their spatial relationships within image patches. Hence our model combines the representational flexibility of graphical models with the efficiency and statistical power of DCNNs. Our method significantly outperforms the state of the art methods on the LSP and FLIC datasets and also performs very well on the Buffy dataset without any training.
This work proposes a method for monocular human pose estimation. Theproposed method is based on a tree-structured graphical model withimage dependent pairwise relationships. These relationships arerepresented by a mixture model and local image measurements are usedto predict a mixture component for each pairwise term. Prediction isbased on features extracted from image patches using DeepConvolutional Neural Networks (DCNN). DCNN are trained to predictconditional probabilities for 1) spatial relations between body partsand 2) presence of individual body parts. Finally, mixture weightsunary and pairwise terms are jointly learned in a max-marginframework.This is an interesting paper with several strong points:- significance. This paper addresses a challenging and highly research problem of  monocular human pose estimation and thus it may have an impact on  significant part of community- model. Using deep learning features for local conditioning of  pairwise terms in a graphical model is an interesting idea.- experimental evaluation. The work provides some insights by breaking  down the performance of individual model components. The importance  of image conditioned pairwise terms for pose estimation performance  is evident.- results. This work shows promising results significantly  improving over state of the art on standard pose estimation benchmarks.However, this paper also has several weak points:- conditioning of pairwise terms in a graphical model for human pose  estimation has been done before by [SJT10] (see citation below) and  [11]. I struggle to see the principal differences between this  method and the previous work, except that this paper uses  stronger image features and local conditioning. Moreover, it is not  clear that local conditioning of pairwise terms has more advantages  over global conditioning, as local conditioning allows to model  pairwise relationships between adjacent body parts *only*, which is  in contrast to the previous method [11] which models longer range  part dependencies. - the proposed method heavily builds on a Yang&Ramanan (Y&R) model  [20,21] and reads like an extension of Y&R model with  image-conditioned pairwise terms.- choosing a number of mixture components for the pairwise terms (11)  is not justified. Y&R model [21] uses 6 components only. Fairer  comparison between two models would be when the number of mixture  components is the same.- using 0-1 loss during training is suboptimal, as at test time  different types of losses measuring the fraction of correctly  predicted body parts/joints are used. A more proper way would be to  use the same loss during testing and training.- comparison to Deep Pose [17] on LSP dataset is not completely fair,  as [17] report results using person-centric annotations, which is a  harder task.- some citations are missing[SJT10]  B. Sapp, C. Jordan, and B. Taskar. Adaptive pose priors for     	 pictorial structures. In CVPR’10. [KU12]	 L. Karlinsky, and U. Shimon. Using linking features in learning     	 non-parametric part models. In ECCV'12- unrealistic poses resulting from data augmentation by rotating each  positive training image through 360 with a step of 10 degrees may  hurt the performance. The paper advocates this procedure by saying  that the patches extracted from the unrealistically rotated poses  still contain information about local part relations. However, it  has been suggested [20,21] that the appearance of body parts is  rotation dependent, and thus is characteristic for particular part  orientations. Moreover, the local patches seem to be large enough to  include a significant portion of the context, e.g. supporting  surfaces for legs in upright or for arms in upside down positions,  which may enhance discriminative power.Minor comments:- saying that the method "performs very well on Buddy dataset without  any training" is a bit misleading. What is actually meant is that  the method was not trained on Buffy dataset, but trained on other  datasets and tested on Buffy.- what is the training/testing time of the method?