This paper handles a challenging problem, unseen attribute-object pair recognition, which asks a model to simultaneously recognize the attribute type and the object type of a given image while this attribute-object pair is not included in the training set. In the past years, the conventional classifier-based methods, which recognize unseen attribute-object pairs by composing separately-trained attribute classifiers and object classifiers, are strongly frustrated. Different from conventional methods, we propose a generative model with a visual pathway and a linguistic pathway. In each pathway, the attractor network is involved to learn the intrinsic feature representation to explore the inner relationship between the attribute and the object. With the learned features in both pathways, the unseen attribute-object pair is recognized by finding out the pair whose linguistic feature closely matches the visual feature of the given image. On two public datasets, our model achieves impressive experiment results, notably outperforming the state-of-the-art methods.
The paper attempts to tackle unseen object-attribute recognition in still images. This follows a line of works that addresses such a problem using embedding (with operators), classification and generative approaches. The paper makes use of the visual attractor network, extracting different visual representations for objects and attributes. A collection of losses (some regularisation and some distance in embedding space) are proposed. I believe the paper (except A below) is technically correct.
This paper was particularly tricky to read. While convincing at parts, there are two worrying statements. One I currently believe (pending the authors' response) is a scientific flaw (A), the other causes the results to be potentially incomparable to published works (B). I detail these next.
(A) Scientific Flaw: Quoting from the paper, at inference (i.e. for a test image), "we propose a voting inference method. For an input image, its visual attractor feature AV is computed, ... By computing the L2 distance ... the pair that corresponds to the minimum distance is taken as the initial attribute-object pair recognition for the input image. For the images belonging to the same pair, we use the recognitions for all images to vote a pair label to be the final attribute-object pair recognition."
In my interpretation, the authors are taking all test images that are labeled using the same pair (e.g. 'sliced-apple' - assuming this is an unseen pair). They then use all these to make a final decision for all these images. 
I believe my interpretation is correct because their statement "images belonging to the same pair" can only exist in testing for their target scenario (unseen object-attribute pair).
If my interpretation is correct, this is A MAJOR FLAW in the approach. How do the authors know that a collection of test images "belong to the same pair"?? This knowledge can only be acquired from labels of the test set and cannot be used as part of any algorithm.
I would very much like for my interpretation to be naive, but I could not find a different explanation for this statement in page 6. The authors note in the conclusion that this voting approach is part of their contribution.
(B) In reporting the results, the authors state that: "Using the grid-search method, we set parameters \alpha; ... [5 parameters]  in Eq.17 to be 1:0; 2:0; 6:0; 3:0; 2:0 for the MIT-States dataset and 1:0; 3:0; 6:0; 1:0; 2:0 for the UTZappos50K
dataset. Attractor parameters cv; cl; Tv; Tl are set as 0:1; 0:2; 20; 10 for the MIT-States dataset and 0:08; 0:16; 25; 15 for the UT-Zappos50K dataset."
This assumes that the authors are grid-searching all parameters, and evaluating the test set performance for each case, then reporting the maximum possible test performance per dataset. The authors do not reference a validation set they use to set these parameters instead. Importantly, the values are SIGNIFICANTLY different for each test set and there is no explanation of why the parameters vary largely. It is not clear how the performance on an new dataset would be, 
Up to my knowledge, the other methods that they compare to in Table 1, have not reported a grid-search over the parameter space for maximum performance of the test set. This IMO makes the comparative evaluation, which the authors deem to be 'impressive' quite unfair. It is not possible to assess the method's performance using these results.