We propose order learning to determine the order graph of classes, representing ranks or priorities, and classify an object instance into one of the classes. To this end, we design a pairwise comparator to categorize the relationship between two instances into one of three cases: one instance is 'greater than,' 'similar to,' or 'smaller than' the other. Then, by comparing an input instance with reference instances and maximizing the consistency among the comparison results, the class of the input can be estimated reliably. We apply order learning to develop a facial age estimator, which provides the state-of-the-art performance. Moreover, the performance is further improved when the order graph is divided into disjoint chains using gender and ethnic group information or even in an unsupervised manner.
This paper presents an order learning method and applies it to age estimation from facial images. It designs a pairwise comparator that categorizes ordering relationship between two instances into ternary classes of greater than, similar, and smaller than. Instead of directly estimating the class of each instance, it learns pairwise ordering relationship between two instances.
For age estimation from facial images, it uses a Siamese network as a feature extractor of the pairwise comparator, concatenates feature maps of dual inputs (test instance and one of the multiple reference instances that are selected from the training data) and applies it to a ternary classifier. Given the softmax probability scores of the classifier results, it estimates the final class as the one that maximizes a consistency rule computed over indicator functions. The comparator is trained to minimize a comparator loss computed over the softmax probability scores.
The paper also provides an extended version for multiple disjoint chains, where each chain may correspond to a higher-level attribute class, for example, gender or ethnic group. When there is no supervision available, it randomly partitions the training set.  and iteratively updates reliability scores and chain memberships. 
Novelty-wise, I consider that the proposed solution to be satisfactorily innovative and on a different vein than the existing methods. 
One concern about the method is that it imposes the geometric ratio (log distance) between the class distances in age estimation, considering the difference between 5 and 10-year-old instances is easier to detect than 65 to 70-year-old instances as stated in the paper. However, as far as it is understood from the provided discussion compared SOTA methods are not retrained or fine-tuned in the same manner. This raises the question of whether the slight performance improvement is a result of the geometric ratio, in particular when computing the cumulative score CS.   
Another concern is that the performance is not stellar as the presented method underperforms in comparison to DRF (Shen et al. 2018) on MORPH II. The presentation of the results for FG-Net is neither complete nor included in the main paper, and results for CLAP2016 are missing. Discussion of the results Table 8 does not seem fair as it is not clear whether MV is retrained keeping the geometric ratio in mind. It might be the case that even for 15-69, MV might attain lover MAEs. 
The paper also misses the latest SOTA, for instance, BridgeNet [1] is mentioned in a sentence, yet its results are omitted from the Tables. BridgeNet, according to its results, is on par with the proposed method. Also, the numbers reported in this paper and the numbers in [1] have a discrepancy, which causes confusion.
[1] Wanhua Li et al. Bridgenet: A continuity aware probabilistic network for age estimation, CVPR 2019.