Convex Elicitation of Continuous Real Properties
A property or statistic of a distribution is said to be elicitable if it can be expressed as the minimizer of some loss function in expectation. Recent work shows that continuous real-valued properties are elicitable if and only if they are identifiable, meaning the set of distributions with the same property value can be described by linear constraints. From a practical standpoint, one may ask for which such properties do there exist convex loss functions. In this paper, in a finite-outcome setting, we show that in fact essentially every elicitable real-valued property can be elicited by a convex loss function. Our proof is constructive, and leads to convex loss functions for new properties.
Summary:  This paper studies elicitation which is defined as the property or statistic of a distribution which can be described as the minimizer of some loss function in expectation. More specifically, the focus of this paper is in studying which properties can be explained as the minimizer of convex loss functions. The authors prove that in the finite outcome setting, under some assumptions, every elicitable real-valued property can also be elicited by a convex loss function. The authors start the paper by providing an overview of the prior literature and go over their assumptions. Then, they state the theorem and give some intuition on how the proof goes through. Finally, they conclude the paper by providing two examples and applying their constructive technique to yield corresponding convex losses.  Evaluation:  The paper is nicely written and well-motivated. As authors also mention, in many practical applications, dealing with convex losses is easier than general loss functions as various techniques can be applied for finding their minimum.   Authors have addressed the limitation of their method in their discussion, however I really think that the generalization to infinite outcome space is very important. In other words, I believe that this limitation is very restrictive and needs to be better addressed. For this more general case, authors could have either provided some proofs (positive results) with maybe some additional assumptions, or they could maybe provide some counter-examples (negative results).   Furthermore, even in the case of finite outcome setting, the second example in section 5 shows that even the constructive way of their theorem (or more precisely, the constructed weight function \lambda) produces a very complex \lambda. So other than a more simpler setting of Example 1, the proof is not practically constructive. I understand that the authors were able to address this concern for Example 2 (which I think it is nice), but still this construction might be far from practical purposes in general.  Based on these points, I think the paper is in a very good shape in terms of clarity, but can be further improved in technical and also practical aspects.  AFTER REBUTTAL:  The authors have addressed all my concerns and I am willing to increase my score from 6 to 8. I believe the idea of convex elicitation is very important and this paper makes a huge contribution to the field. Hopefully, the result can be extended to the infinite outcome space in the near future.