Linear Decision Rule as Aspiration for Simple Decision Heuristics
Several attempts to understand the success of simple decision heuristics have examined heuristics as an approximation to a linear decision rule. This research has identified three environmental structures that aid heuristics: dominance, cumulative dominance, and noncompensatoriness. This paper develops these ideas further and examines their empirical relevance in 51 natural environments. The results show that all three structures are prevalent, making it possible for simple rules to reach, and occasionally exceed, the accuracy of the linear decision rule, using less information and less computation.
This work turns NIPS on its head: In NIPS, linear 
      threshold classifiers are usually considered to be a baseline model, 
      but in this work, linear threshold classifiers are considered too 
      sophisticated to characterize human decision making, and various 
      simpler alternatives are explored. The work appears to make two 
      important contributions: (1) to show that for natural environments, 
      various decision-making short cuts perform well, and (2) to introduce 
      novel decision short cuts that are probabilistic generalizations of 
      previously proposed short cuts. The paper is very well written 
      and accessible. It should have more discussion of the implications of 
      the results. The authors should also make their theoretical 
      contribution clearer. Is approximate (cumulative) dominance is new to 
      this work? The motivation for the approximation might be made clearer 
      as well. I believe the reason is that the conditions in section 2 
      often do not apply, but an approximate rule would be more often 
      applicable, and as long as it is fairly accurate, it should boost 
      performance. This appears to be the case in the results. One 
      additional point about the exposition: The introduction fails to give all 
      the background necessary to motivate the work to a typical NIPS attendee. 
      Before discussing fast-and-frugal heuristics, the authors should discuss 
      the empirical literature showing that simple linear-threshold classifiers 
      are better than human decision makers (e.g., 
      http://www.sciencemag.org/content/243/4899/1668). If you start from this 
      perspective, it motivates the treatment of the linear-threshold 
      classifier as the target, and the question of what heuristics 
      approximate this target. Without this background for the reader, I suspect 
      the typical NIPS attendee will wonder the most sophisticated model of 
      decision making is a linear-threshold classifier. The work claims 
      to be a serious analysis of structure of natural environments, but I'm 
      far from clear whether the 51 real-world data sets involve the type of 
      decision making tasks that humans might typically make. It might be 
      worth moving Table 1 to the supplementary materials (along with the 
      page of references), and instead to present an argument for why these 
      data sets are representative of human decision making tasks. I 
      was hoping the authors would compare their 51 natural data sets to 
      randomly generated data sets, where the weights and features are drawn 
      from specified distributions. It would also be interesting to 
      determine the distributional assumptions that lead to success of the 
      various decision making short cuts. I am confused about the trade 
      off in setting the cutoff value for the approximate dominance schemes: 
      A high cutoff means that the approximate-dominance relationship 
      predicts the correct decision, but also that fewer 
      approximate-dominance relationships will hold between pairs. The trade 
      off is not discussed. Was the chosen cutoff (.99) arbitrary, or are the 
      results sensitive to this choice?