A value relativistic decision theory predicts known biases in human preferences
A value relativistic decision theory predicts known biases in human preferences Nisheeth Srivastava (nsriva@cs.umn.edu) Department of Computer Science, 200 Union Street SE Minneapolis, MN 55455 USA Paul R Schrater (schrater@umn.edu) Department of Psychology, 75 East River Road Minneapolis, MN 55455 USA Abstract Traditional models of decision making assume the existence of an external frame of reference for measuring the value of possible outcomes. We show that this fundamental assump tion prevents classical decision models from predicting realis tic decision making behavior. Making an alternative relativis tic assumption about the nature of reward leads us to formalize a rational agent as one which minimizes its internal decision computational costs while retaining satisfactorily predictive models of its external environment. In computational evalu ation, our model replicates previously unexplained 'irrational' behavior of human subjects. Keywords: Cognitive models; neural coding; decision theory; behavioral biases Introduction Economists have used the terms 'animal spirits' and 'cogni tive biases' (Akerlof Shiller, 2009) to explain the persis tence of behaviors incompatible with accepted definitions of rationality. Consistent observation of such 'predictably irra tional' (Ariely, 2009) behavior in both controlled and uncon trolled settings has reduced confidence in the neo classical view of human decision making as a rational enterprise. Contemporary explanations for these biases typically draw on evolutionary arguments (Gigerenzer Goldstein, 1996) especially the idea of that deviations from traditional ratio nality represent specialized adaptations to the social and en vironmental conditions of mankind's early history. Thus, while on one hand, rational models of decision making lie discredited through their inability to explain the existence of cognitive biases, the prominent alternative approaches create heuristic based theories that are limited in their ability to pro duce generalizable causal explanations for decision making processes. The absence of a realistic, principled theory of hu man decision making is deeply problematic, since models of decision making are central to the formulation of immensely consequential social and economic policies. Given the continued failure of variants of existing theories to offer compelling explanations for how biological agents make decisions, we believe that it is necessary to re evaluate the foundational assumptions in these theories and find plau sible alternatives. How does an agent represent its possi ble options in an environment? How does it assign value to various options? On what basis does it select between these options? The canonical responses to these questions, obtained through a history of research stretching from J S Mill (Mill, 1874) to von Neumann (Neumann Morgenstern, 1953) are that option possibilities are represented as environ mental states, value is assigned to these options in the form of numerical reward, and the goal of the agent is assumed to be the maximization of its long term cumulative reward. These assumptions are foundational to both homo economi cus (Persky, 1995) models of economic choice and reinforce ment learning (Barto Sutton, 1998) In this paper, we ques tion each of these three dogmas and replace them with alter natives obtained from a relativistic view of both how agents evaluate possible outcomes and how they evaluate their own existence as predictive agents. At a fundamental level, the assignment of reward values as sumes the existence of some fixed reference against which the value of its actions can be measured. This simple assumption causes us to formulate rewards as absolute numeric quanti ties, with a "no reward" state as the origin. Furthermore, this view renders us unable to consider any meaningful goal for an agent other than maximal reward accumulation. That is, if we detach the notion of value from the agent's environment specific need at the moment of the decision and instead fix it with respect to some Platonic origin, we can no longer mean ingfully speak of satisfying needs, only of optimizing utility. In this paper, we try to capture the notion of value as the agent's needs by making the valuation process relative to both the agent's current policy and current set of options. This changes the canonical decision modeling approach in three ways: one, it captures the embodied view of cognition; two, we discard the association of numerical rewards to particu lar outcomes all rewards are relative comparisons of value; three, decision making is seen to be a process of identify ing outcomes from the agent's current situation and evaluat ing those options relative to the agents understanding of what has constituted the best policy for this situation, based on the agent's past experience. By formalizing these insights, we show that the assump tion of value relativity, while exotic at first sight, can be eas ily adapted into a tractable choice learning algorithm. We further show that the choice predictions of this algorithm pre dict the choices of human subjects across different decision tasks that have heretofore been considered irrational. The unforced emergence of a number of previously unconnected cognitive biases from our decision model provides empirical support for its foundational premises. We conclude with a brief discussion of some implications of our findings.
CogSci