Rationalizability is a solution concept in game theory. The general idea is to provide the weakest constraints on players while still requiring that players are rational and this rationality is common knowledge among the players. It is more permissive than Nash equilibrium. Both require that players respond optimally to some belief about their opponents' actions, but Nash equilibrium requires that these beliefs be correct while rationalizability does not.  Rationalizability was first defined, independently, by Bernheim (1984) and Pearce (1984).

Definition 

Given a normal-form game, the rationalizable set of actions can be computed as follows: Start with the full action set for each player. Next, remove all actions which are never a best reply to any belief about the opponents' actions -- the motivation for this step is that no rational player could choose such actions. Next, remove all actions which are never a best reply to any belief about the opponents' remaining actions -- this second step is justified because each player knows that the other players are rational. Continue the process until no further actions are eliminated. In a game with finitely many actions, this process always terminates and leaves a non-empty set of actions for each player. These are the rationalizable actions.

Constraints on beliefs 

Consider a simple coordination game (the payoff matrix is to the right).  The row player can play a if he can reasonably believe that the column player could play A, since a is a best response to A.  He can reasonably believe that the column player can play A if it is reasonable for the column player to believe that the row player could play a.  She can believe that he will play a if it is reasonable for her to believe that he could play a, etc.

This provides an infinite chain of consistent beliefs that result in the players playing (a, A).  This makes (a, A) a rationalizable pair of actions.  A similar process can be repeated for (b, B).  

As an example where not all strategies are rationalizable, consider a prisoner's dilemma pictured to the left.  Row player would never play c, since c is not a best response to any strategy by the column player.  For this reason, c is not rationalizable.   

Conversely, for two-player games, the set of all rationalizable strategies can be found by iterated elimination of strictly dominated strategies. For this method to hold however, one also needs to consider strict domination by mixed strategies. Consider the game on the right with payoffs of the column player omitted for simplicity. Notice that "b" is not strictly dominated by either "t" or "m" in the pure strategy sense, but it is still dominated by a strategy that would mix "t" and "m" with probability of each equal to 1/2. This is due to the fact that given any belief about the action of the column player, the mixed strategy will always yield higher expected payoff. This implies that "b" is not rationalizable.

Moreover, "b" is not a best response to either "L" or "R" or any mix of the two. This is because an action that is not rationalizable can never be a best response to any opponent's strategy (pure or mixed). This would imply another version of the previous method of finding rationalizable strategies as those that survive the iterated elimination of strategies that are never a best response (in pure or mixed sense).

In games with more than two players, however, there may be strategies that are not strictly dominated, but which can never be the best response.  By the iterated elimination of all such strategies one can find the rationalizable strategies for a multiplayer game.

Rationalizability and Nash equilibria 

It can be easily proved that a Nash equilibrium is a rationalizable equilibrium; however, the converse is not true.  Some rationalizable equilibria are not Nash equilibria.  This makes the rationalizability concept a generalization of Nash equilibrium concept.  

As an example, consider the game matching pennies pictured to the right.  In this game the only Nash equilibrium is row playing h and t with equal probability and column playing H and T with equal probability.  However, all pure strategies in this game are rationalizable.

Consider the following reasoning: row can play h if it is reasonable for her to believe that column will play H.  Column can play H if its reasonable for him to believe that row will play t.  Row can play t if it is reasonable for her to believe that column will play T.  Column can play T if it is reasonable for him to believe that row will play h (beginning the cycle again).  This provides an infinite set of consistent beliefs that results in row playing h.  A similar argument can be given for row playing t, and for column playing either H or T.

See also
 Self-confirming equilibrium

Footnotes

References 

Bernheim, D. (1984)  Rationalizable Strategic Behavior.  Econometrica 52: 1007-1028.
Fudenberg, Drew and Jean Tirole (1993)  Game Theory.  Cambridge: MIT Press.  
Pearce, D. (1984)  Rationalizable Strategic Behavior and the Problem of Perfection.  Econometrica 52: 1029-1050.
Ratcliff, J. (1992–1997)  lecture notes on game theory, §2.2: "Iterated Dominance and Rationalizability"

Game theory