A Probabilistic Model of Social Decision Making based on Reward Maximization
A fundamental problem in cognitive neuroscience is how humans make decisions, act, and behave in relation to other humans. Here we adopt the hypothesis that when we are in an interactive social setting, our brains perform Bayesian inference of the intentions and cooperativeness of others using probabilistic representations. We employ the framework of partially observable Markov decision processes (POMDPs) to model human decision making in a social context, focusing specifically on the volunteer's dilemma in a version of the classic Public Goods Game. We show that the POMDP model explains both the behavior of subjects as well as neural activity recorded using fMRI during the game. The decisions of subjects can be modeled across all trials using two interpretable parameters. Furthermore, the expected reward predicted by the model for each subject was correlated with the activation of brain areas related to reward expectation in social interactions. Our results suggest a probabilistic basis for human social decision making within the framework of expected reward maximization.
In this paper authors propose the POMDP model to simulate human decision making in the Public Goods Game (PGG). They explain the machinery of the model, its fit to behavioral PGG data and comparison to other similar models. They also suggest that the model has neural correlates with reward expectations correlated with activity in two brain areas. 
This is an interesting modeling and model-based analysis study, providing insights into the machinery of human social decision making and possibly its neural correlates. The paper is generally well written and combines advances that could be interesting to both experimental and modeling audience. However, some of its aspects (particularly interpretation of estimated parameters and fMRI analyses) should be improved for it to be acceptable to NIPS.  More specific comments: - What is the meaning of groups of 5 individuals if computer generates actions of 19 others? More details about how these actions are generated would be helpful. - Biological plausibility of the used value iteration algorithm should be discussed. - Equation 1 is hard to understand: what is the meaning of the second E on each line? - Why is it assumed that all group members have the same theta_c as the subject? - Regarding table 1: the models should be better explained. What are the free parameters/their number for each model? That should normally be included in model comparison. Which differences are statistically significant? - fMRI small volume correction (SVC) based on 10mm (radius or diameter?) spheres should be justified based on literature. How is the region center chosen for this purpose?  - There is a typo in the first T value - it should be 3.45 I suppose - T values are too small to be called "strong correlation". Generally, whole brain analyses are preferable to SVC. - At the beginning of 5.3 do you mean "the best FITTING alpha_0 and beta_0"? - Do alpha and beta themselves have no neural correlates? Either way this should be reported. Alpha and beta have nice mathematical grounding but their suitability for psychological interpretation is less clear. It seems ratios or sums could be more informative.  --- Response to author rebuttal: Thanks for addressing some of my concerns. However, I would like to clarify a few aspects which should be addressed in the final version. * While the use of SVC is understandable, it may lead to missing important effects in brain areas outside of the defined ones, whose relevance could also be justified and whose peak T-values may even be higher than of the reported ones. For example, many SVC-based studies of memory focus on hippocampus and exclude parahippocampal gyrus or prefrontal cortex, which were shown to have pronounced activity patterns in whole-brain studies. For this reason please provide whole-brain statistics as well (if necessary, in supplementary material), including peaks outside of the defined area with larger T values than reported. * If ratios and sums of alpha and beta are more informative, why not correlate them with neural data or behavior instead of simply mentioning this aspect in the discussion? If there are no effects, this should be reported as well.