Threshold Learning for Optimal Decision Making
Decision making under uncertainty is commonly modelled as a process of competitive stochastic evidence accumulation to threshold (the drift-diffusion model). However, it is unknown how animals learn these decision thresholds. We examine threshold learning by constructing a reward function that averages over many trials to Wald's cost function that defines decision optimality. These rewards are highly stochastic and hence challenging to optimize, which we address in two ways: first, a simple two-factor reward-modulated learning rule derived from Williams' RE-INFORCE method for neural networks; and second, Bayesian optimization of the reward function with a Gaussian process. Bayesian optimization converges in fewer trials than REINFORCE but is slower computationally with greater variance. The REINFORCE method is also a better model of acquisition behaviour in animals and a similar learning rule has been proposed for modelling basal ganglia function.
This paper represents an interesting application of two existing learning algorithm, Williams' reinforce algorithm and Bayesian optimization, on estimating the decision threshold in 2-AFC task. The threshold is estimated by minimizing the difference between the reward sum of the optimal decision policy and the sum of collected reward. The author also compares the performance of the two algorithms and maps the SPRT decision performance over main parameter space.   This paper looks technically sound. I like the main the idea of this paper because estimating decision threshold is an important problem in decision science. 
Technical quality: This paper focuses on estimating a decision threshold - the problem which is not sufficiently studied in this community. Two existing algorithms are used to estimate the threshold and their performances are compared against each other. The paper is interesting and technically sound.  Novelty: The idea of estimating the threshold has some novel contribution to the computational psychology society. However, the algorithms used in this paper are not novel because the author uses these algorithm without much modifications based on the previous work [5],[6] & [7]. Moreover, in this paper,  other parameters need to be fixed when the optimal threshold is being estimated. Since other parameters are fixed, a naive approach can be searching the parameter space of the threshold to minimize the cost function, which can be computed by averaging trials simulated with SPRT method. The complexity of this problem is similar to other problems aiming at estimating parameters with fixed threshold.I think the main novel contribution of this paper is trying to estimate the decision threshold which has not been sufficiently studied.  Potential impact: This paper estimates a constant decision threshold in 2-AFC task with infinite horizon. I think this paper will attract attention from the researchers in psychology and cognitive science society. However, in most psychology experiment, the 2-AFC task needs to be finished within a certain deadline, thus making the threshold change over time within each trial. I would like to see a more sophisticated estimation algorithm to estimate the time-dependent threshold under finite horizon. Moreover, this paper would have more impact if the algorithm can jointly learn the threshold with other parameters.   Clarity and presentation: The paper is not very clearly written. The method section misses a lot of important information. It would be hard for some readers who are trying to replicate the work by reading through this paper. For example, in section 3.2 REINFORCE METHOD, the author should provide an illustrative network figure so that the reader can have a clearer idea about what your inputs are and how your network is trained. The information in Sec 3.3 is also vague. I understand that the author lists corresponding references including more details. However, the main framework of those algorithms should be clearly explained in this paper. I would recommend the author includes more information about the algorithms and condense the long discussion section (cut by half). Also, please check and correct a few typo, e.g.,  1.For equation (6), I think you mean "-NC_risk/c" 2.Line 114 "work work well"  --------------- After discussion, I realized that optimal threshold has already been studied by other researchers with different approaches, so I lowered the impact score as the the advantage of the methodology used in this paper is not clear when compared to the others. I also increased the clarity score after I read the paper again.