Efficient inference for time-varying behavior during learning
The process of learning new behaviors over time is a problem of great interest in both neuroscience and artificial intelligence. However, most standard analyses of animal training data either treat behavior as fixed or track only coarse performance statistics (e.g., accuracy, bias), providing limited insight into the evolution of the policies governing behavior. To overcome these limitations, we propose a dynamic psychophysical model that efficiently tracks trial-to-trial changes in behavior over the course of training. Our model consists of a dynamic logistic regression model, parametrized by a set of time-varying weights that express dependence on sensory stimuli as well as task-irrelevant covariates, such as stimulus, choice, and answer history. Our implementation scales to large behavioral datasets, allowing us to infer 500K parameters (e.g., 10 weights over 50K trials) in minutes on a desktop computer. We optimize hyperparameters governing how rapidly each weight evolves over time using the decoupled Laplace approximation, an efficient method for maximizing marginal likelihood in non-conjugate models. To illustrate performance, we apply our method to psychophysical data from both rats and human subjects learning a delayed sensory discrimination task. The model successfully tracks the psychophysical weights of rats over the course of training, capturing day-to-day and trial-to-trial fluctuations that underlie changes in performance, choice bias, and dependencies on task history. Finally, we investigate why rats frequently make mistakes on easy trials, and suggest that apparent lapses can be explained by sub-optimal weighting of known task covariates.
The paper presents a new estimator for a dynamic logistic regression model used to characterize human or animal behavior in the context of binary decisions tasks. Despite the very large number of parameters such a model can be estimated robustly and with tolerable computational cost by exploiting the structure of the posterior covariance. The estimator is then applied to a delayed auditory  discrimination task in humans and animals. The results show signatures of learning and history interference in the rodent compared to human. Additionally the model fit manages to predict behavior in test data. The model is simple and interpretable which will likely make the use of the tools popular within the community.  Comments: - the choice of prior for w is a bit odd - since the variance of the marginal for w_t grows linearly with t; Also the fact that the noise is independent seems somewhat limiting - how about things like fluctuation in attention those would likely affect the processing of both stimuli and potentially even some of the history... - it would be nice if you would directly compare the dynamics of w with the dynamics of learning, as measured behaviorally; the prediction estimates in fig 3 goes into that direction but it would be interesting to see a direct visualization of the two.  - similarly do the fluctuations in ws predict fluctuations in performance in humans? - it wasn't clear if the code associated will be released with the paper  Minor: - unspecified w_0: I am assuming w_0 = 0  L should be defined somewhere explicitly in eq 3 - unclear what 'behavioral realization' is supposed to mean in relation to figure 2: are individual lines MAP estimates for w, where the true w is kept fixed but the observation noise is redrawn independently? And are the hyperparameters estimated in panel b the same way or are the ws redrawn for each experiment? -at some point the text mentions sparse priors for w which could be misleading - it's just the prior covariance matrix is sparse.  Post rebuttal: Thanks for the clarifications. I still the prior should be justified by more than mathematical convenience but it's ok as long as the text justifies it accordingly.