Point process latent variable models of larval zebrafish behavior
A fundamental goal of systems neuroscience is to understand how neural activity gives rise to natural behavior. In order to achieve this goal, we must first build comprehensive models that offer quantitative descriptions of behavior. We develop a new class of probabilistic models to tackle this challenge in the study of larval zebrafish, an important model organism for neuroscience. Larval zebrafish locomote via sequences of punctate swim bouts-brief flicks of the tail-which are naturally modeled as a marked point process. However, these sequences of swim bouts belie a set of discrete and continuous internal states, latent variables that are not captured by standard point process models. We incorporate these variables as latent marks of a point process and explore various models for their dynamics. To infer the latent variables and fit the parameters of this model, we develop an amortized variational inference algorithm that targets the collapsed posterior distribution, analytically marginalizing out the discrete latent variables. With a dataset of over 120,000 swim bouts, we show that our models reveal interpretable discrete classes of swim bouts and continuous internal states like hunger that modulate their dynamics. These models are a major step toward understanding the natural behavioral program of the larval zebrafish and, ultimately, its neural underpinnings.
The authors propose a marked process latent variable model that leverages Gaussian processes for continuous latent sates, a generalized linear model for discrete latent states and an inference network for efficient inference. Results on real data suggest that the proposed approach is interpretable and outperforms standard baselines on held out data.  I really enjoyed reading the paper, it is very well written. That being said, the notation is sloppy and lack of details makes it difficult to appreciate the contributions of the paper. The authors seemed to have assumed that the reader is very familiar with point processes, Gaussian processes, variational inference and deep learning. To name a few, no distinction between the covariance function K() and the covariance matrix K is made; the forms of the parameter functions for Gamma and Gaussian distributions in the generative process below line 156 are never introduced or defined (closed-form conditional posteriors are available or are they specified as neural networks?); the form of the generalized linear model \pi_\theta() is never made explicit; no details of the RNN are provided; it is not clear whether the inducing points are estimated, set on a grid or why 50 or 25 inducing points are enough; in Table 2, no details about Poisson, GRP and AR apre provided. How were the many parameters of all models selected?  In a paper like this with so many details missing, a supplementary file with formulation, inference, model architecture details and experimental setup details is a must, however, none was provided. Besides, lack of space seems to be not a problem provided that Figure 2 is not terribly informative, and Figures 3-5 could be made a lot smaller without sacrificing readability.  Minor: - eq. (1), N_m not defined, typo? - line 137, GP mean and covariance functions not defined. - line 156, undefined quantities, \mu_\theta(), \Sigma_\theta(), a_\theta(), b_\theta(), m_n|1:n-1, K_n|1:n-1 - line 252, locomotion - line 259, remove finds or reveals  In their rebuttal, the authors have provided enough detail about the approach and experiments. I am happy to recommend acceptance (score changed from 3 to 7).