Forecasting Treatment Responses Over Time Using Recurrent Marginal Structural Networks
Electronic health records provide a rich source of data for machine learning methods to learn dynamic treatment responses over time. However, any direct estimation is hampered by the presence of time-dependent confounding, where actions taken are dependent on time-varying variables related to the outcome of interest. Drawing inspiration from marginal structural models, a class of methods in epidemiology which use propensity weighting to adjust for time-dependent confounders, we introduce the Recurrent Marginal Structural Network -a sequence-to-sequence architecture for forecasting a patient's expected response to a series of planned treatments. Using simulations of a state-of-the-art pharmacokinetic-pharmacodynamic (PK-PD) model of tumor growth [12], we demonstrate the ability of our network to accurately learn unbiased treatment responses from observational data -even under changes in the policy of treatment assignments -and performance gains over benchmarks.
Post-rebuttal comment: In their response, the authors answered my primary question about breaking apart their model (which consists of two RNN components) to identify more specifically where the gains of this model come from. They also explained how Figure 4 contains similar comparisons, and clarified my question about the gamma = 0 setting. I think the paper would benefit from a little more clarity around Figure 4 to minimise the confusion I experienced, but otherwise I remain confident in my assessment of the quality of the paper. I note also that their response to me also addressed one of the concerns of Reviewer 3.  Summary of work: This work is focused on estimating treatment responses through time in the presence of time-dependent confounders using marginal structural networks. Marginal structural networks have been used in epidemiology and other statistical fields, and exploit inverse probability of treatment weighting (IPTW) to adjust for bias. They require two components - the covariate-dependent probability of treatment assignment (propensity) and the treatment-dependent response. In this work, they use RNNs (LSTMs) to approximate each of these components. They use a commonly-used synthetic dataset to develop and test their model, comparing against standard marginal structural networks and a Bayesian approach.  Explanation of overall score: This is a strong submission overall - the experiments are reasonably convincing, the architecture and approach is well explained, and the problem seems important. I also think the idea of bringing approaches used elsewhere (e.g. epidemiology) for causal inference to a NIPS audience, and demonstrating how neural networks can be beneficial, is a worthy endeavour.  Explanation of confidence score: I am not very familiar with one of the primary comparisons in this work (the BTRC model) and therefore can’t guarantee that the comparison was fair.  Quality:  * The analyses include two important experiments: varying the level of time-dependent confounding (specifically, how treatment assignment depends on the patient’s current state, which is the tumour diameter in these experiments), and varying the prediction horizon (for different levels of confounding). * In the first experiment they also compare their proposed approach with a hybrid where the IPTWs are derived from the “standard” MSM model. Further along this vein, I would be interested to know how well a “vanilla” RNN would perform at this task. That is, if the task in Figure 4 is simply to predict the treatment response at the next time-point, how well can you do if you neglect the MSM modelling entirely, and treat it as a standard regression problem? I think, for readers not familiar with MSMs, this comparison might help to motivate why the MSM framework is necessary. * Equivalently, it would be interesting to see how well a MSM performs if the IPTWs are estimated using RNNs, but the treatment response itself is given by the standard regression model. In essence, this work proposes an upgrade of MSMs with two new parts, and demonstrates that this works better. But then the question is where did the greatest performance gain come from? * I am slightly confused by the performance in Figure 4 when there is no time-dependent confounding. In this case, shouldn’t all IPTWs (if estimated correctly) be the same? If this is true, then shouldn’t the performance of the R-MSN and the Seq2Seq/Logistic baselines be the same? My intuition in this case is that the RMSE should be driven by the baseline difficulty to predict if a treatment will be assigned (since, according to (9), gamma = 0 indicates random assignments), on top of the error of the treatment-dependent response prediction. If the latter part uses the same model, then how does the RMSE differ? * To thoroughly demonstrate the power of the RNNs to adapt to different types of response models, I would have liked to see experiments on a different dataset.  Clarity:  * The submission is well-organised and fairly clearly written. My biggest complaint would be that there is relatively less detail about how the final model specifically looks - it would perhaps be sufficient to refer back to equation (4) after the R-MSN has been defined (around equation (5)), to make it clear how the RNNs are fitting in to the MSM model. * There are some small errors in the text of 4.3 - in Step 2, it says Figure 3(c) instead of Figure 3(b), and later says “decoder” when it should probably say “encoder”. (I only point these out because these descriptions are quite important!). * As far as I know, there is no limit on the length of the references for the main paper, so I would recommend to move all references appearing in the main text to the main bibliography, it’s a little confusing otherwise. * The appendix contains a good amount of detail about training and hyperparameter optimisation, making this paper likely easy to reproduce 🙂 (I hope the authors will also provide code when feasible)   Originality:   * To the best of my knowledge the idea to parametrise elements of a MSM using RNNs is original. * The appendix contains an extended related work section, helpfully placing the work in context.  Significance:  * The problem of accounting for time-dependent confounders is important in various domains of applied machine learning. * This idea to me seems like a nice application of methods from deep learning towards more traditional statistical sciences, which may also help to publicise models such as MSMs to the NIPS community.