LEARNING PROTEIN STRUCTURE WITH A DIFFERENTIABLE SIMULATOR
The Boltzmann distribution is a natural model for many systems, from brains to materials and biomolecules, but is often of limited utility for fitting data because Monte Carlo algorithms are unable to simulate it in available time. This gap between the expressive capabilities and sampling practicalities of energy-based models is exemplified by the protein folding problem, since energy landscapes underlie contemporary knowledge of protein biophysics but computer simulations are challenged to fold all but the smallest proteins from first principles. In this work we aim to bridge the gap between the expressive capacity of energy functions and the practical capabilities of their simulators by using an unrolled Monte Carlo simulation as a model for data. We compose a neural energy function with a novel and efficient simulator based on Langevin dynamics to build an end-toend-differentiable model of atomic protein structure given amino acid sequence information. We introduce techniques for stabilizing backpropagation under long roll-outs and demonstrate the model's capacity to make multimodal predictions and to, in some cases, generalize to unobserved protein fold types when trained on a large corpus of protein structures.
The paper proposes a new end-to-end training framework for computational prediction of protein structure from sequence. 
This is a very important problem and any progress due to new data and/or methods for utilizing may have high impact. 
The paper presents several technical contributions in the modelling and training procedure - for example, automatic transformation between Cartesian and angular coordinates, using Langevin dynamics, and imputation method to get fine atomic coordinates. 
The overall breadth and depth of the methods presented in the paper are impressive. The paper describes a quite complicated systems with multiple modules interacting between them. The paper doesn't describe the system built in enough details, although many of the details are given in the appendix.  
Figure. 6 presents a scheme of the entire system, but it lacks details about the different modules, and it is not clear how they interact and how their training together is performed. 
The pseudo-code boxes describing Algorithms 1-4, and Table 2 describing the representation are informative and helpful, and more descriptions of this type would help. 
For example:    - In Algorithm 3, what do 'CartesianStep' and 'ClippedInternalStep' mean? where are they described? (should have their own boxes/description).  
		- I didn't see an Algorithm describing the atomic imputation part. 
		- It would be good to add a high-level pseudo-code for the entire end-to-end training algorithm. In it there could be calls to Algorithms 1-4 when needed.  
There is also no single place where all the parameters used by the authors to achieve their empirical results are presented 
(e.g. learning rates, Gaussian kernel widths, how are random time steps for enforcing Lipschitz condition chosen etc.). 
In addition, the empirical loss defined in eq. (8) is a sum of 6 different losses. It is not clear how are these very different losses scaled to the same 'units', which ones are more important, 
if and how are constants multiplying them chosen to give lower/higher weights to some of the losses etc. - I guess these choices will have a large effect on the training. 
The authors present generalization results of their trained model in predicting 3D structures from CATH at different generalization level
(i.e. different similarity levels to the training set proteins). It is not clear to me how good are these results, except that they are shown 
to be better than a baseline simple model. How well does the author's model compare to other recently suggested end-to-end models? 
(the authors mention AlQuraishi, Anand&Huang, papers). How do they compare to state-of-the art structure prediction programs? (e.g. CASP winners)? 
I realize giving an automatic end-to-end solution is interesting even if performance is below that of best programs, but still it would be good to know gaps.
If such comparisons are less meaningful/not practical to perform this should be argued convincingly. 
It would also be useful to add some metrics of running time - it is not clear how computationally heavy and scalable is the author's model and training, compared to other methods. 
There are many typos and inconsistent notations which makes it harder for the reader to understand the paper. 
For example, 'Figure ??' in multiple locations, wrong Figure referenced, using s vs. S for sequence - S is defined as an L*20 matrix but in the appendix there are
3 indices: s_{i,l,j} and it looks like different sequences in alignment should be denoted s_i. 
Equation for M_{l,j} isn't clear: j is used both as fixed index and index in summation. 
The indexing in 'orientation vectors' v-hat_ij definition seems off (the formula of base vectors gives 0/0)