Thanks for the details @smth  More PyTorch goodness for everyone 
The original PyTorch 1.0 blog post was not very clear on these details, and I think it is important for users to understand general PyTorch directions, especially when TensorFlow develops eager mode, and heterogenous compilation technologies start off. Btw, any plans for TensorComprehensions / Halide tighter integration? (https://people.csail.mit.edu/tzumao/gradient_halide/gradient_halide.pdf 19)