Solved by ptrblck in post #2 
              
              
                Yes, all the mentioned operations will be tracked by Autocrat such that the gradient will be passed to the corresponding values during the backward pass.