You are explicitly calling .detach() on the output of the model. So you explicitly break the history as no gradient will be computed beyond the detach operation. You will need to remove it (and remove the use of numpy Tensors as we cannot differentiate through these operations, you need to use only Tensors).