Pytorch refuses to release the computation graph no matter what I try. I am trying to incorporate the loss of the gradient with respect to the input, but this is seemingly unprecedented in the Pytorch community, has it never been done before?!
Relevant code:
    def train(self, train_loader, epoch, epochs):
        self.net.train()
        train_loss = 0.0
        for i, data in enumerate(train_loader):
            features, labels, label_derivatives = data

            features.requires_grad = True
            outputs = self.net(features)
            outputs.backward(features, create_graph=True)
            derivatives = features.grad

            self.optimizer.zero_grad()
            loss1 = (outputs - labels).pow(2).mean()
            loss2 = (derivatives - label_derivatives).pow(2).mean()
            loss = loss1 + loss2
            loss.backward()
            self.optimizer.step()
            train_loss += float(loss.item())

        return train_loss / len(train_loader)