So this can actually be a bit strange. I think your method will work better since you need to train two networks at once (presumably)
When you call the loss function, the first Variable can require gradient, the second cannot. The second argument also doesn’t have to be volatile. They can both be Variables, you just have to mark the second as not requiring gradient. For example, this code (should) work.
self.criterion = nn.L1Loss()
self.delta = Variable(torch.FloatTensor([0]), requires_grad=True)
next_layer_grad = Variable(torch.FloatTensor([0]))
loss = self.criterion(self.delta, next_layer_grad)

If you wanted to do it this way you could clone one output and mark it as not requiring gradient, take L1Loss, then swap inputs and do it again.