Yes you certainly can.
You simply need to set requires_grad=True for the input vectors as such:
inputs.requires_grad_(True)
logits = model(inputs)
loss = self.criterion(logits, labels) + self.regularizer
loss.backward()
self.optimizer.step()

This will compute the gradient of the loss w.r.t. the inputs when you call loss.backward() (which will be stored in inputs.grad). You also need to add the inputs to your optimizer, otherwise, optimizer.step() will not modify them.