You can do that by defining a custom autograd.Function 1. This allows you to do anything you want in the forward. The caveat is that you have to give the derivative yourself in the backward (as PyTorch cannot be aware of it when you are not using tensors).
Best regards
Thomas