I really appreciate your answer, but the next problem is how I can know the function is run in GPU or CPU, is there any flag indicates it? I tried to pass a bool argument from outside of the function but another error raises.

RuntimeError: expected a Variable argument, but got bool

Is it because of the torch.autograd.Function accepts Variable only? if so, how can I determine the function is run in GPU or not?
Thank you!