I think what you have is a neural network layer, which I believe is CPU/GPU agnostic.
The inputs/outputs are Variables, which can either be on the CPU or GPU. You can check that with the following:
inputs = Variable(torch.randn(2,2))
inputs.is_cuda  # will return false
inputs = Variable(torch.randn(2,2).cuda())
inputs.is_cuda # returns true