Pytorch works with a recursive calling.
A nn.Module  instance (aka, neural network/layer) is usually composed by other nn.Module instances.
once you have defined the main one (let’s call it model) you can move all the instances and subinstances (layers) inside that by doing model=model.cuda()
nn.Modules are composed of 2 main parts.
The init method in which you define the layers (which are also nn.Modules as I aforementioned) and the forward, which defines how those layers are used.
Following good practices, code shouldn’t hardcode allocation (whether cuda or cpu) in neither init or forward.
That means that, once you have a model, you can choose whether to process it using the cpu or cuda.
For the former case you have to do nothing (as tensors are defined as cpu tensors by default). For the latter you just need to allocate the model and inputs in cuda:
Pseudocode
model = model.cuda()
for batch in dataloader:
   output = model(batch.cuda())