I think it is because nn.Linear() creates a new layer on the fly, and it is not placed on GPU. Try to create it once in the init method, so it will be moved on GPU with the entire net. If you cant do it in init, so you can move the new layer on the GPU every time:
new_layer = nn.Linear(out.size(1), 5)
new_layer = new_layer.to(self.device)
out = new_layer(out)
something like that