Solved by ptrblck in post #2 
              
              
                All nn.Parameters and buffers (registered via self.register_buffer) will automatically be pushed to the specified device when calling model.to(device). The difference would be that the former are trainable (requires_grad is set to True by default) while the latter do not. In case you don’t want to s…