Moving model and data between cpu and gpu with .to(device) or .cpu() or .cuda() could make the code messy if not done properly.
Are there any recommend guideline/standard practice to go about it?
What about moving data to gpu in the forward(self, ...) function of a network and returning results after moving them back to cpu?
something along the lines of
    def forward(self ,x):
        x = x.cuda()
        x = self.fc(x)
        x = x.cpu()
        return x

Is this a good idea or are there reasons to not go in this direction?
Thanks.