Hi everyone! I have a quick question regarding the tutorial posted on pytorch for computer vision training, specifically at this link: Transfer Learning for Computer Vision Tutorial — PyTorch Tutorials 1.10.0+cu102 documentation. In the train model function, I don’t exactly understand why the output is not detached during training (see snippet below). Wouldn’t outputs from the model still be attached to the computational graph when to get preds below? Another question with the code is I don’t believe the function moves the output to cpu during the validation phase when repeating the calculation to obtain predictions - this is something I’ve typically seen done. Is that not standard practice? Am I missing something here or is that all correct? Thanks for the help!
with torch.set_grad_enabled(phase == ‘train’):
outputs = model(inputs)
_, preds = torch.max(outputs, 1)
loss = criterion(outputs, labels)
                # backward + optimize only if in training phase
                if phase == 'train':
                    loss.backward()
                    optimizer.step()