dmandair:

In the train model function, I don’t exactly understand why the output is not detached during training (see snippet below).


If you detach the output during training, no gradients would be calculated for the trainable model parameters and the model would thus not train at all.



 dmandair:

Wouldn’t outputs from the model still be attached to the computational graph when to get preds below?


If you are concerned about detaching the output during validation: you wouldn’t need to detach the tensor manually as the validation phase disables the gradient calculation via:
with torch.set_grad_enabled(phase == ‘train’):




 dmandair:

Another question with the code is I don’t believe the function moves the output to cpu during the validation phase when repeating the calculation to obtain predictions - this is something I’ve typically seen done. Is that not standard practice?


No, I don’t believe it’s standard practice. Where have you seen it and would you know what the reason would be?