Optimization, Pytorch, Machine Learning, Deep Learning, Tensorboard.
suspicions, including “to”, “copy_”, and “cudaStreamSynchronize”. This combination usually indicates that data is being copied from the CPU into the GPU — not something we want to be happening in the middle of our loss calculation. We now know that we have a performance issue in our loss function