Optimization, Pytorch, Machine Learning, Deep Learning, Tensorboard.
into the pred tensor in the weighted_nll are defined by the r and target tensors. While the target tensor already resides on the GPU, the r tensor, which was defined on the previous line, does not. This, once again, triggers an inefficient host-to-GPU data copy. Optimization #3: Replace range with