If you do not print the value, but instead use a SummaryWriter for Tensorboard, is there a way to add the value to the writer in an asynchronous way (e.g. in a separate thread, when the GPU has finished fetching the loss, it will be added to the writer corresponding to the unique step specified in the add_scalar argument)?
Have not seen this anywhere but would be faster especially when training times are high.