Pytorch, Optimization, Deep Learning, Artificial Intelligence, Tensorboard.
model(inputs) loss = loss_fn(outputs, label) optimizer.zero_grad(set_to_none=True) with torch.profiler.record_function('backward'): loss.backward() with torch.profiler.record_function('optimizer_step'): optimizer.step() prof.step() batch_time = time.perf_counter() - t0 if step > 1: # skip first