ptrblck:

avior using a specific model or always?
In any case, could you post a minimal code snippet to reproduce this behavior as well as your current setup (PyTorch, CUDA, cudnn versions, which GPU y


thank you for reply!
I thought this is generally happens
this is code I use for train
    disp_L = model(imgL, imgR)
    loss = F.smooth_l1_loss(dispL, dispL_GT, size_average=True)
    optimizer.zero_grad()
    loss.backward() # here is peak of overshoot in iteration 1
    optimizer.step()

current environment
model : any model(ex. PSMNet, Segment, yoloV4, …)
GPU : RTX 2070
CUDA : CUDA 11
pytorch : 1.7
I check GPU memory usage by
watch -n 0.01 nvidia-smi