Making closures work with dynamic gradient scaling (specifically, the fact that dynamic gradient scaling occasionally skips optimizer.step() if any grads were inf/nan) is tricky, and we haven’t heard any use cases that absolutely needed it (LBFGS is the only one I’m aware of and no one’s asked for that).
Can you implement SAM without a closure like this?
  
      

      GitHub
  

  
    

GitHub - davda54/sam: SAM: Sharpness-Aware Minimization (PyTorch) 3

  SAM: Sharpness-Aware Minimization (PyTorch). Contribute to davda54/sam development by creating an account on GitHub.