RAdam+LA 588
from radam import RAdam
from optimizer import Lookahead
base_optim = RAdam(model.parameters(),lr = 0.001) 
optimizer =  Lookahead(base_optim, k=5, alpha=0.5)

RAdam 102 | LAMB 26 | Ralamb 41 | Lookahead 76