alex.veuthey:

dulers’ step method, which is different from that of the optimize


Thanks. I was mistake when copy paste. Actually, the first net used betas=(0.5, 0.999) while the second net use betas=(0.9, 0.999). That is reason why I used two seperated scheduler