I consulted the official documentation of Adam & AdamW and noticed that the implementation of weight-decay in Adam also followed the Decoupled Weight Decay Regularization 4 (torch.optim — PyTorch 1.7.0 documentation 2) which is the same for Adam. Does that mean that currently, Adam & AdamW are the same w.r.t. weight decay?