I would also assume that the SWALR object is not interfering with the standard training routine and it seems an additional swa_lr entry is created in the para_groups as seen here 2.
However, I’m currently unsure where this swa_lr is used, since the model update doesn’t seem to use here 2.