Solved by Mario_Parreno in post #2 
              
              
                Using torch.no_grad() updates statistics: 
swa_model.train()
with torch.no_grad()
    for indx, batch in enumerate(train_loader):
        image = batch["image"].type(torch.float).cuda()
        _ = swa_model(image)

Check with: 
for module in s 
wa_model.modules():
    if isinstance(module, torch.nn…