I solved the problem but there is something I can’t understand , in this block of code the problem is solved and the generated average model work as expected
model_1=torch.load('Ensemble_Modelfold-1.pt')
model_2=torch.load('Ensemble_Modelfold-2.pt')
model_3=torch.load('Ensemble_Modelfold-3.pt')
model_4=torch.load('Ensemble_Modelfold-4.pt')
model_5=torch.load('Ensemble_Modelfold-5.pt')

for key in model_1.state_dict():
    model_1.state_dict()[key] = (model_1.state_dict()[key] + model_2.state_dict()[key]+  model_3.state_dict()[key]+model_4.state_dict()[key] + model_5.state_dict()[key] ) / 5
    
Average_Model = MyEnsemble( modelA, modelB, modelC , modelD).to(device)
Average_Model.load_state_dict(model_1.state_dict())

but If we use this block which almost similar to the first one, the generated average model gives me the same issue
model_1=torch.load('Ensemble_Modelfold-1.pt')
model_2=torch.load('Ensemble_Modelfold-2.pt')
model_3=torch.load('Ensemble_Modelfold-3.pt')
model_4=torch.load('Ensemble_Modelfold-4.pt')
model_5=torch.load('Ensemble_Modelfold-5.pt')

sd1=model_1.state_dict()
sd2=model_2.state_dict()
sd3=model_3.state_dict()
sd4=model_4.state_dict()
sd5=model_5.state_dict()

for key in sd1:
    sd1[key] = (sd1[key] + sd2[key] + sd3[key]+sd4[key] + sd5[key] ) / 5
    
Average_Model = MyEnsemble( modelA, modelB, modelC , modelD).to(device)
Average_Model.load_state_dict(sd1)

So, Why this is happening ?