Not sure why, but after replacing the following line :
layer_data['average_derrevative'] = torch.clone(d_p).detach()
with the following :
 layer_data['average_derrevative'] = torch.zeros_like(layer_data['params'].data)
solved my problem. To be honest, I’m not sure why the first command was a mistake. If someone can explain it will be gr8.