I’m not sure, if I understand the use case correctly, but in case you would like to use the gradients in updates_list, you could assign these values to the .grad attributes of the parameters of the global_model and call optimizer.step() afterwards.