wenjunli-0:

However, this works can be done in TensorFlow, because they simply take out the gradients and do the update multiple times.


This would be wrong, wouldn’t it?
Could you take a look at this post 22 and check, if you are using the same workflow?