In this case, if the above analysis that Adam leads to a lot of computation compared to the forward, then I would expect so yes. Note that momentum or weight decay would slow it down of course.
@siarblack if you can try that, that would be a good way to confirm our hypothesis