No, the thing I had in mind as C is what is done e.g. in the optimizers or discussed (not quite as clearly as I would need to understand it) on the moving average wikipedia page, see the paragraph starting with  It can also be calculated recursively without introducing the error ….
This isn’t the average over the entire dataset as it weights the later losses more than the former.
Best regards
Thomas