Gradient accumulation shouldn’t be a problem, as no new .grad attributes will be created and the already created ones will be used.
Did the check for empty annotations work, i.e. are you still running out of memory?