At some point someone has to write a 3d unfold…
You could use checkpointing to trade more compute for less memory. Regarding speed (of the backward?), I seem to recall discussing how to write a faster and deterministic backward for unfold with someone somewhere, but I don’t know if it ever made it to a patch.