I don’t think it’s the algorithms as much as the lack of people doing it.
For example, for many operations (Upscaling comes up frequently) it would be relatively easy to make the cuda backward deterministic by having each grad_in location treated by one thread.
For others, using a index_add/scatter_add-style operation, the Embedding backward may serve as a blueprint.
Best regards
Thomas