In aten/src/ATen/native/embedding/Embedding.cu 3, embedding_dense_backward_cuda essentially computes and index_add  – the gradient for embedding matrix row i is the sum of all input gradient elements where i was embedded in the forward. (Disregard the scale_grad_by_weight and only consider the false case.)
Two variants are implemented

For small batches (< embedded786 elements in total) a direct variant is implemented in embedding_backward_feature_kernel. That is somewhat elaborate cuda with explicit communication amongst the threads to ensure determinism. If you didn’t do much cuda programming, this will be an opportunity to learn a lot, but at <100 lines of code with ample comments it should be OK.
For large batches one falls back to sorting the indices (this is still done in embedding_dense_backward_cuda) and then hand off to embedding_backward_cuda_kernel in EmbeddingBackwardKernel.cu 1. Here again, you can assume offset2bag to be undefined. It’ll do quite a few steps to partition the work and then execute it, but it’s implemented as thrust calls and a few fairly simple kernels.

So it takes a quite a bit of work to do this properly, but the ingredients are all there.
Best regards
Thomas