@smth To add a little bit more detail, I ran nvprof on both embedding and embedding bag layers; the results are in this gist 19.
The curx of the problem seems to be that the cunn_LookupTableBag_accGradParametersKernel is about twice as slow as cunn_LookupTable_accGradParametersKernel, far outweighing any gains from not allocating intermediate results when summing over some dimensions of embedding layers.