Thank you for the speedy response!
I gleaned from the source comments that the CPU implementation is a temporary solution; however, even on the GPU EmbeddingBag is substantially slower. In the snippet I posted above, it’s ~25% slower; if you increase the size of the embedding layer and the number of indices used to
    layer = layer_class(100000, 256)
    indices = Variable(torch.ones(10 ** 5).long().view(10 ** 3, 10 ** 2))

it becomes twice as slow as the Embedding layer followed by a sum (measured on the Amazon p2.xlarge instance).