I do realize that Im waking up an old thread. Apologies. However I dont get results as I expected: see code below
Running a slight modification of this benchmark code ( to load the gpu sufficientlly and sync cuda) on pytorch 1.3,  Driver Version: 435.21       CUDA Version: 10.1; TitanRTX  does not justify the claim that EmbeddingBag is more efficient. In CPU it is 50% better, in GPU marginally worse.
caveat, Ive been working with vision mostly, only just now started looking at Embedings to combine text and vision. If I am using embeddings incorrectly please let me know.
Perhaps to get the full utility of embedding/bag i need to run deeper network? Or?
import time
import torch
import torch.nn as nn
def time_layer(layer_class, repetitions=100, device='cpu'):
    mul = 1
    if device == "cuda":
        mul = 10

    layer = layer_class(100000*mul, 512)
    layer.to(device=device)
    indices = torch.ones(10000*mul, device=device, dtype=torch.int64).view(1000*mul, 10)
    offsets = torch.arange(0, indices.numel(), indices.size(1), device=device, dtype=torch.int64)

    start = time.time()
    for _ in range(repetitions):
        if isinstance(layer, nn.EmbeddingBag):
            result = layer(indices.view(-1), offsets).view(indices.size(0), -1)
        else:
            result = layer(indices)
            result = result.sum(1)

        loss = result.sum()
        loss.backward()

    print(result.size(), result.device)
    if device == "cuda":
        torch.cuda.synchronize()
    return time.time() - start

if __name__ == '__main__':
    DEVICE = 'cpu'
    print(DEVICE)
    embedding_time = time_layer(nn.Embedding, device=DEVICE)
    embedding_bag_time = time_layer(nn.EmbeddingBag, device=DEVICE)

    print('Embedding:\t{},\nEmbeddingBag:\t{}'.format(
        embedding_time,
        embedding_bag_time))
    
    Z = torch.zeros(1, device='cuda') # isolate cuda init from timing
    DEVICE = 'cuda'
    print(DEVICE)

    embedding_time = time_layer(nn.Embedding, device=DEVICE)
    embedding_bag_time = time_layer(nn.EmbeddingBag, device=DEVICE)

    print('Embedding:\t{},\nEmbeddingBag:\t{}'.format(
        embedding_time,
        embedding_bag_time))
python embedbag.py 
``` Results
cpu
torch.Size([1000, 512]) cpu
torch.Size([1000, 512]) cpu
Embedding:      6.203256607055664,
EmbeddingBag:   4.765470743179321
cuda
torch.Size([10000, 512]) cuda:0
torch.Size([10000, 512]) cuda:0
Embedding:      1.6973094940185547,
EmbeddingBag:   1.9424304962158203

Is this code using the embeddingbag wrong?
Thank you