@ptrblck Thanks for your comment and validating my intuition here. I had not really seen this EmbeddingBag layer before, but I can see now that it actually combines the data batch into a long vector and then uses the offsets to identify the boundaries of each example. Haha, it is not a structure that I was accustomed to, since the regular Embedding layer does not work that way.
Does this EmbeddingBag layer work better than standard embedding layers? Seems like I can still use existing Glove embeddings and other downloaded embeddings with the EmbeddingBag too.