Solved by ptrblck in post #2 
              
              
                I guess you might have been using plain Python lists or dicts to store the embedding layers. 
If that case, use nn.ModuleList/Dict instead, which will make sure to properly register these modules and push them to the desired devices via the to() operation on the parent model.