Solved the issue by using torch.nn.ModuleList.
Basically, the problem was that the list I was creating, was on the CPU.
self.embedding_layers = nn.ModuleList()
for k, v in categ_embs_dims.items():
    self.embedding_layers.append(nn.Embedding(v, self.config["cat_embs"][k]))

Then I also changed the forward part into this:
x_cat = [emb_layer(x_cat[:, idx]) for idx, emb_layer in enumerate(self.embedding_layers)]
x_cat = torch.cat(x_cat, 1)

Now it works on both CPU and CUDA gpu.