Based on the code x_categorical should have the shape [batch_size, len(self.all_embeddings), *], since you are indexing this tensor in dim1.
If you are passing x_categorical as a tensor with the shape [3, 1], self.all_embeddings can only contain a single nn.Embedding layer.
Could you explain your use case a bit more and how you would like to use the for loop?