Rohan_Kumar:

1- nn.Embeddings basically creates a 2d matrix of random weights, if pretrained is not provided. And starts training them via grad descent, is this right? If so does this mean there are additional vocab_size*emb_dim parameters to train?


nn.Embeddings create a 2d matrix. But, you can choose the weight initialization (eg: random or something else). Yes, if you choose to train these embeddings, there are additional vocab_size*emb_dim parameters to train.



 Rohan_Kumar:

2- which brings to my follow up question, does the pretrained embeddings only function is to speed up training (by whatever factor, probably huge tho), I had the following graph of loss vs iterations without pretrained embeddings for 5 epochs. Do you think there might be any other issues in my code except embeddings?


Pre-trained embeddings not only reduce the number of parameters to train (hence reducing the train time), but also bring the “knowledge” of words (eg: word2vec) and context (eg: BERT embeddings). Hence can be directly used for downstream tasks.
Clearly, your model is not learning much (loss not decreasing with epochs). It’s hard to figure out the reason just on the basis of this plot. Better use average loss (for each epoch) to getter a better sense of loss and investigate further.