Thank you very much. I get it now. In parallel I am training keras model with almost same hyperparameters but I observe that keras model runs faster and is giving me better results. Pytorch model is outputting the same character again and again. Both the models have been trained for the same  number of epochs. That is why I am doubting the correctness of my model in pytorch. I will incorporate all the changes you suggested and will retrain. Does the model build look good otherwise or am I still missing something? BTW heres the keras code:
seq_len = len(dataX[0]) 
input_layer = Input(shape=(seq_len,))
embedding_layer = Embedding(len(char_to_id)+1,100,input_length=seq_len)
embeds = embedding_layer(input_layer)
lstm_layer = LSTM(50,input_shape=(seq_len,100),return_sequences=False)
lstm_out = lstm_layer(embeds)
dense_layer = Dense(len(char_to_id),activation='softmax')
dense_out = dense_layer(lstm_out)
model = Model(input_layer,dense_out)
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['acc'])
model.fit(dataX,dataY,batch_size=100,epochs=3)