Sure!
I used the configuration for forward() in my second post,
for training i simply do
label = torch.LongTensor(input_embedding.size(0).fill_(label_num)) 
#label_num is some label corresponding to the 3 batches,
# since all the 3 batches at a time belong to the same label, maybe it doesn't matter possibly the reason
# for giving me good result.

out = model_lstm(input_embeddings)
# input_embeddings has size (5,3,1024) where 5 is the sequence length and 3 is the number of batches
loss = criterion (out.view(out.size(0),-1),label)

loss.backward()
op.step()

To calculate the validation accuracy in the train function I just do
_,predicted_label = out.max(dim=1)
for element in predicted_label.cpu().data.numpy().astype(int):
    conf_matrix[label.cpu().data[0],element]+=1
    # conf_matrix is a confustion matrix I make in the beggining
    # note that I only use the 0th position of the true label 
    # since at a time the batches loaded belong to the same label.

I am guessing that the good results despite moving from num_batches =1 to num_batches > 1 is because of the way I load the data. It may seem that batch and sequence is commutative of sorts since at a time all the frames in embedding belong to the same label.
What I mean is
1 2 3 4 5
6 7 8 9 A
B C D E F
Although I have organized the 1 2 3 4 5 part as one sequence in batch dim = 0 , the way I fed data into the linear layer might be looking at 1 6 B as one batch instead but since 1 and 6 and B have the same label and similar appearance, the results are still “good”