amosella, you are right I think there is something that is shifting the random state in my 2nd pipeline. So when I kept shuffle = False for training dataset both pipelines produced the same result.
Other things I tried are:

Removed batch norm from the model and shuffle = True gave different results(from 2nd epoch onwards ) implies there is no problem with batch norm (I have the changed title name of the thread).

As 1st epoch results are same for both the pipelines I think adding test predictions is changing the random state.

removed test predictions from the second pipeline and added validation predictions one more time. It is shown below

for epoch_number in range(epochs):
        Train the model
        Make predictions on validation 
        Make predictions on validation             # 2nd time
        save the model if validation score increases..

and to my surprise the results are same as my second pipeline.