NN arch
[input] -> [embeddings] -> [BiLSTM] (two hidden layers torch.cat) -> 
    [FC layers incl dropout] -> [output]

Code Structure

grid search for best learning rate, batch size, FC - #layers, width, etc.
chose best model based on F1 on validation
re-train final model from scratch using “best” parameters from grid search, and compute metrics

Seeds have been set before EVERY training run torch.manual_seed(some value) and     torch.cuda.manual_seed_all(some value)
The following works fine (best from grid search get perfectly replicated)

training entire network starting with uninitialized embeddings (object instantiated as  nn.Embedding(#embed, edim, padding_idx=0))
importing embeddings and setting up embedding layer using
nn.Embedding.from_pretrained(my_embeddings, freeze=True, padding_idx=0)


When the freeze flag is changed to False, the model is not reproducible!  The code is EXACTLY the same as before (as in 2).  The only thing changed is the freeze=False.
Should I be setting some other seed? Or is this a known issue?