A loss of 0.693 means that the model is not doing any better than a coinflip which is consistent with not learning anything after the random initialization. I don’t see anything obvious that indicates why the model cannot learn more than random chance. Can you share some other parts of the code, such as where the optimizer is defined?