How are you splitting your data into training and validation sets? My hunch is that your data isn’t shuffled before splitting, which can result in your training set not being representative of the dataset as a whole.