The dependency on the random seen could point towards e.g. a bad initialization of the model and I agree that it feels rather like a band-aid than a proper solution.
It’s hard to isolate the real root cause, since you don’t have the initial code anymore and thus cannot compare different runs. However, I would check the initialization as well as shuffling of the dataset etc., which are all depending on the seed.