How did you process the data that the spikes went away?
Based on your initial post, you are dealing with the ratio of 50:1 in your imbalanced dataset.
As I explained, the accuracy might be misleading for it and you should get a ~98% accuracy if your model just outputs the majority class.