It should gradually decrease. (some fluctuation is ok, but not strong fluctuation as in your case). If the model is correct the fluctuation might be due to bad hyperparameters (probably lr or momentum). Also, do not use any data augmentation, weight decay, dropout or any other fancy regularization trick.