The high variance in the loss and/or accuracy is most likely due to the small datasets.
144 samples is a tiny dataset and I’m skeptical, if you would be able to train a neural network with it at all, such that it can generalize to new, unseen data.