It shouldn’t be unstable but it’s a complicated metric. There are a lot of moving pieces when running the model on cpu.
One suggestion to help you is maybe use Google Colab. It gives you free GPU and that helps a lot.