Your steps seem logic and apparently the debugging worked.
Congrats! Now if your model is learning, you could try to scale it up a bit, e.g. adding data augmentation, and see if you can get a better accuracy.