@Bhack
I did now know about image augmentation, will check that out thanks 
It seems like you are correct, here is my total loss from steps 0 to 10k. Really spiky and the loss on step one is only 1… So you’re saying I need additional data to solve this? My 500 image data I’ve been using is good so there should not be any problems there…