Hey thanks for your reply. I realised that I had extra classifier layers in the backbone which were not used since I took the whole mobilenet model. I managed to remedy it by just taking the .features portion of the original backbone network.