This is the post I based my code on (though I later changed the three optimizer’s to one). I have managed to further isolate my problem in the presence of BatchNormalization in my feature extractor (see original post for update)