Hello @Nick_Young. I have a similar problem like you but the difference is that I have never trained any network. What I have done was to transfer weights in keras to pytorch.  The result are so different. I swear I check every line of code to verify are there any bugs are not. Do you find any difference between keras and pytorch?