Hi Peter,
Thank you for your help. I removed that I’m still seeing the same issue. After three epochs I see an accuracy of
epoch1: loss=7.85208056640625 acc=0
epoch2: loss=5.965506981658936 acc=0.0125
epoch3: loss=23.37786726989746 acc=0.0004
I tried adding batch normalization to my shortcut functions (as I wasn’t doing that before for the convolutions there) but I am still seeing lower accuracy than I would expect for these epochs. Do you see anything else I might change? After further investigating I think there is something wrong with my overall structure but I’m not sure as to what it is.