@ayalaa2 thanks for your time.
The gradients are nearly zero, but that is not quite the point. The learning rate and all other parameters are the exact same from the Keras model I am referring to. Needless to say the gradients in Keras were much larger.
I have gotten this model to replicate performance I had in Keras, but I made a few changes in the code and haven’t been able to since. The script I have posted is nearly the exact code I had at that point in time, I can’t seem to troubleshoot it.
I tagged this as autograd in case the issue had to do with how I am setting up the optimizer, as I am new to pytorch.