@colesbury Thank you for such a quick response.  Although the model is still yielding problematic results in Pytorch using ReLU, I was at least able to get it to converge using selu instead. I guess hoping the model would behave the same in different frameworks was an unrealistic expectation in the first place.