Solved by ptrblck in post #2 
              
              
                You could additionally try to print the graphs of both models (assuming you want to compare the forward/backward pass) and check if the computation graphs are also equal.