Solved by lm_modeler in post #3 
              
              
                If someone else runs into this challenge and you don’t have a specific solution in mind, the best thing to do seems to be to carefully follow each and every tensor. In my case, I was using a transformer model whose wte layer was called on the lm_head, meaning that the lm_head had to (counter intuiti…