Solved by albanD in post #3 
              
              
                The two will actually give you the same result if you write them this way. 
I would recommend the 2nd one like @tux because in the first one, you first create the graph for the softmax then discard it when you detach. In the second one, you just never create that part of the graph. 
But the differen…