Solved by AlphaBetaGamma96 in post #2 
              
              
                The forward method in nn.Linear is defined as y = xW^T + b, so yes the weight matrix is transposed in the operation. You can read more in the docs, 
https://pytorch.org/docs/stable/generated/torch.nn.Linear.html?highlight=nn%20linear#torch.nn.Linear