That certainly sounds like the issue. Here’s the implementation https://github.com/pytorch/pytorch/blob/5bb13485b8484a37f9afad67582512cf53ed13cb/torch/nn/_functions/linear.py#L35; 5 I can see a few tricky lines but there isn’t (at least to me) an obvious bug. Maybe you can add some print statements to it locally and try to narrow things down? Or feel free to just open a github issue.