Look at the broadcasting semantics on the docs: http://pytorch.org/docs/master/notes/broadcasting.html 11
You have two tensors that don’t have the same size, so the broadcasting semantics comes in.
First, (16) is compared to (16, 1) in the trailing dimension first. 1 can be broadcasted to 16 so the tensors are broadcastable. What happens (according to the doc) is that extra empty dimensions are prepended to the smaller tensor, so it effectively has size (1, 16).
Now, comparing (1, 16) with (16, 1) results in a result tensor with size (16, 16), which is what you’re getting.