You can check the note about broadcasting here 5. But the main point in there is the first sentence “Many PyTorch operations support NumPy Broadcasting Semantics .”: we follow numpy’s semantic because many users are used to them.