Solved by Vasiliy_Kuznetsov in post #2 
              
              
                Yes, this is correct.  You can check this out in the source code, here: pytorch/functional_modules.py at b89827b73f7881a8108518452768654894862e5d · pytorch/pytorch · GitHub