Solved by albanD in post #2 
              
              
                Hi, 
The autograd engine only records the “basic” operations done on Tensors. So any logic that you add around it will not impact the whole autograd engine. 
The overhead you will have is the overhead of running more python code. Depending on the size of your net, that can be completely negigeable (…