Solved by albanD in post #4 
              
              
                Hi, 
As a general rule, any op that is not provided by pytorch will break the graph (should ideally fail to run if the Tensor requires grad). 
But here you can simply use torch.split or torch.chunk and all will be differentiable