Solved by justusschock in post #6 
              
              
                For me using built-in pytorch methods as gather, index_put_, masked_scatter_ or grid_sample instead of direct indexing. 

Note: since some of the functions are equivalent to direct indexing, it could work with flattening the tensor via .view(-1) and index the flattened tensor.