Solved by tom in post #2 
              
              
                I am not 100% sure I understand what you attempt to do, but would scattering a bool tensor True to the indices and then taking the complement (using the ~ unary operator) and applying this as a mask in advanced indexing work? 
inp = torch.arange(25.).view(5, 5)
idx = torch.randint(4, (5, 1)) + torch…