Thanks! torch.split is exactly what I was looking for. Just a follow-up question based on your response. Is slicing via [ ] safe for autograd?
my_tensor[new_indices, :, :]  # can I re-order a 3D tensor like this? or should I use torch.index_select?