I guess I now realize this is very related to:

  
    
    
    How to get mini-batches in pytorch in a clean and efficient way? 
  
  
    X can be a Tensor or a Variable. batch_indices is a LongTensor 
X = torch.randn(10)
batch_indices = torch.LongTensor([0, 1, 2])

X[batch_indices]
Variable(X)[batch_indices]