Yes, you can index a tensor like a Python list or NumPy array. If you want to sample without replacement you can use the TensorDataset and DataLoader classes:
import torch
import torch.utils.data

inputs = torch.randn(124, 3, 32, 32)
targets = torch.LongTensor(124).random_(1, 10)

dataset = torch.utils.data.TensorDataset(inputs, targets)
loader = torch.utils.data.DataLoader(dataset, batch_size=31, shuffle=True)
for images, labels in loader:
  print(images.size(), labels.size())