This is the desired behaviour from python lists.
# my_batch is of dimension (batch_size, seq_length, num_items)
# indices is of dimension (batch_size, num_items)
# x is some value

for i in range(batch_size):
  index = indices[i]
  for j in seq_length:
    for index in indices:
      my_batch[i, j, index] = x

I solved it using scatter_:
# make indices same dimension as my_batch
indices = indices.unsqueeze(1).repeat(1, seq_length, 1)
my_batch = my_batch.scatter_(dim=2, index=indices, src=x)