I noticed that advanced indexing works differently on CPU and GPU with PyTorch 0.4, when multi-index is large by length and contains repeated elements.
Minimal working example:

This code works on CPU and saves the last element encountered in a repeated indexed cell.

len_idx = 150
for i in range(10):
    A = torch.zeros(5, 5, 5)
    idx_along_axis = torch.ones(len_idx).long()
    A[idx_along_axis, idx_along_axis, idx_along_axis] = torch.arange(len_idx)
    print('A[1, 1, 1]:', A[1, 1, 1].item())

produces:
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0


The same code on GPU saves a random element from all values corresponding to a repeated indexed cell.

len_idx = 150
for i in range(10):
    A = torch.zeros(5, 5, 5).cuda()
    idx_along_axis = torch.ones(len_idx).long().cuda()
    A[idx_along_axis, idx_along_axis, idx_along_axis] = torch.arange(len_idx).cuda()
    print('A[1, 1, 1]:', A[1, 1, 1].item())

produces different output each time, e.g. this:
A[1, 1, 1]: 149.0
A[1, 1, 1]: 31.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 31.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 149.0
A[1, 1, 1]: 31.0
A[1, 1, 1]: 31.0
A[1, 1, 1]: 31.0
A[1, 1, 1]: 31.0

This only works for large index arrays, for example, the problem cannot be reproduced for len_idx=30 on my machine. If we take larger len_idx, the set of saved values becomes even more random. Is it a bug in the framework?
Thank you.