@sachinruk Thanks for mentioning, and I would like to know how to extend this to higher dimension.
If we have a 4-D tensor of [batch, channel, height, width], and a set of [x, y] coordinates with shape [batch, num_points, 2] , how to select this 4-D tensor without loop?
Here is my implementation with loop:
Suppose the 4-D feature map [10, 256, 64, 64] in dimension, and coordinates is [10, 68, 2] in dimension(for each batch, there are 68 points that we wanted to select from feature map)
        coord_features = torch.zeros(10, 68, 256)
        feature_map = feature_map.transpose(1,2).transpose(2,3) #reshape to [10, 64, 64, 256]
        for i in xrange(coords.shape[0]): #loop through each sample in batch
            for j in xrange(coords.shape[1]): #loop through each points
                #select coordinate on feature map
                coord_features[i][j] = feature_map[i][coords[i][j][1].type(torch.int64)][coords[i][j][0].type(torch.int64)]