Thx @albanD I didn’t know that there was custom allocator backing up behind the scene.
In my case, with the assumption that codes executes line by line even in the GPU computation, I just used placeholder.data[:, position] = some_tensor   instead of  placeholder[:, position] = some_tensor. It made me circumvent the autograd problem.
I know I need to check if autograd works as expected but I learn a new thing from your comments! Thx alot!