Thanks for your answer.
Yeah. I think the inverse indices will fix my case exactly, but the unique function of GPU implementation currently is not able to do it, also I cannot find a subsitute for it. I hope it can be fixed soon.