Generally, yes. I would assume the GPU implementation show the same behavior as the CPU one.
However, are we forcing the current behavior on the CPU (or is it defined somewhere else) or are we just “lucky” to get the same result for overlapping indices?
Also, thanks for bringing up this issue. 
CC @colesbury, @ngimel