Would splitting the data and sending each chunk to a specific device work?
Something like this could already solve your use case:
data = torch.randn(4, 100)
chunks = data.chunk(4, 0)

res = []
for idx, chunk in enumerate(chunks):
    res.append(my_fun(chunk.to('cuda:{}'.format(idx))).to('cuda:0'))
res = torch.stack(res)