A quick test shows that the data is split sequentially:
import torch
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()

    def forward(self, x):
        print(x, x.device)
        return x

model = MyModel()
model = nn.DataParallel(model)
model.cuda()

batch_size = 2
x = torch.arange(batch_size * 8).float().view(batch_size*8, -1)

out = model(x)

Output:
tensor([[2.],
        [3.]], device='cuda:1') cuda:1
tensor([[0.],
        [1.]], device='cuda:0') cuda:0
tensor([[10.],
        [11.]], device='cuda:5') cuda:5
tensor([[8.],
        [9.]], device='cuda:4') cuda:4
tensor([[14.],
        [15.]], device='cuda:7') cuda:7
tensor([[4.],
        [5.]], device='cuda:2') cuda:2
tensor([[6.],
        [7.]], device='cuda:3') cuda:3
tensor([[12.],
        [13.]], device='cuda:6') cuda:6

If you need to shuffle the data, you would have to apply the shuffling manually before passing the data to nn.DataParallel (assuming you are using this approach).