It will basically split your input in the batch dimension across the ids of the GPUs that you pass at initialization.

So this code is enough:
    model = torch.nn.Module(options)

    ...

    if torch.cuda.is_available():
        ids = [i for i in range(torch.cuda.device_count())]
        model = torch.nn.DataParallel(model, device_ids=ids).cuda()
        os.environ['CUDA_VISIBLE_DEVICES'] = ids.join(',')[:-1]
        print("Using ", len(ids), " GPUs!")

    ...

    model_results = model(input)


Note that some model architectures are not able to be parallelized across devices.

What does it depend on?