Regarding the slowdown, I can reproduce it locally with two GPUs:
Using DataParallel on 2 GPUs
For epoch 999, loss is: 8.4603910446167.
elapsed: 3.1627652645111084

Not using DataParallel
For epoch 999, loss is: 8.323615074157715.
elapsed: 1.192000389099121

Then I go back to inspect the model:
    def __init__(self, dim):
        super(predictor, self).__init__()
        self.weights = torch.nn.Parameter(torch.zeros(dim, 1, requires_grad=True))
        self.threshold = torch.nn.Parameter(torch.zeros(1, 1, requires_grad=True))

My suspicion is that the model parameters are so light that the overhead of GIL contention, input scattering, output gathering, and model replication in DataParallel forward pass overshadows the speedup brought by multi-gpu training. Are these parameters used in real applications or are you trying to profile DataParallel performance?