I have tried the same dummy data with 1000 samples and the GPU i am using would do it in 0.117 seconds/epoch.
GPU: Tesla P100-SXM2-16GB
CUDA driver version: 10010
cudnn: 7.5.1_10.1
PyTorch: 1.1.0
Can the issue be with data i am loading from files? I am converting it to FloatTensor from pandas dataframe using command
dataX = torch.tensor(dataX.values).to(dtype=torch.float)

Data is available in .csv text file. Number are mentioned in this format.
2.489500679193377281e-03,0.000000000000000000e+00,3.944573057764559962e+02,1.833216381585000068e-02

Thanks.