Hi,
I am training a 7 layers linear fully connected network with 48 neurons in each hidden layer (which gives 14353 learnable parameters). My data has 3 input features and 1 output. Data size is around 51230. I am using Dataloader with 20 batches. However the time improvement from CPU to GPU is only of 30-40% reduction in training time. After experimentation, i have noticed that GPU would only give significant time improvement if the total number of learnable parameters in significantly increased, say to the order of millions (then we can have training time reduced around 7 times). Can we not achieve significant benefit from GPU for NN model with 14353 parameters?
Overall if i train it for 200 epochs, these are time comparisons:
For 14k model parameters:
CPU: 5.7 min
GPU: 4.1 min
For 3.54 millions model parameters:
CPU: 31.0 min
GPU: 4.27 min
Is there any other way i can reduce my training time for around 14k model parameters?