Thanks for the answer @jerinphilip ! Makes sense why it only uses one gpu in the case of fit
So is there anything I can do which can help me in parallelizing the fit method? Or the only way to parallelize a model is to call the forward from dataparallel wrapped model itself?