@ptrblck Thank you for replying. I understand it now.
Actually I used regularization(weight_decay in SGD) and also data augmentation(horizontal_flip), but somehow it is still overfitting. Few hours back I tried to add dropout layers to see if the problem of overfitting can be solved. But that too did not solve the problem. Now I do not know whats the next step.