To avoid overfitting, you could add a dropout layer before your first fully connected layer, and see if it help. If it do not help, you should try to unfreeze a couple of layers in your model, from my experience, it could help a lot to give your model enough capacity to learn deeper pattern that can be generalized.