You should strongly consider data augmentation in some meaningful way. If you’re attempting to do classification then think about what augmentations might add useful information and help distinguish classes in your dataset.  In one of my cases, introducing background variation increased recognition rate by over 50%.  Basically, with small datasets there is too much overfitting so you want the network to learn real-world distinctions vs. irrelevant artifacts like backgrounds / shadows etc.
Alternatively, as Andrew Ng said: “get more data”     <-- seems to be the cure to everything.