As an FYI, it is also quite common in (niche) NLP research where we often have to create our own labeled datasets manually, resulting in rather small datasets. But as you say it is not always “worth” the time and effort because transfer learning already works quite well. You may be better off simply doing something like gradually unfreezing layers or explicitly over-fitting (depending on your use-case!).