My data had size 100k examples if I remember correctly. You can most likely get away with a lot less without a big trade-off on performance if you tune correctly but more data is almost always better. Maybe on the order of a couple hundred or thousand depending on the complexity of the task?
Note that details related to training and/or how you structure your inputs to be similar to what is seen during pretraining start to have larger impact when you go down to small dataset sizes. If you have a really small dataset and your task is similar enough to summarization, that’s when you may see some lift by trying to use the existing prompt. There was a paper by huggingface on prompts and data efficiency during fine tuning a while back. IMO, try both ways and see what works best, I’d be interested in hearing any results you come up with.