If the domain of your dataset isn’t very specialized from the pre-training datatset then training tokenizer won’t help much. Also note that if you train a new tokenizer then you’ll need to do the pre-training again, and when you train your tokenizer from scratch you don’t need to add the new tokens any more since you already trained tokenizer.  Hope this makes sense.

Is my dataset large enough

IMO this is highly subjective question and depends on the quality of the data, the task etc.
@sshleifer might have some advice  for this