Hello @Adi-0-0-Gupta and @naumov-al.
There is an overview of the training task of Language Modeling for T5 in the T5 page 3 on the Hugging Face site at Unsupervised denoising training 13.
And you will get scripts for training as said in this text from Hugging Face:

If you’re interested in pre-training T5 on a new corpus, check out the run_t5_mlm_flax.py 13 script in the Examples directory.

To train the T5 tokenizer vocab on your specific domain, this script should help: t5_tokenizer_model.py 4
Source: Example scripts 4 in the T5 page

pre-training: the run_t5_mlm_flax.py 13 script allows you to further pre-train T5 or pre-train T5 from scratch on your own data. The t5_tokenizer_model.py 4 script allows you to further train a T5 tokenizer or train a T5 Tokenizer from scratch on your own data. Note that Flax (a neural network library on top of JAX) is particularly useful to train on TPU hardware.