What do the following parameters mean during the initialization of T5 model?
Hello,

  I am aware of the general Transformer model and I believe it is the same model used in T5 architecture. 

I know we have the input vocab_size which is the total number of vocabulary size. Besides this, the important parameters would be the embedding size (the size of the embedding of each token), the number of layers, the number of heads and others.

Particularly, looking at the T5Config class that is used to initialize a T5 model, what is the d_kv and d_model parameter? 

Is d_model the size of the embedding and in which case what is the d_kv? The docstring is not really clear for me.

Thanks for your help.
I added a description of `d_kv`. `d_model` is as you say the size of the embedding. `d_kv` is the size of the key, query, value projections. 

If you look at this blog post: http://jalammar.github.io/illustrated-transformer/

`d_model` corresponds to the size of a vector `x_1` and `d_kv` of a vector `q_1, k_1, v_1`