Flan-T5-small and T5-small have different number of layers?
### System Info

Hi there, it appears that `google/flan-t5-small` and `t5-small` have different number of layers:
- Flan-T5 config: https://huggingface.co/google/flan-t5-small/blob/main/config.json
- T5 config: https://huggingface.co/t5-small/blob/main/config.json

I only find this inconsistency with *small*. The rest of the sizes (base/large/3b/11b) seem to match up for these two sets of models. I have not been able to find much information on this.

Is there a reason for Flan-T5-small to have more layers than its non instruction-tuned counterpart? I assume they should be equal. Thank you!

### Who can help?

CC: @sgugger 
cc @younesbelkada <|||||>@sgugger Ah perhaps the authors inherited the flan-t5-small checkpoint from the improved `google/t5-v1_1-small` instead of `t5-small`. Its config file is a better match.<|||||>Hi @taidnguyen,
This is absolutely correct, 
According to the `t5x` repository, flan-t5 are derived from the `t5-v1_1` family as their config files refer to the config files of the `t5-v1_1` models. This information can be found on the original repository that hosts the original flan-t5 weights: https://github.com/google-research/t5x/blob/main/docs/models.md#flan-t5-checkpoints 
Hope this helps!<|||||>@younesbelkada That helps - thank you! They documented this in their repo (as you show) but not in their paper, so definitely a surprise to me assuming that the inherited checkpoint of Flan-T5 is the original T5. I will close the issue.