❓ Why multiplying the output of T5 by some scalar before LM head ?
# ❓ Questions & Help

I'm wondering why multiplying the outputs of T5 by some scalar before inputting in the LM head :

https://github.com/huggingface/transformers/blob/1d2332861f5225aef17bd7e75abc670a72239081/src/transformers/modeling_tf_t5.py#L1147


I think this would be a great question for our new forum, with a title like "T5 architecture": https://discuss.huggingface.co/ - would you mind posting it there? This seems like an interesting question people would probably like to see there.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
