Fix FNet reference to tpu short seq length
# What does this PR do?

This PR fixes #13684. 

Should there also be a test for this particular case (TPU usage, less than 4096 seq length) ? 

@LysandreJik @patrickvonplaten
@gchhablani are you btw. working on a flax integration of FNet :thinking: -> I've seen an attempt in this PR #12454 but it was closed. Would be really interesting to train FNet on TPU - GPU is working for me, but I only have one available...