🌟 New model addition: FNet
# 🌟 New model addition: FNet

FNet is a highly efficient Transformer-like encoder architecture, wherein the self-attention sublayers have been wholly replaced by standard, unparameterized Fourier Transforms.


I would like to help adding this!

## Open source status

* [x] the model implementation is available: https://github.com/google-research/google-research/tree/master/f_net
* [x] the model weights are available: https://github.com/google-research/google-research/tree/master/f_net
* [x] who are the authors: (@ilyaeck @santiontanon) (Not sure, googled the authors' name + github, sorry if it's incorrect)
Somebody is already working on this, see #12335 <|||||>Thanks @NielsRogge , weird that I didn't see it when I searched.<|||||>@cccntu I believe what you want for the JAX/Flax community week is a Flax model. It seems unlikely that I will finish the PR in the next week. Maybe, you can start working on the Flax model parallely?

Or, we can discuss over slack and then try to finish both.

@patil-suraj @patrickvonplaten wdyt? Is it easier to go from PyTorch to Flax? Or it doesn't matter at all? In case PT is needed, I am willing to spend my time next week on this and try to finish it.

<|||||>@gchhablani Yes! I would love to add the Flax part. 
@patil-suraj @patrickvonplaten  I have a few questions before I proceed: 
* There is no license in the original repo, should I email the authors for permission for code and weights? 
* How much of the original model code should I modify, other than wrapping it in huggingface/transformers classes?
Should we refactor it for better weight alignment with pytorch code e.t.c?

Thanks!

<|||||>Great @cccntu! Let's discuss over Slack.