Deep Learning, Attention, Transformers, Optical Computing, AI.
With Joseph Wilson In this article we will discuss a promising neural network architecture, FNet, proposed recently by Google research. FNet is a interesting approach to neural network design, in that it replaces the traditional attention mechanism in transformer networks with a novel operation: