Cannot import name swish from transformers.activations
I have installed `transformers v4.4.1` and `tensorflow v2.4.1`. 

I tried to run
`from transformers.activations import gelu, gelu_new, swish`. 

I get the error like this:
`ImportError: cannot import name 'swish' from 'transformers.activations' (/Users/array/opt/miniconda3/lib/python3.7/site-packages/transformers/activations.py)`

Is there any solutions for this error?

Thank you🙏

Hi! `swish` is not importable because it isn't available. `swish` is another name for `silu`, but arrived after it so the name you can use is `silu`:

```py
>>> from transformers.activations import silu
```

However, in our `ACT2FN` dict we have support for both `swish` and `silu`, so that you can do:
```py
>>> from transformers.activations import ACT2FN
>>> swish = ACT2FN["swish"]
>>> silu = ACT2FN["silu"]
```<|||||>Thank you