<p dir="auto">Is there interest in adding the <a href="https://arxiv.org/abs/1908.08681" rel="nofollow">mish</a> activation function to <code class="notranslate">jax.nn</code>?</p>
<blockquote>
<p dir="auto">It has a much smoother curve vs relu, and in theory, that drives information more deeply through the network. It is slower than ReLU, but on average adds about 1-3% improvement vs ReLU. - LessW2020, Fastai forums</p>
</blockquote>
<p dir="auto">While implementing setdiff1d, I've come across this weird case where the <code class="notranslate">np.in1d</code> would return duplicate items if it finds such items and the <code class="notranslate">np.in1d</code> implementation would return the set-like array.</p>
<p dir="auto">I've attached a code sample to reproduce this below:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from jax import numpy as jnp # version: 0.2.3
import numpy as np # version: 1.19.2

a = jnp.array([3,-1,0,0,7,-3,-5,0,3,-5,-3,7,0,1,-1,-1,0,1,-1,0,-7,-4,3,-2])
b = jnp.array([-2,-3,-1,0,3,-2,6,0,1,0,0,-5,-2,0,0,0,-2,-2,6,2,-6,-3,-2,-3])

lax_ar = jnp.in1d(a, b, invert=True)
np_ar = np.in1d(a, b, assume_unique=True, invert=True)

print(a[lax_ar])
print(a[np_ar])"><pre class="notranslate"><code class="notranslate">from jax import numpy as jnp # version: 0.2.3
import numpy as np # version: 1.19.2

a = jnp.array([3,-1,0,0,7,-3,-5,0,3,-5,-3,7,0,1,-1,-1,0,1,-1,0,-7,-4,3,-2])
b = jnp.array([-2,-3,-1,0,3,-2,6,0,1,0,0,-5,-2,0,0,0,-2,-2,6,2,-6,-3,-2,-3])

lax_ar = jnp.in1d(a, b, invert=True)
np_ar = np.in1d(a, b, assume_unique=True, invert=True)

print(a[lax_ar])
print(a[np_ar])
</code></pre></div>
<p dir="auto">Upon further verification of this issue I've found that the problem reproduces due to numpy's implementation of in1d. I believe we might need to implement the sorting based option for longer sequences.</p>
<p dir="auto">Also, if it is possible, I would like to try and implement it :).</p>