I want to use a custom activation function that has a random component that gets applied to every neuron individually.
If I use the standard method and call the activation function on a layer, it applies the same value to every neuron in that layer.
I am looking for the most efficient way to have the activation function affect every neuron individually and would appreciate any advise on the topic.
The only method I currently know about involves splitting the input tensor and applying the function each part and then concatenating the result, but I am hoping there is a more efficient way