<p>Regularizer's are used as a means to combat over fitting.They essentially create a cost function penalty which tries to prevent quantities from becoming to large.  I have primarily used kernel regularizers. First I try to control over fitting using dropout layers. If that does not do the job or leads to poor training accuracy I try the Kernel regularizer. I usually stop at that point. I think activity regularization would be my next option to prevent outputs from becoming to large. I suspect weight regularization effectively can pretty much achieve the same result.</p>

CC BY-SA 4.0
2020-07-19T00:50:58.523Z
2020-07-19T00:50:58.523Z
Gerry P
<p>For Keras on TensorFlow, a layer class constructor comes with these:</p>
<ul>
<li><code>kernel_regularizer</code>=...</li>
<li><code>bias_regularizer</code>=...</li>
<li><code>activity_regularizer</code>=...</li>
</ul>
<p>For example, Dense layer:<br/>
<a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense#arguments_1" rel="nofollow noreferrer">https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense#arguments_1</a></p>
<p>The first one, <code>kernel_regularizer</code> is easy to understand, it regularises weights, makes weights smaller to avoid overfitting on training data only.</p>
<p>Is <code>kernel_regularizer</code> enough? When should I use <code>bias_regularizer</code> and <code>activity_regularizer</code> too?</p>

<neural-networks><keras><regularization><weights>
When would bias regularisation and activation regularisation be necessary?
Dan D.
# When would bias regularisation and activation regularisation be necessary?

(Asked by: username_0 on 2020-07-18)

For Keras on TensorFlow, a layer class constructor comes with these:

* `kernel_regularizer`=...
* `bias_regularizer`=...
* `activity_regularizer`=...

For example, Dense layer:  
<https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense#arguments_1>

The first one, `kernel_regularizer` is easy to understand, it regularises weights, makes weights smaller to avoid overfitting on training data only.

Is `kernel_regularizer` enough? When should I use `bias_regularizer` and `activity_regularizer` too?

**Tags:** neural-networks, keras, regularization, weights


---

## 1 answers

### Answer by username_1 on 2020-07-19. Score: 2 | Accepted answer

Regularizer's are used as a means to combat over fitting.They essentially create a cost function penalty which tries to prevent quantities from becoming to large. I have primarily used kernel regularizers. First I try to control over fitting using dropout layers. If that does not do the job or leads to poor training accuracy I try the Kernel regularizer. I usually stop at that point. I think activity regularization would be my next option to prevent outputs from becoming to large. I suspect weight regularization effectively can pretty much achieve the same result.


---

