<p>I found a <a href="https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.1421&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer">paper</a> about using an Unscented Kalman Filter(UKF) for traning a neural network.</p>
<p>The UKF filter is modified so it works for parameter estimation. Assume that we have a neural network model <span class="math-container">$\hat d_k = G(x_k, W_k)$</span> where <span class="math-container">$G$</span> is the neural network, <span class="math-container">$x_k$</span> is the input <strong>vector</strong>, <span class="math-container">$W_k$</span> is the parameter gain <strong>matrix</strong> and <span class="math-container">$\hat d_k$</span> is the output <strong>vector</strong>.</p>
<p>This paper counts servral methods to train a neural network.</p>
<ul>
<li>Gradient descent</li>
<li>Quasi-Newton</li>
<li>UKF parameter estimation</li>
</ul>
<p><strong>So my questions for you are:</strong></p>
<ol>
<li><p>What's the benefit for using a kalman filter for training a neural network compared to other optimization algorithms?</p>
</li>
<li><p>I remember that I have been using gradient descent methods and they work, but I have always used regularization. Is that due to the noise in the data?</p>
</li>
<li><p>If I'm using a UKF-filter as parameter estimation, I can avoid regularization then?</p>
</li>
<li><p>Is this UKF parameter estimation algorithm only made for 1 layer neural network, or can it be used for deep neural networks as well?</p>
</li>
</ol>

CC BY-SA 4.0
2021-09-14T21:53:17.000Z
Hello, asking multiple questions in a single post is not recommended. Please try to split the questions into multiple posts.
hanugm
CC BY-SA 4.0
2021-09-15T22:41:22.227Z
@hanugm Yes! I created a new question with only 1 question. https://ai.stackexchange.com/questions/31701/using-parameter-estimation-for-training-a-neural-network
euraad
<neural-networks><training><kalman-filter>
What's the benefit for using a Kalman filter for training a neural network compared to other optimization algorithms?
euraad
# What's the benefit for using a Kalman filter for training a neural network compared to other optimization algorithms?

(Asked by: username_0 on 2021-09-14)

I found a [paper](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.1421&rep=rep1&type=pdf) about using an Unscented Kalman Filter(UKF) for traning a neural network.

The UKF filter is modified so it works for parameter estimation. Assume that we have a neural network model $\hat d\_k = G(x\_k, W\_k)$ where $G$ is the neural network, $x\_k$ is the input **vector**, $W\_k$ is the parameter gain **matrix** and $\hat d\_k$ is the output **vector**.

This paper counts servral methods to train a neural network.

* Gradient descent
* Quasi-Newton
* UKF parameter estimation

**So my questions for you are:**

1. What's the benefit for using a kalman filter for training a neural network compared to other optimization algorithms?
2. I remember that I have been using gradient descent methods and they work, but I have always used regularization. Is that due to the noise in the data?
3. If I'm using a UKF-filter as parameter estimation, I can avoid regularization then?
4. Is this UKF parameter estimation algorithm only made for 1 layer neural network, or can it be used for deep neural networks as well?

**Tags:** neural-networks, training, kalman-filter

> **Comments:**
> **username_1**: Hello, asking multiple questions in a single post is not recommended. Please try to split the questions into multiple posts.  
> **username_0**: @username_1 Yes! I created a new question with only 1 question. https://ai.stackexchange.com/questions/31701/using-parameter-estimation-for-training-a-neural-network  

---

## 0 answers

