<p>If I understand correctly, your question is whether it's possible to randomly initialize some layers in a neural network and not apply gradient descent to them.</p>
<p>I believe the answer is yes. In synthesizer, they use a random matrix in the &quot;attention layer&quot; and did not train them, and still achieved reasonable results. Though I'm not sure how &quot;optimal&quot; that is. See here:
<a href="https://arxiv.org/abs/2005.00743" rel="nofollow noreferrer">https://arxiv.org/abs/2005.00743</a></p>
<p>I'm sure this applies to other networks (e.g. CNNs) as well. Is there a particular application you have in mind for this?</p>

CC BY-SA 4.0
2022-08-27T21:17:14.447Z
As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Community
CC BY-SA 4.0
2022-08-27T20:04:14.363Z
2022-08-27T20:04:14.363Z
PeaBrane
<p>Is it possible to reach the optimal values ​​for the parameters by not applying the decreasing gradient in some layers?</p>

CC BY-SA 4.0
2022-08-27T00:51:14.710Z
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
Community
CC BY-SA 4.0
2023-09-22T00:16:23.677Z
How will you set initial value of the kernel in the layer?
Cloud Cho
<neural-networks><deep-learning>
Reach optimal values by not decreasing gradient
Faeze
# Reach optimal values by not decreasing gradient

(Asked by: username_0 on 2022-08-27)

Is it possible to reach the optimal values ​​for the parameters by not applying the decreasing gradient in some layers?

**Tags:** neural-networks, deep-learning

> **Comments:**
> **username_1**: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.  
> **username_2**: How will you set initial value of the kernel in the layer?  

---

## 1 answers

### Answer by username_3 on 2022-08-27. Score: 1

If I understand correctly, your question is whether it's possible to randomly initialize some layers in a neural network and not apply gradient descent to them.

I believe the answer is yes. In synthesizer, they use a random matrix in the "attention layer" and did not train them, and still achieved reasonable results. Though I'm not sure how "optimal" that is. See here:
<https://arxiv.org/abs/2005.00743>

I'm sure this applies to other networks (e.g. CNNs) as well. Is there a particular application you have in mind for this?

> **Comments:**
> **username_1**: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  

---

