<p>I want to use a hypernetwork on an entire vision backbone (39m parameters).</p>
<p>The hypernetwork structure looks like:</p>
<p>512 -&gt; 512 -&gt; 512 -&gt; 39m</p>
<p>Unfortunately, the last layer means the hypernetwork has billions of parameters.</p>
<p>What could I do to reduce the parameter count of the hypernetwork?</p>
<p>(The 512-dimensional input vector is a noise vector that acts like a key that describes what details to focus on).</p>

CC BY-SA 4.0
2024-10-28T21:24:24.783Z
why do you need a hypernetwork for such a large model? what kind of task are you trying to solve?
Alberto
CC BY-SA 4.0
2024-10-29T00:22:48.470Z
2 convnet encoders for multimodal embedding and 1 decoder for one of the modalities. The entire network is conditioned by a noise vector for embedding diversity under uncertainty.
Richie Bendall
CC BY-SA 4.0
2024-10-29T10:38:48.990Z
just pass the noise as part of the input
Alberto
<network-design>
Hypernetwork too many parameters problem
Richie Bendall
# Hypernetwork too many parameters problem

(Asked by: username_0 on 2024-10-28)

I want to use a hypernetwork on an entire vision backbone (39m parameters).

The hypernetwork structure looks like:

512 -> 512 -> 512 -> 39m

Unfortunately, the last layer means the hypernetwork has billions of parameters.

What could I do to reduce the parameter count of the hypernetwork?

(The 512-dimensional input vector is a noise vector that acts like a key that describes what details to focus on).

**Tags:** network-design

> **Comments:**
> **username_1**: why do you need a hypernetwork for such a large model? what kind of task are you trying to solve?  
> **username_0**: 2 convnet encoders for multimodal embedding and 1 decoder for one of the modalities. The entire network is conditioned by a noise vector for embedding diversity under uncertainty.  
> **username_1**: just pass the noise as part of the input  

---

## 0 answers

