<p>Yes, you can do that, and it is a standard practice. One famous example is the <a href="https://arxiv.org/abs/1409.4842" rel="nofollow noreferrer">&quot;Inception&quot; network architecture</a>. To keep inner subnets from &quot;dying out&quot;,  several outputs from inner layers are extracted and passed into FC-&gt;Softmax. Then all the outputs are averaged in the loss function.</p>
<p>From practical point of view, you won't be able to implement such things with basic <code>Sequential</code> model construction. Most libraries allow you to move beyond it, though, and this distinction is usually well-documented. For example in the <a href="https://www.tensorflow.org/guide/keras/functional" rel="nofollow noreferrer">tensorflow guide on functional API</a> it states right in the introduction:</p>
<blockquote>
<p>The functional API is a way to create models that are more flexible than the <code>tf.keras.Sequential</code> API. The functional API can handle models with non-linear topology, shared layers, and even multiple inputs or outputs.</p>
</blockquote>
<p>Hope that'll help you to get started.</p>

CC BY-SA 4.0
2021-04-11T17:46:54.307Z
2021-04-11T17:46:54.307Z
Kostya
<p>For a network of the form:</p>
<pre><code>Input(10)
Dense(200)
Dense(100+10)
Dense(20)
Output()
</code></pre>
<p>Those <code>+10</code> outputs are what I want to add to the standard <code>20</code> outputs, for my loss function.</p>
<p>Is this possible - in theory or even with some pre-existing library?</p>

<neural-networks><tensorflow><keras><objective-functions><generative-adversarial-networks>
Is it possible to use an internal layer's outputs in a loss function?
Tobi Akinyemi
# Is it possible to use an internal layer's outputs in a loss function?

(Asked by: username_0 on 2021-04-10)

For a network of the form:
Input(10)
Dense(200)
Dense(100+10)
Dense(20)
Output()

Those +10 outputs are what I want to add to the standard 20 outputs, for my loss function.
Is this possible - in theory or even with some pre-existing library?

**Tags:** neural-networks, tensorflow, keras, objective-functions, generative-adversarial-networks


---

## 1 answers

### Answer by username_1 on 2021-04-11. Score: 2

Yes, you can do that, and it is a standard practice. One famous example is the ["Inception" network architecture](https://arxiv.org/abs/1409.4842). To keep inner subnets from "dying out", several outputs from inner layers are extracted and passed into FC->Softmax. Then all the outputs are averaged in the loss function.

From practical point of view, you won't be able to implement such things with basic `Sequential` model construction. Most libraries allow you to move beyond it, though, and this distinction is usually well-documented. For example in the [tensorflow guide on functional API](https://www.tensorflow.org/guide/keras/functional) it states right in the introduction:

> The functional API is a way to create models that are more flexible than the `tf.keras.Sequential` API. The functional API can handle models with non-linear topology, shared layers, and even multiple inputs or outputs.

Hope that'll help you to get started.


---

