<p>I ended up working with a neural network <span class="math-container">$N(\cdot)$</span> characterized by the following structure:<span class="math-container">$$N(x)=V[\text{ReLU}(Ax+b)+Cx+d],$$</span>
with input <span class="math-container">$x \in \mathbb{R}^n$</span> and parameters <span class="math-container">$V\in\mathbb{R}^{1\times m}$</span>, <span class="math-container">$A$</span>, <span class="math-container">$C \in \mathbb{R}^{m \times n}$</span> and <span class="math-container">$b$</span>, <span class="math-container">$d \in \mathbb{R}^m$</span>. Does anyone know what sort of &quot;standard&quot; architecture it represents?</p>

CC BY-SA 4.0
2023-07-18T16:13:16.500Z
Are there any constraints on the parameters? Otherwise, if $A$ and $b$ can vary freely, then $A-C$ and $b-d$ look like they don't do much, because neither $A$ nor $b$ appear anywhere else, which would allow $A-C$ and $b-d$ to be arbitrary with no loss of generalisation if you simply dropped the subtractions.
Neil Slater
CC BY-SA 4.0
2023-07-18T16:56:15.710Z
@NeilSlater You're right - just edited, accordingly.
GuidoLaremi
<neural-networks><machine-learning><deep-learning>
A neural network with peculiar architecture?
GuidoLaremi
# A neural network with peculiar architecture?

(Asked by: username_0 on 2023-07-18)

I ended up working with a neural network $N(\cdot)$ characterized by the following structure:$$N(x)=V[\text{ReLU}(Ax+b)+Cx+d],$$
with input $x \in \mathbb{R}^n$ and parameters $V\in\mathbb{R}^{1\times m}$, $A$, $C \in \mathbb{R}^{m \times n}$ and $b$, $d \in \mathbb{R}^m$. Does anyone know what sort of "standard" architecture it represents?

**Tags:** neural-networks, machine-learning, deep-learning

> **Comments:**
> **username_1**: Are there any constraints on the parameters? Otherwise, if $A$ and $b$ can vary freely, then $A-C$ and $b-d$ look like they don't do much, because neither $A$ nor $b$ appear anywhere else, which would allow $A-C$ and $b-d$ to be arbitrary with no loss of generalisation if you simply dropped the subtractions.  
> **username_0**: @NeilSlater You're right - just edited, accordingly.  

---

## 0 answers

