Neural networks - what is the point of having sigmoid activation function $\sigma(.)$ AND sigmoid g(t)? Just so we're all on the same page, this is the classic neural network set up as I understand it:
$$z_j=\sigma(\alpha_0j+\alpha_j^T x)$$
$$t=\beta_0+\Sigma^k_{j=1}\beta_j z_j)=\beta_0+\beta^Tx $$
$$ y=f(x)=g(t)  $$
(I may have missed a few bits off like the hat on the y and some bold to show the vectors, but you get the idea)
$\sigma(.)$ is the activation function which can be a sigmoid function (can also be tanh, ReLu or threshold), but then in the final line there is another sigmoid function in g(t).
What is the logic in having two sigmoid functions? I get that this is what happens, but haven't yet seen any justification for the setup like this.
Please dumb it down as much as possible for me. I do not understand the meaning of the word "patronise".
Sometimes it feels like neural networks are just dumping input into a massive bag of arbitrary linear algebra, stirring it around, and hoping the output is something that you expect it to be...

The outline you've described sounds like a neural network with 1 hidden layer with an activation function $\sigma$ and 1 standard logistic function output neuron $g$ :
$$
\hat y = g(W_2 \sigma(W_1x+b_1)+b_2).
$$
where
$g(z) = \frac{1}{1+\exp(-z)} = \frac{\exp(z)}{1 + \exp(z)}$.
Some losses require that the model outputs a probability, such as the binary cross-entropy loss. Using the logistic function $g$ enforces that the prediction $\hat y$ is a probability, the sigmoid function gives real numbers between 0 and 1 (and sum to 1 across mutually exclusive outcomes), i.e. probabilities. The logistic function is not the only continuous, monotonic function that maps real numbers to numbers between 0 and 1 (and sum to 1 across mutually exclusive outcomes), it's just  a very commonly used function. For some other common examples, see Difference between logit and probit models.
If the loss we're using does not require that the predictions are probabilities, then we don't have to use $g$ as the output function. An alternative function, including the identity function, could be used.
The hidden layers use a nonlinear activation function $\sigma$ to allow the network to learn a nonlinear basis. See: What does the hidden layer in a neural network compute? If we use a linear activation function, such as the identity function, then the neural network is restricted to being a composition of linear operations, which is itself linear.
There's no requirement that $g$ and $\sigma$ are the same function. Modern neural network researchers will choose $g$ and $\sigma$ according to whatever they need the network to do, and it's more typical that $g$ and $\sigma$ will not have any requirement to be the same function.
