Neural networks - what is the point of having sigmoid activation function $\sigma(.)$ AND sigmoid g(t)? Just so we're all on the same page, this is the classic neural network set up as I understand it:
$$z_j=\sigma(\alpha_0j+\alpha_j^T x)$$
$$t=\beta_0+\Sigma^k_{j=1}\beta_j z_j)=\beta_0+\beta^Tx $$
$$ y=f(x)=g(t)  $$
(I may have missed a few bits off like the hat on the y and some bold to show the vectors, but you get the idea)
$\sigma(.)$ is the activation function which can be a sigmoid function (can also be tanh, ReLu or threshold), but then in the final line there is another sigmoid function in g(t).
What is the logic in having two sigmoid functions? I get that this is what happens, but haven't yet seen any justification for the setup like this.
Please dumb it down as much as possible for me. I do not understand the meaning of the word "patronise".
Sometimes it feels like neural networks are just dumping input into a massive bag of arbitrary linear algebra, stirring it around, and hoping the output is something that you expect it to be...

The sigmoid functions in the hidden layers introduce nonlinearity. That is, they bend the output and let output values increase and then decrease and then increase again (or whatever), behavior that is not possible with a linear relationship.
The sigmoid function on the output neuron compresses the final value into the interval $(0,1)$. This is often (but not necessarily) to give an output that is a probability in a so-called "classification" problem that has a binary output category (e.g., dog or cat) and gives the probability of being in either category. If you know or someday learn about generalized linear models, this is related to the link function. I once posted about this on the AI Stack.
That the function in the hidden neurons and the function in the output neuron can be the same is more coincidence than anything, and they do not have to be the same.
