Do neural networks create a separating hyperplane? https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ mentions that neural networks learn a representation of the data so as to make the classes linearly separable. What I fail to see is how does a neural network create a separating hyperplane that separates the classes. How do we know that it creates a hyperplane? What's the math behind it?

I think I may have answered my question.....
Let $x$ be the last hidden layer before the output layer in the neural network
Say I have $n$ classes
Let the multi-class decision rule be simply the class with the highest output value
Mathematically, we classify $x$ as belonging to a class $i$ if $\forall$$j$:
$f(W_ix+b_i) > f(W_{j\neq i}x+b_j)$ where $f$ is a nonlinear activation.
We now solve for the values of $x$ which satisfy this decision rule (i.e. $\{x|decision(x) = i\}$):
If $f$ is monotonically increasing this implies $f(W_ix+b_i) > f(W_{j\neq i}x +b_j) \iff W_ix + b_i > W_{j\neq i}x+b_j$
bringing everything to one side gives:
$\implies (W_{i1}-W_{j1})x_1+(W_{i2}-W_{j2})x_2+(W_{i3}-W_{j3})x_3 +...+ (b_i -b_j) > 0$
so this decision rule does correspond to a separating hyperplane
Edit: it's a hyperplane for fixed $j$, but the inequality must hold for all $j$, so I guess it must be the region that's in common/intersection of all of the hyperplanes.....?
It would be interesting to obtain an explicit expression and solve for the set $x$ that satisfies a decision rule where instead of $x$ being the layer before the output layer, it instead would be the input to a multi-layer neural network
