Can a neural network with only $1$ hidden layer solve any problem? Can a network with a single hidden layer of $N$ neurons (where $N\le\infty$) approximate any arbitrary function so that this network’s error approaches $0$ as $N$ approaches $\infty$?

The canonical paper for this is Hornik, (1989), for a pdf.
The paper demonstrates that given a neural net with a single layer of sufficient number of nodes, where the activation function is a 'squashing' function, we may approximate any Borel measurable function to arbitrary accuracy.
NB. a net with ReLU activation functions does have this property, as we can make a squashing function from a pair of ReLU function https://papers.nips.cc/paper/2018/file/03bfc1d4783966c69cc6aef8247e0103-Paper.pdf
