Can a neural network with only $1$ hidden layer solve any problem? Can a network with a single hidden layer of $N$ neurons (where $N\le\infty$) approximate any arbitrary function so that this network’s error approaches $0$ as $N$ approaches $\infty$?

The questions as stated ask "Can a neural network solve and problem?" and "Can a neural network approximate any arbitrary function?" The answer to both of these questions is "No, there is no such theorem."
However, it's common for people learning about neural networks for the first time to mis-state the so-called "universal approximation theorems," which provide the specific technical conditions under which a neural network can approximate a function. OP's questions appear to allude to some version of the Cybenko UAT. Here's a statement of the Cybenko UAT from Wikipedia.

Fix a continuous function $\sigma:\mathbb{R}\rightarrow \mathbb{R}$(activation function) and positive integers $d,D$.  The function $\sigma$ is not a polynomial if and only if, for every continuous function function $f:\mathbb{R}^d\to\mathbb{R}^D$ (target function), every compact subset $K$ of $\mathbb{R}^d$, and every $\epsilon >0 $ there exists a continuous function $f_{\epsilon}:\mathbb{R}^d\to\mathbb{R}^D$ (the layer output) with representation: $$
  f_{\epsilon} = W_2 \circ \sigma \circ  W_1,
$$
where $W_2,W_1$ are composable affine maps and $\circ$ denotes component-wise composition, such that the approximation bound:
$$
 \sup_{x \in K}\, \| f( x ) - f_{\epsilon} ( x ) \| < \varepsilon
$$
holds for any $\epsilon$ arbitrarily small (distance from $f$ to $f_\epsilon$ can be infinitely small).

So it's not that any function can be approximated -- the function must be continuous. The error is only smaller than $\epsilon$ on a certain compact subset $K$, and for some unspecified $D$.
In a practical setting where you have some data and you'd like to estimate a neural network, the Cybenko UAT is silent -- it doesn't tell you how to go about estimating $W_2, W_1$, nor does it tell you how to choose the width $D$ of the hidden layer, to achieve this precision.
And finally, achieving small approximation error to certain kinds of functions is not the same as "solving any problem."
(There are a number of approximation theorems for NNs, which reach different conclusions (but are similar in theme) and have different hypotheses.)
