$\sin(x)$ is a counterexample to the universal approximation theorem Inspired by AnoE's answer to the problem Can a neural network with only 1 hidden layer solve any problem?, I want to rigorously prove that the sine function over the real line cannot be well approximated by a shallow neural network. So we want to show that
$$ \sup_{x\in \mathbb{R}} |f(x) - \sin(x)| \geq 1$$
for any, say, neural network $f$ with 1 hidden layer and ReLU activation function.
Can you give me some ideas on how to approach this problem?

The classical (Cybenko) universal approximation theorem has a condition about the function being approximated on a compact space.
On the real line, the Heine-Borel theorem says that compacts sets are the closed and bounded sets.
Therefore, the Cybenko universal approximation theorem does not apply to a function over the whole real line. If you approximate $\sin(x)$ over a compact space, such as a closed interval, then the theorem holds.
