A new network with super approximation power is introduced. This network is built with Floor (\lfloor x\rfloor) and ReLU (\max\{0,x\}) activation functions and hence we call such networks as Floor-ReLU networks. It is shown by construction^+ and L\inN^+,}}  that Floor-ReLU networks with width \max\{d,\, 5N+13\} and depth 64dL+3 can pointwise approximate a Lipschitz continuous } function f on [0,1]^d with an exponential approximation rate 3 \mu\sqrt{d\,N}  ^{-  \sqrt{L}}, where \mu is the Lipschitz constantof f. {  More generally for an arbitrary continuous function f on [0,1]^d with a modulus of continuity \omega_f(\cdot), the constructive approximation rate is \omega_f(\sqrt{d}\,N^{-\sqrt{L}})+2\omega_f(\sqrt{d}){N^{-\sqrt{L}}}. As a consequence, this new network overcomes  the curse of dimensionality in approximation power since this approximation order is\le  } essentially \sqrt{d} times a function of N and L independent of d .
A new network with super approximation power is introduced. This network is built with Floor (\lfloor x\rfloor) or ReLU (\max\{0,x\}) activation function in each neuron and hence we call such networks as Floor-ReLU networks. For any hyper-parameters N\in\mathbb{N^+ and L\inN^+,}} it is shown that Floor-ReLU networks with width \max\{d,\, 5N+13\} and depth 64dL+3 can uniformly approximate a H \"o}lder function f on [0,1]^d with an approximation rate 3 \,N} \lambda d^{\alpha/2 ^{- \alpha \sqrt{L}}, where \alpha \in(0,1 and \lambda are the H{\"o lder order and constant, respectively. More generally for an arbitrary continuous function f on [0,1]^d with a modulus of continuity \omega_f(\cdot), the constructive approximation rate is \omega_f(\sqrt{d}\,N^{-\sqrt{L}})+2\omega_f(\sqrt{d}){N^{-\sqrt{L}}}. As a consequence, this new class of networks overcome the curse of dimensionality in approximation power when the variation of \omega_f(r) as r\rightarrow 0 is moderate (e.g., \omega_f(r)\le r^\alpha for H \"o}lder continuous functions), since the major term to be concerned in our approximation rate is essentially \sqrt{d} times a function of N and L independent of d within the modulus of continuity.
R
and
or
R
functions
function in each neuron
R
It is shown by construction
For any hyper-parameters N\in\mathbb{N
A
it is shown
R
pointwise approximate a Lipschitz continuous
uniformly approximate a H
A
\"o
A
lder
D
exponential
D
\mu\sqrt{d
A
\lambda d^{\alpha/2
A
\alpha
R
\mu is the Lipschitz constantof f.
\alpha \in(0,1
A
and \lambda are the H
A
\"o
A
lder order and constant, respectively.
R
network overcomes
class of networks
A
overcome
R
since this approximation order is
when the variation of \omega_f(r) as r\rightarrow 0 is moderate (e.g., \omega_f(r)
A
r^\alpha for H
A
\"o
A
lder continuous functions), since the major term to be concerned in our approximation rate is
R
.
within the modulus of continuity.