Answers.Body: <p>Yes, for a broad class, they actually do, with probability 1:</p>
  <p>[i] Wang, Yanzhi, et al. &quot;Universal approximation property and equivalence of stochastic computing-based neural networks and binary neural networks.&quot; Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019 <a href="https://arxiv.org/abs/1803.05391" rel="nofollow noreferrer">https://arxiv.org/abs/1803.05391</a></p>
  <p>[ii] Yayla, Mikail, et al. &quot;Universal Approximation Theorems of Fully Connected Binarized Neural Networks.&quot; <a href="https://arxiv.org/abs/2102.02631" rel="nofollow noreferrer">arXiv preprint arXiv:2102.02631 (2021)</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-08T20:52:13.910Z
Answers.LastActivityDate: 2023-03-08T20:52:13.910Z
Answers.UserDisplayName: Tolga Birdal
Body: <p>Since the Universal approximation theorem shows that standard multilayer feedforward networks with as few as a single hidden layer, sufficient hidden units, and arbitrary bounded and nonconstant activation function can approximate any continuous function with arbitrary accuracy, does this theory also apply to binary neural networks? Is there a specific mathematical proof that binary neural networks can fit any function.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-12T13:01:32.057Z
Comments.Text: What do you mean by binary here.  Do you mean a single output neuron with a sigmoid activation?  Do you mean a two neuron output with some undefined activation?
Comments.UserDisplayName: David Hoelzer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-15T08:25:18.370Z
Comments.Text: @DavidHoelzer Binary neural network is a neural network after the extreme quantification of the general neural network, and the weight and activation in the network are only represented by 1bit.
Comments.UserDisplayName: user68072
Tags: <neural-networks><math><proofs><function-approximation><universal-approximation-theorems>
Title: Is there a mathematical proof of the universal approximation theorem for neural networks with binary weights?
UserDisplayName: user68072
ThreadText: # Is there a mathematical proof of the universal approximation theorem for neural networks with binary weights?
  
  (Asked by: username_0 on 2023-02-12)
  
  Since the Universal approximation theorem shows that standard multilayer feedforward networks with as few as a single hidden layer, sufficient hidden units, and arbitrary bounded and nonconstant activation function can approximate any continuous function with arbitrary accuracy, does this theory also apply to binary neural networks? Is there a specific mathematical proof that binary neural networks can fit any function.
  
  **Tags:** neural-networks, math, proofs, function-approximation, universal-approximation-theorems
  
  > **Comments:**
  > **username_1**: What do you mean by binary here.  Do you mean a single output neuron with a sigmoid activation?  Do you mean a two neuron output with some undefined activation?  
  > **username_0**: @username_1 Binary neural network is a neural network after the extreme quantification of the general neural network, and the weight and activation in the network are only represented by 1bit.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-03-08. Score: 0 | Accepted answer
  
  Yes, for a broad class, they actually do, with probability 1:
  
  [i] Wang, Yanzhi, et al. "Universal approximation property and equivalence of stochastic computing-based neural networks and binary neural networks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019 <https://arxiv.org/abs/1803.05391>
  
  [ii] Yayla, Mikail, et al. "Universal Approximation Theorems of Fully Connected Binarized Neural Networks." [arXiv preprint arXiv:2102.02631 (2021)](https://arxiv.org/abs/2102.02631).
  
  
  ---
  
  