Body: <p>Assume I am given a binary neural network where the activation values are constrained to be 0 or 1 (by clipping the ReLU function). Additionally, assume the neural network is supposed to work in a noisy environment where some of the activation values may be randomly flipped, i.e. 0 -> 1 or 1 -> 0.</p>
  
  <p>I am trying to train such neural network in a way that it would be resilient to the noisy environment. I assume training with dropout would make the neural network somewhat resilient to noises where one is flipped to zero (1 -> 0).</p>
  
  <p>What are some ways that allow me to make the neural network resilient to the other kind of noise which flips zeros to ones (0 -> 1)? Is it theoretically valid to introduce a dropout-like algorithm which flips some zeros to ones during training but does not backpropagate the gradients through those flipped nodes?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-16T21:59:33.783Z
Comments.Text: Welcome to SE:AI!  (Be advised, we typically prefer one question per post, but I'm going to allow it, as they are all related.)
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-16T22:48:07.940Z
Comments.Text: @DukeZhou Thanks. Yes, they are all somewhat related. I just used an enumerated list to allow easier reference to different parts of my question.
Comments.UserDisplayName: Matt
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-17T14:54:08.203Z
Comments.Text: @Mahdi I really suggest you edit your post and ask only **one question**. If a reader or user looks at your posts and sees many questions, he/she may be discouraged immediately, because it's already too much complexity. I suggest you focus on one question and problem here. Ask the other question in different posts.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-17T21:56:25.153Z
Comments.Text: @nbro I have updated the question.
Comments.UserDisplayName: Matt
Tags: <training><dropout>
Title: How to make binary neural networks resilient to flipped activation values?
UserDisplayName: Matt
ThreadText: # How to make binary neural networks resilient to flipped activation values?
  
  (Asked by: username_0 on 2020-04-16)
  
  Assume I am given a binary neural network where the activation values are constrained to be 0 or 1 (by clipping the ReLU function). Additionally, assume the neural network is supposed to work in a noisy environment where some of the activation values may be randomly flipped, i.e. 0 -> 1 or 1 -> 0.
  
  I am trying to train such neural network in a way that it would be resilient to the noisy environment. I assume training with dropout would make the neural network somewhat resilient to noises where one is flipped to zero (1 -> 0).
  
  What are some ways that allow me to make the neural network resilient to the other kind of noise which flips zeros to ones (0 -> 1)? Is it theoretically valid to introduce a dropout-like algorithm which flips some zeros to ones during training but does not backpropagate the gradients through those flipped nodes?
  
  **Tags:** training, dropout
  
  > **Comments:**
  > **username_1**: Welcome to SE:AI!  (Be advised, we typically prefer one question per post, but I'm going to allow it, as they are all related.)  
  > **username_0**: @username_1 Thanks. Yes, they are all somewhat related. I just used an enumerated list to allow easier reference to different parts of my question.  
  > **username_2**: @Mahdi I really suggest you edit your post and ask only **one question**. If a reader or user looks at your posts and sees many questions, he/she may be discouraged immediately, because it's already too much complexity. I suggest you focus on one question and problem here. Ask the other question in different posts.  
  > **username_0**: @username_2 I have updated the question.  
  
  ---
  
  ## 0 answers
  
  