<p>Assume I am given a binary neural network where the activation values are constrained to be 0 or 1 (by clipping the ReLU function). Additionally, assume the neural network is supposed to work in a noisy environment where some of the activation values may be randomly flipped, i.e. 0 -> 1 or 1 -> 0.</p>

<p>I am trying to train such neural network in a way that it would be resilient to the noisy environment. I assume training with dropout would make the neural network somewhat resilient to noises where one is flipped to zero (1 -> 0).</p>

<p>What are some ways that allow me to make the neural network resilient to the other kind of noise which flips zeros to ones (0 -> 1)? Is it theoretically valid to introduce a dropout-like algorithm which flips some zeros to ones during training but does not backpropagate the gradients through those flipped nodes?</p>

CC BY-SA 4.0
2020-04-16T21:59:33.783Z
Welcome to SE:AI!  (Be advised, we typically prefer one question per post, but I'm going to allow it, as they are all related.)
DukeZhou
CC BY-SA 4.0
2020-04-16T22:48:07.940Z
@DukeZhou Thanks. Yes, they are all somewhat related. I just used an enumerated list to allow easier reference to different parts of my question.
Matt
CC BY-SA 4.0
2020-04-17T14:54:08.203Z
@Mahdi I really suggest you edit your post and ask only **one question**. If a reader or user looks at your posts and sees many questions, he/she may be discouraged immediately, because it's already too much complexity. I suggest you focus on one question and problem here. Ask the other question in different posts.
nbro
CC BY-SA 4.0
2020-04-17T21:56:25.153Z
@nbro I have updated the question.
Matt
<training><dropout>
How to make binary neural networks resilient to flipped activation values?
Matt
# How to make binary neural networks resilient to flipped activation values?

(Asked by: username_0 on 2020-04-16)

Assume I am given a binary neural network where the activation values are constrained to be 0 or 1 (by clipping the ReLU function). Additionally, assume the neural network is supposed to work in a noisy environment where some of the activation values may be randomly flipped, i.e. 0 -> 1 or 1 -> 0.

I am trying to train such neural network in a way that it would be resilient to the noisy environment. I assume training with dropout would make the neural network somewhat resilient to noises where one is flipped to zero (1 -> 0).

What are some ways that allow me to make the neural network resilient to the other kind of noise which flips zeros to ones (0 -> 1)? Is it theoretically valid to introduce a dropout-like algorithm which flips some zeros to ones during training but does not backpropagate the gradients through those flipped nodes?

**Tags:** training, dropout

> **Comments:**
> **username_1**: Welcome to SE:AI!  (Be advised, we typically prefer one question per post, but I'm going to allow it, as they are all related.)  
> **username_0**: @username_1 Thanks. Yes, they are all somewhat related. I just used an enumerated list to allow easier reference to different parts of my question.  
> **username_2**: @Mahdi I really suggest you edit your post and ask only **one question**. If a reader or user looks at your posts and sees many questions, he/she may be discouraged immediately, because it's already too much complexity. I suggest you focus on one question and problem here. Ask the other question in different posts.  
> **username_0**: @username_2 I have updated the question.  

---

## 0 answers

