<p>I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.</p>
<p>I was told that batch normalization layers might be trying to find weights that are good for all while training, which is a problem since the distribution of my synthetic data is not exactly equal to the distribution of the real data. So, the idea would be to have different 'copies' of the weights of batch normalization layers. So that the neural network estimates different weights for synthetic and real data, and uses just the weights of real data for evaluation.</p>
<p>My question is, how to perform batch normalization in the aforementioned case? Is there already an implementation of batch norm layers in PyTorch that solves the problem?</p>

CC BY-SA 4.0
2021-07-26T12:50:57.207Z
Does the second part of the question falls in to the category of **asking for an API, library, or dataset (to solve a specific problem).** which is off-topic? @nbro
hanugm
CC BY-SA 4.0
2021-07-26T13:00:08.180Z
I am interested in first part of the question.
hanugm
CC BY-SA 4.0
2021-07-26T13:00:24.457Z
Oh, in this case? Where should i post this question?
Manveru
CC BY-SA 4.0
2021-07-26T13:01:26.963Z
The question is nice. But i am not sure about the part  **Is there already an implementation of batch norm layers in PyTorch that solves the problem?** only
hanugm
CC BY-SA 4.0
2021-07-26T13:02:54.083Z
The question **what is the best and easiest way to implement that?** is on-topic since it is asking for reconciling batch norm over multiple distributions.
hanugm
CC BY-SA 4.0
2021-07-26T13:03:02.400Z
Oh I see. I would like to implement it, but I dont know enough pytorch to change it alone. That is why I would like some hints
Manveru
CC BY-SA 4.0
2021-07-26T13:07:28.887Z
You can remove the words **best** and **easiest** since they are superlative. Instead you can ask, **how to perform batch normalization in aforementioned case?**
hanugm
CC BY-SA 4.0
2021-07-26T13:18:00.743Z
@hanugm If "Is there already an implementation of batch norm layers in PyTorch that solves the problem?" is just a secondary/additional question, i.e. it's not the main question, and by "how to perform batch normalization in the aforementioned case?" the OP is asking, for example, for the pseudocode that would allow them to implement the approach to solve their problem, the question is fine (I think).
nbro
<deep-learning><datasets><pytorch><batch-normalization><data-augmentation>
Batch normalization for multiple datasets?
Manveru
# Batch normalization for multiple datasets?

(Asked by: username_0 on 2021-07-26)

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.

I was told that batch normalization layers might be trying to find weights that are good for all while training, which is a problem since the distribution of my synthetic data is not exactly equal to the distribution of the real data. So, the idea would be to have different 'copies' of the weights of batch normalization layers. So that the neural network estimates different weights for synthetic and real data, and uses just the weights of real data for evaluation.

My question is, how to perform batch normalization in the aforementioned case? Is there already an implementation of batch norm layers in PyTorch that solves the problem?

**Tags:** deep-learning, datasets, pytorch, batch-normalization, data-augmentation

> **Comments:**
> **username_1**: Does the second part of the question falls in to the category of **asking for an API, library, or dataset (to solve a specific problem).** which is off-topic? @username_2  
> **username_1**: I am interested in first part of the question.  
> **username_0**: Oh, in this case? Where should i post this question?  
> **username_1**: The question is nice. But i am not sure about the part  **Is there already an implementation of batch norm layers in PyTorch that solves the problem?** only  
> **username_1**: The question **what is the best and easiest way to implement that?** is on-topic since it is asking for reconciling batch norm over multiple distributions.  
> **username_0**: Oh I see. I would like to implement it, but I dont know enough pytorch to change it alone. That is why I would like some hints  
> **username_1**: You can remove the words **best** and **easiest** since they are superlative. Instead you can ask, **how to perform batch normalization in aforementioned case?**  
> **username_2**: @username_1 If "Is there already an implementation of batch norm layers in PyTorch that solves the problem?" is just a secondary/additional question, i.e. it's not the main question, and by "how to perform batch normalization in the aforementioned case?" the OP is asking, for example, for the pseudocode that would allow them to implement the approach to solve their problem, the question is fine (I think).  

---

## 0 answers

