Answers.Body: <p>Batch Normalization is achieved through a normalization step that fixes the means and variances of each layer's inputs. For a Batch of size N, computing the mean and variance will have a complexity of O(N) per layer. Actual normalization can be considered an operation of O(1) with parallelism or vectorization (even without that, re-centring and re-scaling will be O(N) operations.)</p>
  <p>Reference: <a href="https://en.wikipedia.org/wiki/Batch_normalization" rel="nofollow noreferrer">Wikipedia</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-28T08:17:49.047Z
Answers.Text: thank you. since I am not good enough at math could you please let me  know what does O(1) and O(N) mean? is it possible to share the answer with an example?
Answers.UserDisplayName: code_lover
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-29T12:00:06.710Z
Answers.Text: Those refer to Big O notation. From Wikipedia: "In computer science, big O notation is used to classify algorithms according to how their run time or space requirements grow as the input size grows." Please note that, out of convention, constant factors are disregarded in Big O notation. So, for instance, computer scientists will consider O(k*N) or O(N/2) both as O(N).
Answers.UserDisplayName: postnubilaphoebus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-28T00:18:42.460Z
Answers.LastActivityDate: 2022-11-28T00:18:42.460Z
Answers.UserDisplayName: Naman Rajput
Body: <p>Could you please let me know weather it is possible to have a computational complexity formula for batch normalization technique or not? If someone can help me in this regard I will be appreciated.</p>
  
Tags: <batch-normalization><computational-complexity>
Title: computational complexity for batch normalization technique
UserDisplayName: code_lover
ThreadText: # computational complexity for batch normalization technique
  
  (Asked by: username_0 on 2022-11-27)
  
  Could you please let me know weather it is possible to have a computational complexity formula for batch normalization technique or not? If someone can help me in this regard I will be appreciated.
  
  **Tags:** batch-normalization, computational-complexity
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-11-28. Score: 1
  
  Batch Normalization is achieved through a normalization step that fixes the means and variances of each layer's inputs. For a Batch of size N, computing the mean and variance will have a complexity of O(N) per layer. Actual normalization can be considered an operation of O(1) with parallelism or vectorization (even without that, re-centring and re-scaling will be O(N) operations.)
  
  Reference: [Wikipedia](https://en.wikipedia.org/wiki/Batch_normalization)
  
  > **Comments:**
  > **username_0**: thank you. since I am not good enough at math could you please let me  know what does O(1) and O(N) mean? is it possible to share the answer with an example?  
  > **username_2**: Those refer to Big O notation. From Wikipedia: "In computer science, big O notation is used to classify algorithms according to how their run time or space requirements grow as the input size grows." Please note that, out of convention, constant factors are disregarded in Big O notation. So, for instance, computer scientists will consider O(k*N) or O(N/2) both as O(N).  
  
  ---
  
  