Answers.Body: <p>Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers).</p>
  <p><strong>For CNNs</strong> all the standard convolutional layers - <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1D" rel="nofollow noreferrer">Conv1D</a>, <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D" rel="nofollow noreferrer">Conv2D</a> and <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv3D" rel="nofollow noreferrer">Conv3D</a>,- have the <code>padding</code> argument. The padding values can be <code>valid</code> or <code>same</code> for 2d and 3d convolutions. And extra <code>causal</code> type of padding is possible for 1d convolutions and the documentation refers to this paper: <em><a href="https://arxiv.org/abs/1609.03499" rel="nofollow noreferrer">WaveNet: A Generative Model for Raw Audio</a></em> -  which sounds quite close to what you are interested in.</p>
  <p>This <a href="https://github.com/vdumoulin/conv_arithmetic" rel="nofollow noreferrer">animations</a> might be useful to get a bit more intuition about the convolutions and strides/padding. The general consensus is that using <code>same</code> padding is advantageous for model performance - your network gets more information about the borders of your inputs (and deeper networks are possible).</p>
  <p><strong>For sequential models</strong> padding is even more important. Training samples are usually  of unequal lengths, so you have to pad them with a special token (usually called <code>[PAD]</code> that gets encoded as 0). Here are some examples of this mentioned in <a href="https://www.tensorflow.org/guide/keras/masking_and_padding" rel="nofollow noreferrer">tensorflow docs</a>, <a href="https://huggingface.co/transformers/preprocessing.html#everything-you-always-wanted-to-know-about-padding-and-truncation" rel="nofollow noreferrer">huggingface.transformers</a> or <a href="https://albertauyeung.github.io/2020/06/19/bert-tokenization.html#padding-token-pad" rel="nofollow noreferrer">BERT tutorial</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-21T14:13:03.903Z
Answers.LastActivityDate: 2021-04-21T14:13:03.903Z
Answers.UserDisplayName: Kostya
Body: <p>Is padding, before feature extraction with VGGish, a good practice?</p>
  <p>Our padding technique is to find the longest signal (which is loaded <code>.wav</code> signal), and then, in every shorter signal, put zeros to the size of the longest one. We need to use it because one size of input data is desirable.</p>
  <p>Perhaps there is any other techniques you recommend?</p>
  <p>The difference between padding before and after the features extraction by accuracy is quite big - more than 20%. Using padding before extraction gives 97% accuracy.</p>
  <p>I'd be glad to read your feedback, and explain me why that happens, and tell me if that kind of padding is correct action or is there a better solution.</p>
  
Tags: <feature-extraction><accuracy><signal-processing><feature-engineering><padding>
Title: Is it a good practice to pad signal before feature extraction?
UserDisplayName: Dawid_K
ThreadText: # Is it a good practice to pad signal before feature extraction?
  
  (Asked by: username_0 on 2021-04-21)
  
  Is padding, before feature extraction with VGGish, a good practice?
  
  Our padding technique is to find the longest signal (which is loaded `.wav` signal), and then, in every shorter signal, put zeros to the size of the longest one. We need to use it because one size of input data is desirable.
  
  Perhaps there is any other techniques you recommend?
  
  The difference between padding before and after the features extraction by accuracy is quite big - more than 20%. Using padding before extraction gives 97% accuracy.
  
  I'd be glad to read your feedback, and explain me why that happens, and tell me if that kind of padding is correct action or is there a better solution.
  
  **Tags:** feature-extraction, accuracy, signal-processing, feature-engineering, padding
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-04-21. Score: 1 | Accepted answer
  
  Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers).
  
  **For CNNs** all the standard convolutional layers - [Conv1D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1D), [Conv2D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) and [Conv3D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv3D),- have the `padding` argument. The padding values can be `valid` or `same` for 2d and 3d convolutions. And extra `causal` type of padding is possible for 1d convolutions and the documentation refers to this paper: *[WaveNet: A Generative Model for Raw Audio](https://arxiv.org/abs/1609.03499)* - which sounds quite close to what you are interested in.
  
  This [animations](https://github.com/vdumoulin/conv_arithmetic) might be useful to get a bit more intuition about the convolutions and strides/padding. The general consensus is that using `same` padding is advantageous for model performance - your network gets more information about the borders of your inputs (and deeper networks are possible).
  
  **For sequential models** padding is even more important. Training samples are usually of unequal lengths, so you have to pad them with a special token (usually called `[PAD]` that gets encoded as 0). Here are some examples of this mentioned in [tensorflow docs](https://www.tensorflow.org/guide/keras/masking_and_padding), [huggingface.transformers](https://huggingface.co/transformers/preprocessing.html#everything-you-always-wanted-to-know-about-padding-and-truncation) or [BERT tutorial](https://albertauyeung.github.io/2020/06/19/bert-tokenization.html#padding-token-pad).
  
  
  ---
  
  