Answers.Body: <p>There no problem with the use of the data in form of an array to classify, whether the audio belongs to a real or fake voice. Just use <code>1d</code> convolutional neural network with downsamplings or some global pooling operations, such that in the final layer the temporal extent of the signal has length 1. This would be the logit for binary classification.</p>
  <p>However, as far as I understand, you get rid of phase after the Fourier transform, but it can be useful for the prediction. Probably, a better approach would be to use <code>mel_spectrogram</code> <a href="https://en.wikipedia.org/wiki/Mel-frequency_cepstrum" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Mel-frequency_cepstrum</a> for this problem.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-09T11:12:47.173Z
Answers.Text: Thanks! Wouldn't using a 1D CNN require time-series data? As you said, at this point we only have the frequencies vs. amplitude left.
  
  We're aware that using mel spectrograms could(possibly) perform better in such a CNN, but we were wondering if other methods could also work while making sense (so if it makes sense to use graph-like images for a CNN, or if there's some factor(s) as to why not)
Answers.UserDisplayName: CatSpaghetti
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-09T10:47:53.157Z
Answers.Text: A 1D CNN can be used, whenever you want to detect local patterns in one dimensional data. This can be used for time series but also in your example. 2D CNNs are used for images, and you basically have an image with one dimension, where the amplitude describes the pixel value. Therefore 1D CNN would be a good fit I think. You could also try working with recurrent cells (LSTM or GRU), but this will probably be much slower, and might not even work better.
Answers.UserDisplayName: Evator
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-08T15:50:46.940Z
Answers.LastActivityDate: 2021-06-08T15:50:46.940Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Body: <p>We want to try and distinguish real voices from (deep)fake voices using the graphs generated by a discrete fourier transform (generated from .wav audio files). We know from each image if it is a real or a fake voice, so it's a supervised classification problem. An image would look like this:</p>
  <p><a href="https://i.sstatic.net/vbqjJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vbqjJ.png" alt="enter image description here" /></a></p>
  <p>We think that real voices generate a graph with clear spikes, whereas fake voices have more noise resulting in less clear spikes. For this reason, we thought of using a CNN to take such an image as input (with x and y-axes ommited), and classify it as real or fake. Our concern is that it's actually a graph and not an image of an object, so we're not sure if this would be a good approach. We could also use the arrays generated from the fourier transform, but we're not sure how we could use that as input as we want to classify if it's real or fake, and not predict y for each x.</p>
  
Tags: <neural-networks><machine-learning>
Title: Can you use a graph as input for a neural network?
UserDisplayName: CatSpaghetti
ThreadText: # Can you use a graph as input for a neural network?
  
  (Asked by: username_0 on 2021-06-08)
  
  We want to try and distinguish real voices from (deep)fake voices using the graphs generated by a discrete fourier transform (generated from .wav audio files). We know from each image if it is a real or a fake voice, so it's a supervised classification problem. An image would look like this:
  
  [![enter image description here](https://i.sstatic.net/vbqjJ.png)](https://i.sstatic.net/vbqjJ.png)
  
  We think that real voices generate a graph with clear spikes, whereas fake voices have more noise resulting in less clear spikes. For this reason, we thought of using a CNN to take such an image as input (with x and y-axes ommited), and classify it as real or fake. Our concern is that it's actually a graph and not an image of an object, so we're not sure if this would be a good approach. We could also use the arrays generated from the fourier transform, but we're not sure how we could use that as input as we want to classify if it's real or fake, and not predict y for each x.
  
  **Tags:** neural-networks, machine-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-06-08. Score: 1
  
  There no problem with the use of the data in form of an array to classify, whether the audio belongs to a real or fake voice. Just use `1d` convolutional neural network with downsamplings or some global pooling operations, such that in the final layer the temporal extent of the signal has length 1. This would be the logit for binary classification.
  
  However, as far as I understand, you get rid of phase after the Fourier transform, but it can be useful for the prediction. Probably, a better approach would be to use `mel_spectrogram` <https://en.wikipedia.org/wiki/Mel-frequency_cepstrum> for this problem.
  
  > **Comments:**
  > **username_0**: Thanks! Wouldn't using a 1D CNN require time-series data? As you said, at this point we only have the frequencies vs. amplitude left.  We're aware that using mel spectrograms could(possibly) perform better in such a CNN, but we were wondering if other methods could also work while making sense (so if it makes sense to use graph-like images for a CNN, or if there's some factor(s) as to why not)  
  > **username_2**: A 1D CNN can be used, whenever you want to detect local patterns in one dimensional data. This can be used for time series but also in your example. 2D CNNs are used for images, and you basically have an image with one dimension, where the amplitude describes the pixel value. Therefore 1D CNN would be a good fit I think. You could also try working with recurrent cells (LSTM or GRU), but this will probably be much slower, and might not even work better.  
  
  ---
  
  