Answers.Body: <h2><strong>Try Rectification</strong></h2>
  <p>Improve the <strong>features</strong> available to your model, <strong>Remove</strong> some of the <strong>NOISE</strong> present in the data.</p>
  <ul>
  <li>In audio data, a common way to do this is to <strong>smooth</strong> the data and then <strong>rectify</strong> it so that the total amount of sound energy over time is more <strong>distinguishable</strong>.</li>
  </ul>
  <pre><code># Rectify the audio signal
  audio_rectified = audio.apply(np.abs)
  </code></pre>
  <ul>
  <li><p>You can also calculate the <strong>absolute value</strong> of each time point. This is also called <strong>Rectification</strong> because you ensure that all time points are positive.</p>
  </li>
  <li><p>Smooth your data by taking the <strong><code>rolling</code></strong> mean in a <strong>window</strong> of say 50 samples</p>
  </li>
  </ul>
  <pre><code>audio_rectified_smooth = audio_rectified.rolling(50).mean()
  </code></pre>
  <ul>
  <li>Calculating the <strong>envelope</strong> of each sound and smoothing it will eliminate much of the noise and you have a cleaner signal.</li>
  </ul>
  <h2><strong>Calculate Spectrogram</strong></h2>
  <ul>
  <li><p>Calculate a <strong>spectrogram</strong> of sound(i.e combining of windows Fourier transforms). This describes what <strong>spectral content</strong> (e.g., <em>low and high pitches</em>) are present in the sound over time. there is a lot more information in a spectrogram compared to a raw audio file. By computing the spectral features, you have a much better idea of what's going on.</p>
  <ul>
  <li><p>this is similar to how we calculate <strong>rolling mean</strong> :</p>
  <ul>
  <li><p>We calculate <strong>multiple Fourier transforms</strong> in a <strong>sliding window</strong> to see how it changes over time. For each time point, we take a window of time around it, calculate a Fourier transform for the window, then slide to the next window. The result is a description of the <strong>Fourier transform</strong> as it changes throughout the <strong>time-series</strong> called a short-time Fourier transform or STFT.</p>
  <ul>
  <li>Choose a windows <strong>size</strong> and <strong>shape</strong></li>
  <li>At a timepoint, calculate the <strong>FFT</strong> for that window</li>
  <li>Slide the window over by one</li>
  <li>Aggregate the results</li>
  </ul>
  </li>
  </ul>
  </li>
  <li><p><strong>Calculating the STFT</strong></p>
  <ul>
  <li>We can calculate the STFT with <code>librosa</code> <span class="math-container">$\rightarrow$</span> <code>import librosa as lr </code></li>
  <li>There are several parameters we can tweak (such as <strong>window size</strong>)</li>
  <li>For our purposes, we'll convert into <strong>decibels</strong> which <strong>normalizes</strong> the average values of all frequencies.</li>
  <li>We can then visualize it with the <code>specshow()</code> function</li>
  </ul>
  </li>
  </ul>
  </li>
  </ul>
  <p>This is how you can calculate the <strong>STFT</strong></p>
  <pre><code># Calculating the STFT
  # Import the functions we'll use for the STFT
  from librosa.core import stft, amplitude_to_db
  from librosa.display import specshow
  
  # Calculate our STFT
  HOP_LENGTH = 2**4
  SIZE_WINDOW = 2**7
  audio_spec = stft(audio, hop_length=HOP_LENGTH, n_fft=SIZE_WINDOW)
  
  # Convert into decibels for visualization
  spec_db = amplitude_to_db(audio_spec)
  
  # Visualize
  specshow(spec_db, sr=sfreq, x_axis='time', y_axis='hz', hop_length=HOP_LENGTH)
  </code></pre>
  <h2><strong>Try Spectral feature engineering</strong></h2>
  <p>you can also perform the Spectral feature engineering on your baby audio data</p>
  <ul>
  <li>since each time series has a different spectral pattern.</li>
  <li>We can calculate these spectral patterns by analyzing the spectrogram.</li>
  <li>For example, <strong>spectral bandwidth</strong> and <strong>spectrum centroids</strong> describe where most of the energy is at each moment in time.</li>
  </ul>
  <pre><code># Calculate the spectral centroid and bandwidth for the spectrogram
  bandwidths = lr.feature.spectral_bandwidth(S=spec)[0]
  centroids = lr.feature.spectral_centroid(S=spec)[0]
   
  # Display these features on top of the spectrogram
  ax = specshow(spec, x_axis='time', y_axis='hz', hop_length=HOP_LENGTH)
  ax.plot(times_spec, centroids)
  ax.fill_between(times_spec, centroids - bandwidths / 2,centroids + bandwidths / 2, alpha=0.5)
  </code></pre>
  <p>Now you can Combine spectral(<code>spec</code> <span class="math-container">$\rightarrow$</span> spectral dataframe) and temporal features in a classifier</p>
  <pre><code>centroids_all = []
  bandwidths_all = []
   
  for spec in spectrograms:
      bandwidths = lr.feature.spectral_bandwidth(S=lr.db_to_amplitude(spec))
      centroids = lr.feature.spectral_centroid(S=lr.db_to_amplitude(spec))
      # Calculate the mean spectral bandwidth
      bandwidths_all.append(np.mean(bandwidths))
      # Calculate the mean spectral centroid
      centroids_all.append(np.mean(centroids))
  # Create our X matrix
  X = np.column_stack([means, stds, maxs, tempo_mean, tempo_max, tempo_std, bandwidths_all, centroids_all])
  </code></pre>
  <h2><strong>for your logistic regression models</strong></h2>
  <ul>
  <li>One of the ways to improve accuracy is by <strong>optimizing the prediction probability cutoff</strong> scores generated by your logit model.</li>
  <li>You can <strong>Normalize</strong> all your features to the same scale before putting them in a machine learning model.</li>
  <li>Look for <strong>class imbalance</strong> in your data.</li>
  <li>You can <strong>Optimize on other Metrics</strong> also such as <strong>Log Loss</strong> and <strong>F1-Score</strong>.</li>
  <li>Tune the hyperparameters of your model. In the case of LogisticRegression, parameter <span class="math-container">$C$</span> is a hyperparameter.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-28T11:57:02.160Z
Answers.Text: thanks alot for giving your time :) the audio files have nearly no noise in it as they are specially recorded sounds from a mobile application of baby crying. i have tried normalizing the values in the array from the range of -1 to 1, TO 0 to 1. Moreover i have tried using ANN which uses spectrograms method and it gave me an accuracy of 100% after 100 epoches. The problem is that i cannot use spectrograms or MFCCs, kindly read my comments in the above question so you will have an idea. Well my data is imbalanced since 222 out of 581 audios are of baby crying. Thanks alot once again! :)
Answers.UserDisplayName: Muhammad Waqar Anwar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-26T14:24:56.450Z
Answers.Text: @MuhammadWaqarAnwar FFT is an efficient algorithm, and libraries for it are available for the arduino - see e.g. https://www.youtube.com/watch?v=djGOOL2mqV8
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-27T21:34:56.833Z
Answers.LastActivityDate: 2021-09-26T19:10:22.197Z
Answers.LastEditDate: 2021-09-26T19:10:22.197Z
Answers.UserDisplayName: Faizy
Body: <p>I am working on a <strong>Baby Crying Detection model</strong> using <strong>logistic regression</strong>.</p>
  <p>Out of <span class="math-container">$581$</span> audios, <span class="math-container">$222$</span> are of a baby crying. Each audio is of <span class="math-container">$5$</span> seconds.</p>
  <p>what I have done is convert each audio into numbers. and those numbers go into a <code>.csv</code> file. so first I took <span class="math-container">$100$</span> samples from each audio, then <span class="math-container">$1000$</span> samples, and then all <span class="math-container">$110250$</span> samples into a <code>.csv</code> file, and at the end of each of them was a number 1 (crying) or 0 (not crying). Then I trained the model using logistic regression from that <code>.csv</code> file.</p>
  <p>The <strong>Problem</strong> I m facing is that with <span class="math-container">$100$</span> samples the <strong>64% accuracy</strong> on each audio,  while with 1000 samples and 110250 samples(Full dataset) it reaches to <strong>66% accuracy</strong> only. <em>How can I improve the accuracy of my model to upto 80% using logistic regression.</em></p>
  <p>I can only use simple <strong>logistic regression</strong> because I have to deploy the model on <strong>Arduino</strong>.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-27T18:44:04.247Z
Comments.Text: How are you processing the raw audio samples "into numbers". For audio processing this is typically some variant of FFT power spectrums. MFCC has been popular for speech. Are you doing anything like that? Please explain more about the pre-processing before reaching the predictive model and [edit] those details into the question. In addition, is sampling triggered by anything (such as hitting a volume threshold)? Or is sampling done at fixed intervals?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-28T11:40:44.747Z
Comments.Text: Simply i use librosa library to get audio files into an array. then that array is pasted into a CSV file. I know about MFCC. lets say i pick 100 samples. I pick 100 samples at equal intervals, then those 100 samples per audio (100x581 table of total audios) go through logistic regression which gives output of 100 weights or coefficients. Then arduino takes 100 samples from live recording, these 100 samples and trained model 100 samples are multiplied one to one, then added, then sigmoid function is applied. if output value is greater than 0.5 it means baby is crying. my friend told this method
Comments.UserDisplayName: Muhammad Waqar Anwar
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-28T11:42:01.883Z
Comments.Text: i cannot use MFCC because i need to deploy the model on arduino and all the calculation on arduino needs 100 weights, while MFCC outputs just one value per audio
Comments.UserDisplayName: Muhammad Waqar Anwar
Tags: <regression><audio-processing><binary-classification><logistic-regression>
Title: How to get more accuracy of the logistic regression model?
UserDisplayName: Muhammad Waqar Anwar
ThreadText: # How to get more accuracy of the logistic regression model?
  
  (Asked by: username_0 on 2021-03-27)
  
  I am working on a **Baby Crying Detection model** using **logistic regression**.
  
  Out of $581$ audios, $222$ are of a baby crying. Each audio is of $5$ seconds.
  
  what I have done is convert each audio into numbers. and those numbers go into a `.csv` file. so first I took $100$ samples from each audio, then $1000$ samples, and then all $110250$ samples into a `.csv` file, and at the end of each of them was a number 1 (crying) or 0 (not crying). Then I trained the model using logistic regression from that `.csv` file.
  
  The **Problem** I m facing is that with $100$ samples the **64% accuracy** on each audio, while with 1000 samples and 110250 samples(Full dataset) it reaches to **66% accuracy** only. *How can I improve the accuracy of my model to upto 80% using logistic regression.*
  
  I can only use simple **logistic regression** because I have to deploy the model on **Arduino**.
  
  **Tags:** regression, audio-processing, binary-classification, logistic-regression
  
  > **Comments:**
  > **username_1**: How are you processing the raw audio samples "into numbers". For audio processing this is typically some variant of FFT power spectrums. MFCC has been popular for speech. Are you doing anything like that? Please explain more about the pre-processing before reaching the predictive model and [edit] those details into the question. In addition, is sampling triggered by anything (such as hitting a volume threshold)? Or is sampling done at fixed intervals?  
  > **username_0**: Simply i use librosa library to get audio files into an array. then that array is pasted into a CSV file. I know about MFCC. lets say i pick 100 samples. I pick 100 samples at equal intervals, then those 100 samples per audio (100x581 table of total audios) go through logistic regression which gives output of 100 weights or coefficients. Then arduino takes 100 samples from live recording, these 100 samples and trained model 100 samples are multiplied one to one, then added, then sigmoid function is applied. if output value is greater than 0.5 it means baby is crying. my friend told this method  
  > **username_0**: i cannot use MFCC because i need to deploy the model on arduino and all the calculation on arduino needs 100 weights, while MFCC outputs just one value per audio  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-03-27. Score: 2
  
  ## **Try Rectification**
  
  Improve the **features** available to your model, **Remove** some of the **NOISE** present in the data.
  
  * In audio data, a common way to do this is to **smooth** the data and then **rectify** it so that the total amount of sound energy over time is more **distinguishable**.
  
  ```
  # Rectify the audio signal
  audio_rectified = audio.apply(np.abs)
  
  ```
  
  * You can also calculate the **absolute value** of each time point. This is also called **Rectification** because you ensure that all time points are positive.
  * Smooth your data by taking the **`rolling`** mean in a **window** of say 50 samples
  
  ```
  audio_rectified_smooth = audio_rectified.rolling(50).mean()
  
  ```
  
  * Calculating the **envelope** of each sound and smoothing it will eliminate much of the noise and you have a cleaner signal.
  
  ## **Calculate Spectrogram**
  
  * Calculate a **spectrogram** of sound(i.e combining of windows Fourier transforms). This describes what **spectral content** (e.g., *low and high pitches*) are present in the sound over time. there is a lot more information in a spectrogram compared to a raw audio file. By computing the spectral features, you have a much better idea of what's going on.
  
    + this is similar to how we calculate **rolling mean** :
  
      - We calculate **multiple Fourier transforms** in a **sliding window** to see how it changes over time. For each time point, we take a window of time around it, calculate a Fourier transform for the window, then slide to the next window. The result is a description of the **Fourier transform** as it changes throughout the **time-series** called a short-time Fourier transform or STFT.
  
        * Choose a windows **size** and **shape**
        * At a timepoint, calculate the **FFT** for that window
        * Slide the window over by one
        * Aggregate the results
    + **Calculating the STFT**
  
      - We can calculate the STFT with `librosa` $\rightarrow$ `import librosa as lr`
      - There are several parameters we can tweak (such as **window size**)
      - For our purposes, we'll convert into **decibels** which **normalizes** the average values of all frequencies.
      - We can then visualize it with the `specshow()` function
  
  This is how you can calculate the **STFT**
  
  ```
  # Calculating the STFT
  # Import the functions we'll use for the STFT
  from librosa.core import stft, amplitude_to_db
  from librosa.display import specshow
  
  # Calculate our STFT
  HOP_LENGTH = 2**4
  SIZE_WINDOW = 2**7
  audio_spec = stft(audio, hop_length=HOP_LENGTH, n_fft=SIZE_WINDOW)
  
  # Convert into decibels for visualization
  spec_db = amplitude_to_db(audio_spec)
  
  # Visualize
  specshow(spec_db, sr=sfreq, x_axis='time', y_axis='hz', hop_length=HOP_LENGTH)
  
  ```
  
  ## **Try Spectral feature engineering**
  
  you can also perform the Spectral feature engineering on your baby audio data
  
  * since each time series has a different spectral pattern.
  * We can calculate these spectral patterns by analyzing the spectrogram.
  * For example, **spectral bandwidth** and **spectrum centroids** describe where most of the energy is at each moment in time.
  
  ```
  # Calculate the spectral centroid and bandwidth for the spectrogram
  bandwidths = lr.feature.spectral_bandwidth(S=spec)[0]
  centroids = lr.feature.spectral_centroid(S=spec)[0]
   
  # Display these features on top of the spectrogram
  ax = specshow(spec, x_axis='time', y_axis='hz', hop_length=HOP_LENGTH)
  ax.plot(times_spec, centroids)
  ax.fill_between(times_spec, centroids - bandwidths / 2,centroids + bandwidths / 2, alpha=0.5)
  
  ```
  
  Now you can Combine spectral(`spec` $\rightarrow$ spectral dataframe) and temporal features in a classifier
  
  ```
  centroids_all = []
  bandwidths_all = []
   
  for spec in spectrograms:
      bandwidths = lr.feature.spectral_bandwidth(S=lr.db_to_amplitude(spec))
      centroids = lr.feature.spectral_centroid(S=lr.db_to_amplitude(spec))
      # Calculate the mean spectral bandwidth
      bandwidths_all.append(np.mean(bandwidths))
      # Calculate the mean spectral centroid
      centroids_all.append(np.mean(centroids))
  # Create our X matrix
  X = np.column_stack([means, stds, maxs, tempo_mean, tempo_max, tempo_std, bandwidths_all, centroids_all])
  
  ```
  
  ## **for your logistic regression models**
  
  * One of the ways to improve accuracy is by **optimizing the prediction probability cutoff** scores generated by your logit model.
  * You can **Normalize** all your features to the same scale before putting them in a machine learning model.
  * Look for **class imbalance** in your data.
  * You can **Optimize on other Metrics** also such as **Log Loss** and **F1-Score**.
  * Tune the hyperparameters of your model. In the case of LogisticRegression, parameter $C$ is a hyperparameter.
  
  > **Comments:**
  > **username_0**: thanks alot for giving your time :) the audio files have nearly no noise in it as they are specially recorded sounds from a mobile application of baby crying. i have tried normalizing the values in the array from the range of -1 to 1, TO 0 to 1. Moreover i have tried using ANN which uses spectrograms method and it gave me an accuracy of 100% after 100 epoches. The problem is that i cannot use spectrograms or MFCCs, kindly read my comments in the above question so you will have an idea. Well my data is imbalanced since 222 out of 581 audios are of baby crying. Thanks alot once again! :)  
  > **username_1**: @username_0 FFT is an efficient algorithm, and libraries for it are available for the arduino - see e.g. https://www.youtube.com/watch?v=djGOOL2mqV8  
  
  ---
  
  