joao_marques:

What this does? zero_padding = tf.zeros([16000] - tf.shape(waveform), dtype=tf.float32)


Adding some annotation, hope this helps:
def get_spectrogram(waveform):
  # Zero-padding for audio waveforms with less than 16,000 samples
  zero_padding = tf.zeros(
      [16000] - tf.shape(waveform),
      dtype=tf.float32)
  # Cast the waveform tensors' dtype to float32.
  waveform = tf.cast(waveform, dtype=tf.float32)
  # Concatenate the waveforms with `zero_padding`, which ensures all audio
  # clips are of the same length.
  equal_length = tf.concat([waveform, zero_padding], 0)
  # Convert the waveforms to spectrograms via a STFT.
  spectrogram = tf.signal.stft(
      equal_length, frame_length=255, frame_step=128)
  # Obtain the magnitude of the STFT.
  spectrogram = tf.abs(spectrogram)

  return spectrogram




 joao_marques:

Why we casting it to 32? waveform = tf.cast(waveform, tf.float32)


You can read about precision and running computations on ML hardware here: https://www.tensorflow.org/guide/mixed_precision:

Today, most models use the float32 dtype, which takes 32 bits of memory. However, there are two lower-precision dtypes, float16 and bfloat16, each which take 16 bits of memory instead. Modern accelerators can run operations faster in the 16-bit dtypes, as they have specialized hardware to run 16-bit computations and 16-bit dtypes can be read from memory faster.

and here: Reduce costs and increase throughput with NVIDIA T4s, P100s, V100s | Google Cloud Blog.



 joao_marques:

Can I create a spectogram from mp3?


Check out this notebook on Kaggle: Audio Data Analysis Using librosa 📈 | Kaggle
librosa.load() # for loading the MP3
librosa.display()
librosa.stft() # to convert to the time-frequency domain using the short-time Fourier transform)
librosa.amplitude_to_db()
librosa.display.specshow() # display a spectrogram

(LibROSA docs)



 joao_marques:

Again I’m no expert in this field and I was just doing this for fun.


More useful resources from the world of research:


A Tutorial on Deep Learning for Music Information Retrieval (arXiv)

Notes on Music Information Retrieval - with notebooks