I’m actually working on a similar problem. To reduce the size of the processed the WAV using Mel-frequency cepstrum, it applies a linear cosine transform of a log power spectrum on a nonlinear mel scale of frequency, by applying that i was able to reduce the size of my data from 300GB to just over 20GB.
I’ve also split my data into 3 segments: development, training and testing. The development set is much smaller than the training ~ 5% of the training set size, and it’s purpose is for quick experimentation. With this approach you can quickly catch errors, and you can move to the larger training sample once you are satisfied with the results.