I would write a script to extract a feature vector from every column and use them to feed e.g. lstm. If you have one label for each spectrogram you should get the lstm output corresponding to the end of your sequence and compare to your label using a loss function so you can do backprop over time. That’s the basic approach.