I think so, just do requires_grad_ on your input. In the end it goes through torchaudio.transforms.functional.spectrogram and uses the torch.stft function. This calls torch.fft (I think), which has a derivative defined.
There are several texts about how the inner parts of PyTorch work, I wrote something simple a long time ago 13 and @ezyang has an awesome comprehensive tour of PyTorch internals 8.
Best regards
Thomas