As I work with Speech Synthesis, from what I have seen people generally have a hard time reconstructing the raw waveforms and I have not seen a lot of people trying to do it as you lose a lot of information in the frequency domain which could be useful for the properties of the Audio data, I think a good idea will be to turn it into more audio friendly representation’s first like Spectrogram or MFCC or Mel-Spectrogram if it is speech.
More information could be found regarding this in torchaudio.transforms.Spectrogram and torchaudio.transforms.MFCC.
Then you can use any vocoder to reconstruct it into the raw waveforms.
I hope it helps