Solved by AfonsoSalgadoSousa in post #7 
              
              
                Managed to fix the problem by not using the following line: 
x = F.pad(x, pad=(self.n_fft // 2, self.n_fft // 2), mode=self.pad_mode)