No, I am so new to this. I wouldn’t even know how to combine both.
For your reference for the decoder, please visit this blog:

  
      
      assemblyai.com
  
  
    

Building an end-to-end Speech Recognition model in PyTorch 33

Deep Learning has changed the game in speech recognition with the introduction of end-to-end models. These models take in audio, and directly output transcriptions. Two of the most popular end-to-end models today are Deep Speech by Baidu, and Listen...


  
  
    
    
  
  


It explains how to create decoder in pytorch.
I hope I helped you.