HI Karthik,
Forgive me if you have looked into this, but have you tried model quantization?
  
      pytorch.org
  
  
    

Quantization — PyTorch 1.7.0 documentation 5



  
  
    
    
  
  


If ONNX is not supported, then your only other bet is to decrease the size of the model or change some parameters like you’ve been trying.