See also:
  

      github.com/huggingface/transformers
  

  
    
  
	  
  

  
    
      How do you do inference in production? 2
    

    
      
        opened 
        
          
        
        Mar 12, 2020
      
      

        
          closed 
        
          
        
        Apr 30, 2020
      
        

      
        
          
          ZhuoranLyu
        
      
    

    
    
  


  
    # ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for b…ugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details

I was wondering how do you guys do inference in production? I tried to convert this model to tensorflow model but failed.
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
This is what I tried:

```
tf_model = TFGPT2LMHeadModel.from_pretrained("tmp/", from_pt=True)
tf.saved_model.save(tf_model,"tmp/saved")
loaded = tf.saved_model.load("tmp/saved")
print(list(loaded.signatures.keys()))  
```
And it returns an empty list

**A link to original question on Stack Overflow**: https://stackoverflow.com/questions/52826134/keras-model-subclassing-examples