Hi Chitra,
I ran into this problem as well and HuggingFace has shown that it can be resolved by changing how the inputs are passed to the transformer layer:
  

      github.com/huggingface/huggingface_hub
  

  
    
  
	  
  

  
    
      Can't upload custom TF models e.g. TFBERT, TFRoBERTa due to model.save() failing 3
    

    
      
        opened 
        
          
        
        Jan 10, 2022
      
      


      
        
          
          ck37
        
      
    

    
    
  


  
    Hello,

I'm interested in sharing models on the HF Hub but it appears that cus…tom TF models can't be shared because their model.save() doesn't work (appears to be a known issue: https://github.com/huggingface/transformers/issues/13610#issuecomment-921134250).

I've [uploaded a Jupyter notebook to colab](https://colab.research.google.com/drive/12uUvOTKUsPTcBMJphhhgglQRT_8pcQmC#scrollTo=VhGQbQvYevrM) which shows the error for simple RoBERTA and BERT models. Note: I ran this in jupyter rather than Colab so it would probably require a little editing to work directly in Colab.

Is it possible to also use save_pretrained() for this use case as well? Even if so it would be good to resolve the model.save() issue for the TF user base, e.g. others are struggling with this https://discuss.tensorflow.org/t/list-index-out-of-range-while-saving-a-trained-model/6901

cc: @Rocketknight1 @LysandreJik 

Thanks,
Chris
  

  

  
    
    
  

  


If you change this line:
embedding = encoder([input_word_ids, input_mask])[0]  # FC: add_input_type_ids for the BERT model

To this (i.e. pass the inputs as separate arguments rather than as a list):
embedding = encoder(input_word_ids, input_mask)[0]  # FC: add_input_type_ids for the BERT model

Are you able to save()?
Cheers,
Chris