How to cosume movement-pruning .h5 models in QnA pipeline
# ❓ Questions & Help

## Details

Hi, I am working on the movement pruning method for QnA models. I have performed all the [given steps](https://github.com/huggingface/transformers/tree/master/examples/movement-pruning) in order to generate the pruned model and then used [this notebook](https://github.com/huggingface/transformers/blob/master/examples/movement-pruning/Saving_PruneBERT.ipynb) to generate the .h5 model files. 

Though I am facing an issue with consuming this model with `QuestionAnsweringPipeline`.

For loading the model; the config file is copied from `BertForQuestionAnswering`, as the pruning repo does not generate any config file.

`BERT_PRUNED_PATH = SERIALIZATION_DIR+'/dbg' +'/BERT_Pruned/'`

`config = BertConfig.from_json_file(BERT_PRUNED_PATH+'config.json')`

*we used squad_sparse.h5 which is renamed to tf_model.h5*

`model_BERT_pruned =
TFBertForQuestionAnswering.from_pretrained(BERT_PRUNED_PATH+'tf_model.h5',config=config)`

**or** 

`model_BERT_pruned =
BertForQuestionAnswering.from_pretrained(BERT_PRUNED_PATH+'tf_model.h5',config=config,from_tf=True)
`

config = 
{
  "architectures": [
    "BertForQuestionAnswering"
  ],
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "type_vocab_size": 2,
  "vocab_size": 30522
}


For pipeline,

`
fill_mask_qna_BERT_pruned = pipeline(
  "question-answering",
model=model_BERT_pruned,
tokenizer=tokenizer,
framework="tf"
)
`

Now, when I test the pipeline on questions and context, I get very random answers, probably because the model is not getting loaded in a proper fashion as it should be. 

@VictorSanh Can you share the instructions on how to load these .h5 format pruned model using Huggingface modules? Or is there any other way to consume the model.
Hello,
I've only work with PyTorch for all the experiments for movement pruning (even though I've converted the checkpoints in the hub to their TF version).
The instructions in the notebook show you how you can load a optimized version of the checkpoint (pruning+quantization) which was saved with hdf5 with an .h5 extension. It is not a tensorflow checkpoint. You would have to adapt the steps in the notebook to do it in tensorflow.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
