readme: ---
  license: other
  ---
  # xs_blenderbot_onnx (only 168 mb)
  onnx quantized version of facebook/blenderbot_small-90M model (350 mb)
  
  Faster cpu inference
  
  ## INTRO
  
  Before usage:
  
    • download blender_model.py script from files in this repo
  
    • pip install onnxruntime
  
  you can use the model with huggingface generate function with its all parameters
  
  # Usage
  
  With text generation pipeline
  
  ```python
  >>>from blender_model import TextGenerationPipeline
  
  >>>max_answer_length = 100
  >>>response_generator_pipe = TextGenerationPipeline(max_length=max_answer_length)
  >>>utterance = "Hello, how are you?"
  >>>response_generator_pipe(utterance)
  i am well. how are you? what do you like to do in your free time?
  ```
  Or you can call the model
  
  ```python
  >>>from blender_model import OnnxBlender
  >>>from transformers import BlenderbotSmallTokenizer
  >>>original_repo_id = "facebook/blenderbot_small-90M"
  >>>repo_id = "remzicam/xs_blenderbot_onnx"
  >>>model_file_names = [
      "blenderbot_small-90M-encoder-quantized.onnx",
      "blenderbot_small-90M-decoder-quantized.onnx",
      "blenderbot_small-90M-init-decoder-quantized.onnx",
  ]
  >>>model=OnnxBlender(original_repo_id, repo_id, model_file_names)
  >>>utterance = "Hello, how are you?"
  >>>inputs = tokenizer(utterance,
                      return_tensors="pt")
  >>>outputs= model.generate(**inputs,
                          max_length=max_answer_length)
  >>>response = tokenizer.decode(outputs[0],
                          skip_special_tokens = True)
  >>>print(response)
  i am well. how are you? what do you like to do in your free time?
  ```
  
  # Credits
  To create the model, I adopted codes from https://github.com/siddharth-sharma7/fast-Bart repository.