hello！
I want to speed up the generation of gpt2 Chinese model！
Convert to onnx as required by the document.
  
      

      huggingface.co
  

  
    

Exporting transformers models

  ONNX / ONNXRuntime: Projects ONNX (Open Neural Network eXchange) and ONNXRuntime (ORT) are part of an effort from leading industries in the AI field to provi...


  

  
    
    
  

  


1、Test according to document code
import onnxruntime as ort
from transformers import BertTokenizerFast
tokenizer = BertTokenizerFast.from_pretrained(“bert-base-cased”)
ort_session = ort.InferenceSession(“onnx/bert-base-cased/model.onnx”)
inputs = tokenizer(“Using BERT in ONNX!”, return_tensors=“np”)
outputs = ort_session.run([“last_hidden_state”, “pooler_output”], dict(inputs))
outputs how should I decode it？
After reading this case.
  

      github.com
  

  
    microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/notebooks/Inference_GPT2_with_OnnxRuntime_on_CPU.ipynb


    {
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (c) Microsoft Corporation. All rights reserved.  \n",
    "Licensed under the MIT License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference PyTorch GPT2 Model with ONNX Runtime on CPU\n",
    "\n",
    "In this tutorial, you'll be introduced to how to load a GPT2 model from PyTorch, convert it to ONNX, and inference it using ONNX Runtime using IO Binding. Note that past state is used to get better performance."
   ]
  },
  {



  This file has been truncated. show original

  

  
    
    
  

  


I really don’t understand. Forgive me for being a little white.
Hope to get help！