GPT-J 6B
Model Description
GPT-J 6B is a transformer model trained using Ben Wang's Mesh Transformer JAX. "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
Original implementation
Follow this link to see the original implementation.
How to use
Download the model by cloning the repository via git clone https://huggingface.co/OWG/gpt-j-6B.
Then you can use the model with the following code:
from onnxruntime import InferenceSession, SessionOptions, GraphOptimizationLevel
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
options = SessionOptions()
options.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL
session = InferenceSession("path/to/model.onnx", sess_options=options)
session.disable_fallback()
TODO
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.