readme: ---
  license: apache-2.0
  tags:
  - vision
  datasets:
  - imagenet-21k
  ---
  
  # ImageGPT (small-sized model) 
  
  ImageGPT (iGPT) model pre-trained on ImageNet ILSVRC 2012 (14 million images, 21,843 classes) at resolution 32x32. It was introduced in the paper [Generative Pretraining from Pixels](https://cdn.openai.com/papers/Generative_Pretraining_from_Pixels_V2.pdf) by Chen et al. and first released in [this repository](https://github.com/openai/image-gpt). See also the official [blog post](https://openai.com/blog/image-gpt/).
  
  
  ## Model description
  
  The ImageGPT (iGPT) is a transformer decoder model (GPT-like) pretrained on a large collection of images in a self-supervised fashion, namely ImageNet-21k, at a resolution of 32x32 pixels. 
  
  The goal for the model is simply to predict the next pixel value, given the previous ones.
  
  By pre-training the model, it learns an inner representation of images that can then be used to:
  - extract features useful for downstream tasks: one can either use ImageGPT to produce fixed image features, in order to train a linear model (like a sklearn logistic regression model or SVM). This is also referred to as "linear probing".
  - perform (un)conditional image generation. 
  
  ## Intended uses & limitations
  
  You can use the raw model for either feature extractor or (un) conditional image generation. 
  
  ### How to use
  
  Here is how to use this model as feature extractor:
  
  ```python
  from transformers import AutoFeatureExtractor
  from onnxruntime import InferenceSession
  from datasets import load_dataset
  
  # load image
  dataset = load_dataset("huggingface/cats-image")
  image = dataset["test"]["image"][0]
  
  # load model
  feature_extractor = AutoFeatureExtractor.from_pretrained("openai/imagegpt-small")
  session = InferenceSession("model/model.onnx")
  
  # ONNX Runtime expects NumPy arrays as input
  inputs = feature_extractor(image, return_tensors="np")
  outputs = session.run(output_names=["last_hidden_state"], input_feed=dict(inputs))
  ```
  Or you can use the model with classification head that returns logits
  ```python
  from transformers import AutoFeatureExtractor
  from onnxruntime import InferenceSession
  from datasets import load_dataset
  
  # load image
  dataset = load_dataset("huggingface/cats-image")
  image = dataset["test"]["image"][0]
  
  # load model
  feature_extractor = AutoFeatureExtractor.from_pretrained("openai/imagegpt-small")
  session = InferenceSession("model/model_classification.onnx")
  
  # ONNX Runtime expects NumPy arrays as input
  inputs = feature_extractor(image, return_tensors="np")
  outputs = session.run(output_names=["logits"], input_feed=dict(inputs))
  ```
  ## Original implementation
  
  Follow [this link](https://huggingface.co/openai/imagegpt-small) to see the original implementation.
  
  ## Training data
  
  The ImageGPT model was pretrained on [ImageNet-21k](http://www.image-net.org/), a dataset consisting of 14 million images and 21k classes. 
  
  ## Training procedure
  
  ### Preprocessing
  
  Images are first resized/rescaled to the same resolution (32x32) and normalized across the RGB channels. Next, color-clustering is performed. This means that every pixel is turned into one of 512 possible cluster values. This way, one ends up with a sequence of 32x32 = 1024 pixel values, rather than 32x32x3 = 3072, which is prohibitively large for Transformer-based models. 
  
  ### Pretraining
  
  Training details can be found in section 3.4 of v2 of the paper.
  
  ## Evaluation results
  
  For evaluation results on several image classification benchmarks, we refer to the original paper.
  
  ### BibTeX entry and citation info
  
  ```bibtex
  @InProceedings{pmlr-v119-chen20s,
    title = 	 {Generative Pretraining From Pixels},
    author =       {Chen, Mark and Radford, Alec and Child, Rewon and Wu, Jeffrey and Jun, Heewoo and Luan, David and Sutskever, Ilya},
    booktitle = 	 {Proceedings of the 37th International Conference on Machine Learning},
    pages = 	 {1691--1703},
    year = 	 {2020},
    editor = 	 {III, Hal DaumÃ© and Singh, Aarti},
    volume = 	 {119},
    series = 	 {Proceedings of Machine Learning Research},
    month = 	 {13--18 Jul},
    publisher =    {PMLR},
    pdf = 	 {http://proceedings.mlr.press/v119/chen20s/chen20s.pdf},
    url = 	 {https://proceedings.mlr.press/v119/chen20s.html
  }
  ```
  
  ```bibtex
  @inproceedings{deng2009imagenet,
    title={Imagenet: A large-scale hierarchical image database},
    author={Deng, Jia and Dong, Wei and Socher, Richard and Li, Li-Jia and Li, Kai and Fei-Fei, Li},
    booktitle={2009 IEEE conference on computer vision and pattern recognition},
    pages={248--255},
    year={2009},
    organization={Ieee}
  }
  ```
  