 Model description  The ImageGPT (iGPT) is a transformer decoder model (GPT-like) pretrained on a large collection of images in a self-supervised fashion, namely ImageNet-21k, at a resolution of 32x32 pixels.   The goal for the model is simply to predict the next pixel value, given the previous ones.  By pre-training the model, it learns an inner representation of images that can then be used to: - extract features useful for downstream tasks: one can either use ImageGPT to produce fixed image features, in order to train a linear model (like a sklearn logistic regression model or SVM). This is also referred to as "linear probing". - perform (un)conditional image generation.   