readme: ---
  title: Video Vision Transformer on medmnist
  emoji: 🧑‍⚕️
  colorFrom: red
  colorTo: green
  sdk: gradio
  app_file: app.py
  pinned: false
  license: apache-2.0
  library_name: keras
  ---
  
  ## Keras Implementation of Video Vision Transformer on medmnist
  
  This repo contains the model [to this Keras example on Video Vision Transformer](https://keras.io/examples/vision/vivit/).
  
  ## Background Information 
  This example implements [ViViT: A Video Vision Transformer](https://arxiv.org/abs/2103.15691) by Arnab et al., a pure Transformer-based model for video classification. The authors propose a novel embedding scheme and a number of Transformer variants to model video clips.
  
  ## Datasets
  We use the [MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification](https://medmnist.com/) dataset.
  
  ## Training Parameters
  ```
  # DATA
  DATASET_NAME = "organmnist3d"
  BATCH_SIZE = 32
  AUTO = tf.data.AUTOTUNE
  INPUT_SHAPE = (28, 28, 28, 1)
  NUM_CLASSES = 11
  
  # OPTIMIZER
  LEARNING_RATE = 1e-4
  WEIGHT_DECAY = 1e-5
  
  # TRAINING
  EPOCHS = 80
  
  # TUBELET EMBEDDING
  PATCH_SIZE = (8, 8, 8)
  NUM_PATCHES = (INPUT_SHAPE[0] // PATCH_SIZE[0]) ** 2
  
  # ViViT ARCHITECTURE
  LAYER_NORM_EPS = 1e-6
  PROJECTION_DIM = 128
  NUM_HEADS = 8
  NUM_LAYERS = 8
  ```