readme: ---
  license: apache-2.0
  tags:
  - image-classification
  datasets:
  - Face-Mask18K 
  ---
  
  # Distilled Data-efficient Image Transformer for Face Mask Detection
  
  Distilled data-efficient Image Transformer (DeiT) model pre-trained and fine-tuned on Self Currated Custom Face-Mask18K Dataset (18k images, 2 classes) at resolution 224x224. It was first introduced in the paper Training data-efficient image transformers & distillation through attention by Touvron et al.  
  
  ## Model description
  
  This model is a distilled Vision Transformer (ViT). It uses a distillation token, besides the class token, to effectively learn from a teacher (CNN) during both pre-training and fine-tuning. The distillation token is learned through backpropagation, by interacting with the class ([CLS]) and patch tokens through the self-attention layers.
  
  Images are presented to the model as a sequence of fixed-size patches (resolution 16x16), which are linearly embedded. 
  
  ## Training Metrics
      epoch                    =          2.0
      total_flos               = 2078245655GF
      train_loss               =       0.0438
      train_runtime            =   1:37:16.87
      train_samples_per_second =        9.887
      train_steps_per_second   =        0.309
  
  ---
  
  ## Evaluation Metrics
      epoch                   =        2.0
      eval_accuracy           =     0.9922
      eval_loss               =     0.0271
      eval_runtime            = 0:03:17.36
      eval_samples_per_second =      18.22
      eval_steps_per_second   =       2.28