readme: ---
  language: 
  - en
  
  tags:
  - text-classification
  - fake-news
  - pytorch
  datasets:
  - Fake News https://www.kaggle.com/datasets/clmentbisaillon/fake-and-real-news-dataset
  metrics:
  - Accuracy, AUC
  ---
  
  ## Model description:
  [Distilbert](https://arxiv.org/abs/1910.01108) is created with knowledge distillation during the pre-training phase which reduces the size of a BERT model by 40%, while retaining 97% of its language understanding. It's smaller, faster than Bert and any other Bert-based model.
  
  [Distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) finetuned on the fake news dataset with below Hyperparameters
  ```
   learning rate 5e-5, 
   batch size 32,
   num_train_epochs=2,
  ```
  Full code available @ [DistilBert-FakeNews](https://github.com/anasserhussien/DistilBert-FakeNews)
  
  Dataset available  @  [Fake News dataset](https://www.kaggle.com/datasets/clmentbisaillon/fake-and-real-news-dataset)
  
  