transformers safetensors distilbert text-classification autotrain_compatible endpoints_compatible region:us
---
{}
---

**Model Description**: 
  The DistilBERT model was proposed in the blog post Smaller, faster, cheaper,
  lighter: Introducing DistilBERT, a distilled version of BERT, and the paper
  DistilBERT, a distilled version of BERT: smaller, faster, cheaper, and
  lighter. DistilBERT is a small, fast, cheap, and light Transformer model
  trained by distilling BERT base. It has 40% fewer parameters than
  bert-base-uncased, runs 60% faster while preserving over 95% of BERT's
  performance as measured on the GLUE language understanding benchmark.

  This model is a fine-tune checkpoint of
  [distilbert-base-cased-distilled-squad](https://huggingface.co/distilbert/distilbert-base-cased-distilled-squad),
  fine-tuned the task layer using [QNLI](https://huggingface.co/datasets/nyu-mll/glue) dataset. Our primary objective was to leverage the model's acquired knowledge during question answering to identify entailment relations between context sentences and questions to explain the model reasoning.

**Model Type**: 
  Transformer-based language model
  
**Language(s)**: 
  English
  
**License**: 
  cc
  
**Related Models**: 
  distilbert/distilbert-base-cased-distilled-squad
  
**How to Get Started with the Model**: >
  ```python

  from transformers import pipeline

  classifier = pipeline("zero-shot-classification",
  model="HeZhang1019/distilbert-base-cased-distilled-squad-qnli-v1")

  sentences = [
  "Hamlet is a play written by William Shakespeare in the early 17th century.", 
  "It is one of his most well-known works, featuring the famous quote 'To be, or not to be.'"
  ]


  question = 'Who wrote "Hamlet"?'

  results = list()
  for sentence in sentences:
      result = classifier(question, sentence)
      results.append((sentence, result['scores'][0]))
  print(results)
  ```
  ```python
  [('Hamlet is a play written by William Shakespeare in the early 17th century.', 0.8046764731407166), ("It is one of his most well-known works, featuring the famous quote 'To be, or not to be.'", 0.39214783906936646)]
  ```

**Uses**:  This model can be used for entailment classification.

**Model Training** 
- Script:
  - [Colab](https://colab.research.google.com/drive/1lQ3hshFZbgnxwuOOYALIA3FSFWDOLN3w?usp=sharing)
- Evaluation:
  - Epoch: 13
  - Training Loss (0.310600)
  - Validation Loss (0.298734)
  - Accuracy (0.886509)
  - Precision (0.886524)
  - Recall (0.886509)
  - F1 Score (0.886502)
