readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  - distilgpt2
  - email generation
  - email
  datasets:
  - aeslc
  - postbot/multi-emails-100k
  
  widget:
  - text: "Good Morning Professor Beans,
  
  Hope you are doing well. I just wanted to reach out and ask if differential calculus will be on the exam"
    example_title: "email to prof"
  - text: "Hey <NAME>,\n\nThank you for signing up for my weekly newsletter. Before we get started, you'll have to confirm your email address." 
    example_title: "newsletter"
  - text: "Hi <NAME>,\n\nI hope this email finds you well. I wanted to reach out and ask about office hours" 
    example_title: "office hours"
  - text: "Greetings <NAME>,\n\nI hope you had a splendid evening at the Company sausage eating festival. I am reaching out because" 
    example_title: "festival"
  - text: "Good Morning Harold,\n\nI was wondering when the next" 
    example_title: "event"
  - text: "URGENT - I need the TPS reports"
    example_title: "URGENT"
  - text: "Hi Archibald,\n\nI hope this email finds you extremely well." 
    example_title: "emails that find you"
  - text: "Hello there.\n\nI just wanted to reach out and check in to"
    example_title: "checking in"
  - text: "Hello <NAME>,\n\nI hope this email finds you well. I wanted to reach out and see if you've enjoyed your time with us"
    example_title: "work well"
  - text: "Hi <NAME>,\n\nI hope this email finds you well. I wanted to reach out and see if we could catch up"
    example_title: "catch up"
  - text: "I'm <NAME> and I just moved into the area and wanted to reach out and get some details on where I could get groceries and"
    example_title: "grocery"
  parameters:
    min_length: 4
    max_length: 128
    length_penalty: 0.8
    no_repeat_ngram_size: 2
    do_sample: False
    num_beams: 8
    early_stopping: True
    repetition_penalty: 5.5
  ---
  
  
  # distilgpt2-emailgen: V2
  
  
  [![colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/gist/pszemraj/d1c2d88b6120cca4ca7df078ea1d1e50/scratchpad.ipynb)
  
  Why write the rest of your email when you can generate it?
  
  ```python
  from transformers import pipeline
  
  model_tag = "postbot/distilgpt2-emailgen-V2"
  generator = pipeline(
                'text-generation', 
                model=model_tag, 
              )
              
  prompt = """
  Hello, 
  
  Following up on the bubblegum shipment."""
  
  result = generator(
      prompt,
      max_length=64,
      do_sample=False,
      early_stopping=True,
  ) # generate
  print(result[0]['generated_text'])
  ```
  
  ## Model description
  
  This model is a fine-tuned version of `distilgpt2` on the postbot/multi-emails-100k dataset.
  It achieves the following results on the evaluation set:
  - Loss: 1.9126
  
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters (run 1/2)
  
  TODO
  
  ### Training hyperparameters (run 2/2)
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0006
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - distributed_type: multi-GPU
  - gradient_accumulation_steps: 8
  - total_train_batch_size: 128
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: cosine
  - lr_scheduler_warmup_ratio: 0.01
  - num_epochs: 4
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 1.9045        | 1.0   | 789  | 2.0006          |
  | 1.8115        | 2.0   | 1578 | 1.9557          |
  | 1.8501        | 3.0   | 2367 | 1.9110          |
  | 1.7376        | 4.0   | 3156 | 1.9126          |
  
  
  ### Framework versions
  
  - Transformers 4.22.2
  - Pytorch 1.10.0+cu113
  - Datasets 2.5.1
  - Tokenizers 0.12.1
  