readme: ---
  language:
  - en
  datasets:
  - Spotify Podcasts Dataset
  tags:
  - t5
  - summarisation
  - pytorch
  - lm-head
  metrics:
  - ROUGE
  pipeline:
  - summarisation
  ---
  
  # T5 for Automatic Podcast Summarisation
  
  This model is the result of fine-tuning [t5-base](https://huggingface.co/t5-base) on the [Spotify Podcast Dataset](https://arxiv.org/abs/2004.04270).
  
  It is based on [Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) which was pretrained on the [C4 dataset](https://huggingface.co/datasets/c4).
  
  
  Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)
  
  Authors: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
  
  ## Intended uses & limitations
  This model is intended to be used for automatic podcast summarisation. As creator provided descriptions
  were used for training, the model also learned to generate promotional material (links, hashtags, etc) in its summaries, as such
  some post processing may be required on the model's outputs.
  
  If using on Colab, the instance will crash if the number of tokens in the transcript exceeds 7000. I discovered that the model
  generated reasonable summaries even when the podcast transcript was truncated to reduce the number of tokens.
  
  #### How to use
  
  The model can be used with the summarisation as follows:
  
  ```python
  from transformers import pipeline
  
  summarizer = pipeline("summarization", model="paulowoicho/t5-podcast-summarisation", tokenizer="paulowoicho/t5-podcast-summarisation")
  summary = summarizer(podcast_transcript, min_length=5, max_length=20)
  
  print(summary[0]['summary_text'])
  ```
  
  ## Training data
  
  This model is the result of fine-tuning [t5-base](https://huggingface.co/t5-base) on the [Spotify Podcast Dataset](https://arxiv.org/abs/2004.04270).
  [Pre-processing](https://github.com/paulowoicho/msc_project/blob/master/reformat.py) was done on the original data before fine-tuning.
  
  ## Training procedure
  
  Training was largely based on [Fine-tune T5 for Summarization](https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers_summarization_wandb.ipynb) by [Abhishek Kumar Mishra](https://github.com/abhimishra91) 
  