readme: ---
  language:
  - en
  tags:
  - t5
  - analysis
  - book
  - notes
  datasets:
  - kmfoda/booksum
  metrics:
  - rouge
  widget:
  - text: I'm just a girl standing in front of a boy asking him to love her.
    example_title: Notting Hill
  - text: Son, your ego is writing checks your body can't cash.
    example_title: top gun
  - text: I really love to eat beans.
    example_title: beans
  - text: >-
      The ledge, where I placed my candle, had a few mildewed books piled up in
      one corner; and it was covered with writing scratched on the paint. This
      writing, however, was nothing but a name repeated in all kinds of
      characters, large and small—Catherine Earnshaw, here and there varied to
      Catherine Heathcliff, and then again to Catherine Linton. In vapid
      listlessness I leant my head against the window, and continued spelling over
      Catherine Earnshaw—Heathcliff—Linton, till my eyes closed; but they had not
      rested five minutes when a glare of white letters started from the dark, as
      vivid as spectres—the air swarmed with Catherines; and rousing myself to
      dispel the obtrusive name, I discovered my candle wick reclining on one of
      the antique volumes, and perfuming the place with an odour of roasted
      calf-skin.
    example_title: Wuthering Heights
  - text: >-
      Did you ever hear the tragedy of Darth Plagueis The Wise? I thought not.
      It’s not a story the Jedi would tell you. It’s a Sith legend. Darth Plagueis
      was a Dark Lord of the Sith, so powerful and so wise he could use the Force
      to influence the midichlorians to create life… He had such a knowledge of
      the dark side that he could even keep the ones he cared about from dying.
      The dark side of the Force is a pathway to many abilities some consider to
      be unnatural. He became so powerful… the only thing he was afraid of was
      losing his power, which eventually, of course, he did. Unfortunately, he
      taught his apprentice everything he knew, then his apprentice killed him in
      his sleep. Ironic. He could save others from death, but not himself.
    example_title: Darth Plagueis The Wise
  inference:
    parameters:
      no_repeat_ngram_size: 2
      max_length: 64
      early_stopping: true
  license: bsd-3-clause
  library_name: transformers
  pipeline_tag: text2text-generation
  ---
  
  
  # T5-large for Lexical Analysis 
  
  - This model was trained a text-to-text task with input text as a summary of a chapter, and the output text as the analysis of that chapter on the [booksum](https://arxiv.org/abs/2105.08209) dataset.
  - it has somewhat learned how to complete literary analysis on an arbitrary input text.
  - **NOTE: this is fairly intensive computationally and recommended to be run on GPU. please see example usage in [this demo notebook](https://colab.research.google.com/gist/pszemraj/8e9cc5bee5cac7916ef9241b66e01b05/demo-t5-large-for-lexical-analysis.ipynb)**
    - The API is set to return max 64 tokens to avoid timeouts on CPU.
  
  ## Example
  
  - PROMPT: 
  
  > Carmen: We're kids, not monsters. Dr. Romero: What's the difference?
  
  - OUTPUT:
  
  > Commentary on Act IV, scenes i-ii In these scenes, we see Dracula's transformation of the children into "monstrous" creatures. Doctor Romero says, "We're kidnapped, but not monsters." This is an important question for the audience to ask: Is there a difference between childhood and adulthood?
  
  _longer examples are available in both [the demo notebook](https://colab.research.google.com/gist/pszemraj/8e9cc5bee5cac7916ef9241b66e01b05/demo-t5-large-for-lexical-analysis.ipynb) and at the bottom of this README._
  
  # checkpoints
  
  - a fine-tuned version of [google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large)
  - 16 epochs of training total
  
  ## Model description
  
  - automatic literary analysis on arbitrary text
  - `booksum` is a dataset created primarily for long-range summarization. It's excellent at that, and usually, this is done with the `chapter` and `summary_text` columns. However, there is a `summary_analysis` column as well, containing literary analysis on the passage in question :eyes:
  - this model was trained on text-to-text with `summary_text` as the input and `summary_analysis` as the output, so it will analyze whatever text for _deeper meaning_.
  
  ## Intended uses & limitations
  
  - given the absence of more specific info, the analysis will reference authors/titles from the training data in the context of the analysis.
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 1e-05
  - train_batch_size: 4
  - eval_batch_size: 4
  - seed: 42
  - distributed_type: multi-GPU
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: cosine
  - num_epochs: 4
  
  ### Training results
  
  
  
  ### Framework versions
  
  - Transformers 4.16.2
  - Pytorch 1.10.0+cu111
  - Datasets 1.18.3
  - Tokenizers 0.11.0
  
  
  ## Extended Examples
  
  the below responses from the model are unedited.
  
  ### A prose passage 