readme: ---
  language:
  - en
  tags:
  - generated_from_trainer
  datasets:
  - health_fact
  model-index:
  - name: longformer-base-health-fact2
    results: 
    - task:
        type: text-classification
        name: Text Classification
      dataset:
        name: health_fact
        type: health_fact
        split: test
      metrics:
      - name: F1
        type: f1
        value: 0.6732897445517078
      - name: Accuracy
        type: accuracy
        value: 0.797242497972425
      - name: False Accuracy
        type: accuracy
        value: 0.8092783505154639
      - name: Mixture Accuracy
        type: accuracy
        value: 0.5323383084577115
      - name: True Accuracy
        type: accuracy
        value: 0.9081803005008348
      - name: Unproven Accuracy
        type: accuracy
        value: 0.4
  ---
  
  # longformer-base-health-fact2
  
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the health_fact dataset.
  It achieves the following results on the VALIDATION set:
  - Loss: 0.5858
  - Micro F1: 0.8122
  - Macro F1: 0.6830
  - False F1: 0.7941
  - Mixture F1: 0.5015
  - True F1: 0.9234
  - Unproven F1: 0.5128
  
  The following are the results on the TEST set:
  - Macro F1: 0.6732897445517078
  - Accuracy: 0.797242497972425
  - False Accuracy: 0.8092783505154639
  - Mixture Accuracy: 0.5323383084577115
  - True Accuracy: 0.9081803005008348
  - Unproven Accuracy: 0.4
  ## Model description
  
  The health fact dataset is for building fact-checking models related to health. Here is how you can use this model:
  ```python
  import torch
  from transformers import pipeline
  
  claim = "A mother revealed to her child in a letter after her death that she had just one eye because she had donated the other to him."
  text = "In April 2005, we spotted a tearjerker on the Internet about a mother who gave up one of her eyes to a son who had lost one of his at an early age. By February 2007 the item was circulating in e-mail in the following shortened version: My mom only had one eye. I hated her… She was such an embarrassment. She cooked for students and teachers to support the family. There was this one day during elementary school where my mom came to say hello to me. I was so embarrassed. How could she do this to me? I ignored her, threw her a hateful look and ran out. The next day at school one of my classmates said, “EEEE, your mom only has one eye!” I wanted to bury myself. I also wanted my mom to just disappear. I confronted her that day and said, “If you’re only gonna make me a laughing stock, why don’t you just die?” My mom did not respond… I didn’t even stop to think for a second about what I had said, because I was full of anger. I was oblivious to her feelings. I wanted out of that house, and have nothing to do with her. So I studied real hard, got a chance to go abroad to study. Then, I got married. I bought a house of my own. I had kids of my own. I was happy with my life, my kids and the comforts. Then one day, my Mother came to visit me. She hadn’t seen me in years and she didn’t even meet her grandchildren. When she stood by the door, my children laughed at her, and I yelled at her for coming over uninvited. I screamed at her, “How dare you come to my house and scare my children! GET OUT OF HERE! NOW!! !” And to this, my mother quietly answered, “Oh, I’m so sorry. I may have gotten the wrong address,” and she disappeared out of sight. One day, a letter regarding a school reunion came to my house. So I lied to my wife that I was going on a business trip. After the reunion, I went to the old shack just out of curiosity. My neighbors said that she died. I did not shed a single tear. They handed me a letter that she had wanted me to have. My dearest son, I think of you all the time. I’m sorry that I came to your house and scared your children. I was so glad when I heard you were coming for the reunion. But I may not be able to even get out of bed to see you. I’m sorry that I was a constant embarrassment to you when you were growing up. You see……..when you were very little, you got into an accident, and lost your eye. As a mother, I couldn’t stand watching you having to grow up with one eye. So I gave you mine. I was so proud of my son who was seeing a whole new world for me, in my place, with that eye. With all my love to you, Your mother. In its earlier incarnation, the story identified by implication its location as Korea through statements made by both the mother and the son (the son’s “I left my mother and came to Seoul” and the mother’s “I won’t visit Seoul anymore”). It also supplied a reason for the son’s behavior when his mother arrived unexpectedly to visit him (“My little girl ran away, scared of my mom’s eye” and “I screamed at her, ‘How dare you come to my house and scare my daughter!'”). A further twist was provided in the original: rather than gaining the news of his mother’s death from neighbors (who hand him her letter), the son instead discovered the woman who bore him lying dead on the floor of what used to be his childhood