Hugging Face's logo

pszemraj
/
long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP13

Summarization
Transformers
PyTorch
longt5
text2text-generation
summary
booksum
long-document
long-form
Eval Results
Model card Files Community
8
long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP13 / README.md
pszemraj's picture
pszemraj
Librarian Bot: Add base_model information to model (#8)
1420129
preview code
|
raw
history blame contribute delete
12.7 kB
metadata
license:
  - apache-2.0
  - bsd-3-clause
tags:
  - summarization
  - summary
  - booksum
  - long-document
  - long-form
datasets:
  - kmfoda/booksum
metrics:
  - rouge
inference: false
base_model: pszemraj/long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP12
model-index:
  - name: pszemraj/long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP13
    results:
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: test
        metrics:
          - type: rouge
            value: 24.4101
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjhmM2NiMDQ1NjI3Zjk4YjkyMTVkMmUwZDU2YWMwZjc4ZmIzMjA1OGZiYzRmNjI3NDk3OWNmOTlkZDMxZmViMyIsInZlcnNpb24iOjF9.wS774e7vxQrf2gCcPhySsET3UaiUsj8E7mQmBS84wz86aT9j1yCqVX-8ozuj896K5wMygbL-TpUbydRIyyHTDw
          - type: rouge
            value: 5.003
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTYyNTFkNWZhOTgwNDg5ZWU5Y2U5NGI4Y2Y2YTMxNjUzOWI0ZWNlNDE1OGYzMjA1YTBmNDE4ZjcyOTZmODE4NiIsInZlcnNpb24iOjF9.AuqDkCgUgDWl8vMyrjTh59QW741UssGxdBqj3GZKy5e5gKadClUA709qgKbpxPIbMEyk38yvXYGplaJf5CnCCA
          - type: rouge
            value: 17.2544
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTBmODZmNWRhMzBhY2MzOGRkZWQzNjAzMGViOGMxYWYyZjNlZmM4YzgzMjkxNTk3M2E1ODAwZjY1M2I2MDZkYyIsInZlcnNpb24iOjF9.Md52aHjujvkxaW-ubJNquiHHHgi-OfRav0ZElVvYhIpU_k0iKEaQZRcw9JYjtG5vZJbQeiWbMzcCOJ999DhrAA
          - type: rouge
            value: 20.9183
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDJjNDc1OTZjY2VmNWRhNmYwZjRjY2JmNTAyNmIwZjRhYjMyMTdlNzY2M2Q4OGQwNTEyYTU0NGVhYWI2ZTk3NSIsInZlcnNpb24iOjF9.nlqol0HEeEjU7509-B9eyohf3CP3EZTibJ1lTvOx3wt8rU5LzEdwFazOTHjpWlcK_rik7jcySdUDe4fGjJtKAQ
          - type: loss
            value: 3.194674015045166
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzRiYmRiYjFkZDhlNGIwYTg3NDUwZTEzZjc5MjllNmJmODQ1YzBjNDM4MzQwNmMzNmNkMzk5N2M2MzZlOWY4MyIsInZlcnNpb24iOjF9._YJqPY9p_N2n7UxAkTeGenH1sVAkC_Z5HzZ6NbzlQoa8-RXTfbEPLw7fSKmlsGNyZxj7L_Bs4COIWzwAMxZSAA
          - type: gen_len
            value: 58.9951
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDJhYzU2Zjg4ZmIyOGRmNTU4MDM2NGZiNzc0NDk3YzZkOTQwMWMwNjMzZDQzZTZiZjk4ZDdmMmI2ODRkYjk3OCIsInZlcnNpb24iOjF9.MG1rcM_qpUhQmAYrsBxyNpcLUrPZw6V_uzYzDAo01kQyZEwJClWgMRVgpsSEnY93Mlu1445QLxkJEByUrfD3BQ
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: billsum
          type: billsum
          config: default
          split: test
        metrics:
          - type: rouge
            value: 37.3648
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWU4ZmZmYzllMzQxM2I4YTUxMjkwYjEzNDk1NjRlYjJiZjYyYWNiNzM4ODMxMGJjMzdhYjFhMzhlNTE5YmYyMiIsInZlcnNpb24iOjF9.9NTlO_5zLC8Y3mkwstviPb9WmMqPmXfWfEN0yONA6WYhh1jPy0gECEb5uF0G6wBMhTPDTqGMWOYIAF2vMeNbDA
          - type: rouge
            value: 12.3316
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTJhZTcxMDc5ODljMDBjYzFmYWIzNTA4M2NiZDUwYTMwNTVjZTUyZTU2M2IwYWE2YjkzMzMzMjg1MDU1OWE1NSIsInZlcnNpb24iOjF9.FRsoRao8qj6A8W7OeIVAoZCEc1HCZEzmKOs0CPkUceF19pk1ngaXt5K6kcPJ-5fYJydtfSuSnuG3aqlOEJeYDQ
          - type: rouge
            value: 22.075
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2FjNTMxMGYyNjgyNjk2YTQwZjM4MTM4Yjg0MTkyN2RmNDE5YTU5ZDNkZDFhZDM2YWRlNDI4M2JlMWYxNDQ3ZCIsInZlcnNpb24iOjF9.wsLUEYGJyMSJPPclOzb1hcRdE-VrZex2Sd5er_XVbe6bY1cRO5DdIn69sE9hmAcltefu4ikpHu2ihbv7qvj4Aw
          - type: rouge
            value: 31.1679
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTUyODVkZGIxYzMyZDczNzU5YjVkNTliZmM4ZTdiYWE2ZjJhNGM3ZDgzMWE3ZjA2MDBhZWQ1ZGY1YzNmZDMwNiIsInZlcnNpb24iOjF9.fPgMnnXY5oPdCn1STZ0HwUiil8OlLZ8ZWZZav_chDIQ7Kh1RKeLy0EG2vEhrB6IlyP7uZ3RmdT9VHM1_khrEAw
          - type: loss
            value: 2.745267391204834
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWQ2NDVmODI2ZTQyNmVjZjRkZDdlMTdiODBkZTlkNTFkODBjNjViMTZhMDVkYTkwYWIyNDFkZWZhZmJhODEwMyIsInZlcnNpb24iOjF9.9JWTqdGEhztS--N8grHY6q2a8taVu65Lr17ocXgudp4imhqr9Bhau2X2G5SLN7c1oYieKtyKcWdDAmVzHyTbDw
          - type: gen_len
            value: 157.3126
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWNiODFmMWQ1ZTkzZGNjNDkwM2ZiZjJlZmQ3N2ExNWJhYmUxYWM2ZGNiYzlhYTY5Y2RhOGVlZDhmN2ZmODQwYSIsInZlcnNpb24iOjF9.sRA9iBS4vzFDZtwM4Vs6Kevj3eiTkS5akApUWTZBCt58YSW8mpoKqsWcnQFEjDCCec-FfV_451OLIetcmDZiCA
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: xsum
          type: xsum
          config: default
          split: test
        metrics:
          - type: rouge
            value: 18.2975
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjJhMjQ0Yzc4ZmNkOWI5YjhmOTlmOTA4MTE0NWM4NGRlNjE0NDIwOTY2ZmQyNjA0ZmE5MjM2YjAyZDZiNWFkNiIsInZlcnNpb24iOjF9.2UJ48OcezjnfMC0dGjksZpAiXRGNAOHniHdN-tQmQPo0vXwRYNTyPrVULnVoBZUvSdycTYvjl0jDKNhZmtGfCA
          - type: rouge
            value: 2.6806
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTlkMmQwZTRmN2JlOTQ0N2I0YjdhOTBmYmU3MzEwNzE2ZjFiOTM4OWMyMWRhNmZjNTBkZWY5OGMwYTZhZDRhYSIsInZlcnNpb24iOjF9.7D-IR1aBxx1goOkbeA3Tzd1Wu0Zfi0yQVSG8HWSboM7J67TBHblFsFCVJE7Z2wZRbBW4WtuDIGAcl1d1_Wu_Aw
          - type: rouge
            value: 11.9453
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGZjNmY5NmU5ODBmMDQyMjhhNzY3NzBlNDEyMTE3NjY1ZmRkZDZkZWI1YTA0ZTA0NzU1MjMzOTNjZDA3YWM1MCIsInZlcnNpb24iOjF9.SlI42pwrWc_OlcBKOPtrYNzvK_DUk6IJlzrrtjvkZX7k1S7bguekAV-_rWHfn_82k8rJ1FQAReasGHu1dZ0aBw
          - type: rouge
            value: 14.2121
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2E2MGE0MTQ1YmU0MTJkOWY3ZDhhODIwYWNhNTE3YWJkZTFhYzM1ZjBmNGExODIzYmU2YzE1ODg4ZjdhZWMwMiIsInZlcnNpb24iOjF9.K5FEsZtSph0FqF5zwetkE-X5AKOlj5g_02DPdl-kEe1azKrBBZy9sDiS0WfIGfwHiRdNvKGKi8t3PAGPsfQwCQ
          - type: loss
            value: 4.836681365966797
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzhlYjA0YzZmYjdmYWQwNDFhNzIzYWNkYzM4OGFlOWJiY2EzYTkxYjk3ZmJmNGQyMGE1ZmYzMDU2MzhhMmVkMiIsInZlcnNpb24iOjF9.uHYwqPBg6K63exBvqt__c82gKi52OhPTRSrcIKHOECCmoXJLJKgFJCuIXGWMJ7UP4HG375e9uqunJB0XwC20DA
          - type: gen_len
            value: 96.2584
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjNjYzQzNmM5NTY2YzVhNzRkZjMxMzhiYTU1MDBiOGZkYjA4YTg0MmQzYzQ3YTk3N2YwMDA5MWNlM2Y4YTFmZiIsInZlcnNpb24iOjF9.dirG9kG6OdNi-YEMWHv0UMrHTjEt6VS9i6fRbbUeZd1OoP2fl6XcKoDIk6Us-cdiyVnCyyhWsMNsUufMAqLtDA
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: launch/gov_report
          type: launch/gov_report
          config: plain_text
          split: test
        metrics:
          - type: rouge
            value: 37.3609
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMGExYjM5ODRiNThlZTU4ZTdhM2ZlZWRlNTgzNzc3N2ZjODk2ZjdlOGZlMDkzNmU2Yjk1NzQzZjQ5YzkwODllMCIsInZlcnNpb24iOjF9.JQIeaQkG-IlinWoyc6FKJZUgpWfqOsDhludqm5MgVsw68gsjo0nSPp_Y_1q26Y4dulZOLlQLyBAm3mlCA8s5Ag
          - type: rouge
            value: 8.6943
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWFjNzJkMzViOGM5YWQ0OGQ4ZTg3NTE5MzU1MjZkZjZiZmVkYTk0ZDhkYjAxMjZiZDVkZTYyYjk4MzRjNTQ3YiIsInZlcnNpb24iOjF9.9XJZ2UF6XyZNNrtp-XOEXC6etoDOFLq1xlIoMFEM9Jinisq3kWguXBiqPQWImLKra5WBm7jU_QIX-Fvn8sP-DA
          - type: rouge
            value: 17.9106
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWQ1MTdmNThiM2FiOGRmZWRmOTNlYWMwYTU1YjRiNTRlMGEwYjBmMmQ0YjQ4MDBhNzMzZmZkNjk3NjU0YzRhMSIsInZlcnNpb24iOjF9.040nGV6pig0Rzq9vkN83ZVWQzyjcVi13L36v0QF-Nhziol_dPPhuvghTlGWXWHwj6amsKzyh8M7rNfwL2TcsAQ
          - type: rouge
            value: 33.8022
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDYwOGRmYzg4ODc2NDExNjhhMjI5MDg3MjI0YTQ5ZDRhM2NjN2Q2ZjM5YTIwZDIxNmY3Y2JlMmMxYTE5MDE4ZiIsInZlcnNpb24iOjF9.S1nynUjLz7z4gf-0WFfPs-ZuZubhN9kXyVSrYNzOdT2gTJmByQWasKreZkVSWus-HNAHR8DhzL6UUWxuDMmAAQ
          - type: loss
            value: 3.4974069595336914
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzkyNmU5NTRhMTkxNjA1M2E1MjdiMTE0MzQyMDc4ODBkNmM1NDg1ZDk4OTNjODk2MThlZGZiYzQxOGE1YzgwMiIsInZlcnNpb24iOjF9.H9Oo0VKvcqAHcVNvjeEPEhQe5HP0v614suyCv75tfFGaPSKTIe3UlBNDdGOtqfUxb2zUNaBQ8MkA66C_Fkq6CA
          - type: gen_len
            value: 243.3453
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWE1MGQzMDc2NDViOGM5ZmVkZjk0NmY0NzliOTBhMmE3NmY5MmUxMTI3NGE2OTQzM2Y1NjdmN2NlZGFlODFlYiIsInZlcnNpb24iOjF9.635fcTp_czTabJUVR_dwpzdkntb4cxEbODAC9MMTKrLKEf9NHqDBJXQ-nBOieW05iCSYzw_tEi8O-QW-sRxDAw
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: kmfoda/booksum
          type: kmfoda/booksum
          config: kmfoda--booksum
          split: test
        metrics:
          - type: rouge
            value: 35.2043
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTRlZTdjZDRlZGMxYzA2NmRkYjBiMzZkY2Q1ODUyYjJkM2QwOTRmMzA3ZmU5MDI5ZmM1MmZkZDUwNzc0NjhmNyIsInZlcnNpb24iOjF9.zrskApkmkhbfQLtlgjf_n6i3WmZcmkDH7Sd-JTzOYAU3yk1_Zl4paGdmpXvyQY48M71qWsBYtEKkhnzrkvCGBA
          - type: rouge
            value: 5.746
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2FlMjU2MzU1MTljZjM0ZmFhMmJlZDAxMTcwZDk3YWE5NjVjYjE0YmEyMTgzY2UyMTVmZDY5ZWM1YmM1ZDA5NSIsInZlcnNpb24iOjF9.5nDuOwa98pon3VW1TazB2Vw1uJgh6pfFMorzgLMJFvhgwYz6_MvLR1dDUeffP4eyw7rGZjBmf039AM7CyKEgCg
          - type: rouge
            value: 15.6794
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjRmYzk3NWFhZDVlODA4YWRiMDU1ZWFhZmMwMWE4MmNkNmNjZWM3ZjUwYzI3MWIxM2Y4MTlhZDk2ZTg5YjkyYSIsInZlcnNpb24iOjF9.TLflM2CYNgz4DNt-TwjgdkTL8ebKckTNnlPVsGLUUGqNI1CvSswzsPedqmntCfKVsH2YAsKsR4ZUb1HtJFsSAw
          - type: rouge
            value: 32.1129
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzdhNWE1YjRjNGUzYWYyNzM4MjIyYThiODJhODU2OGVlOTYxOGNhZmQ4Mjk2ZDUwNmU0MGQwNjQ5NTk2MzU4ZiIsInZlcnNpb24iOjF9.5yvTmPktBuyzoVNHn7UHcci3OrZLTm7e9d_lQkJq8UwzUuso1wHoy_gdvnvpn2DvUfdcBi5sXgG4mtFnVnGgBw
          - type: loss
            value: 2.945225238800049
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTgxNGRiN2RkMzQ5MjI2OGI0MTljZTY5ZDQyMzc5MjhmNzdhZWQ2NmJhYTgzOTRlMGY2YzkzZWE2NzVkYzVmNCIsInZlcnNpb24iOjF9.VkkP4-S6ZoozLj-iuY7tdsrSR0q1JLQXfgPv_0u2sJuv6x9RYMdCpfJHbqYbirV63b9w28USSwaAAMnz-LoJAA
          - type: gen_len
            value: 307.5493
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmQ1YTgxYmRhYWViYjhhNmYzNjdlYzVhMTNmZTBkY2RiOTRlMTUzNTIzY2RjOTNhMjRmNGRmYjQyNTBmZWRiMiIsInZlcnNpb24iOjF9.7ItU-AQXB4EEj9U9kJceteBQbA5MkZoegeLhCdpZepEaXzqr6Zg3yHLCD9zL_6Svb9uxuin678KOT5Zf-2YWCQ

long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP13

Evaluating some metric results before merging with the "main" wip version

This model is a fine-tuned version of pszemraj/long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP12 on the kmfoda/booksum.

The "base" checkpoint that I update when a training session is productive is here

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0006
  • train_batch_size: 2
  • eval_batch_size: 1
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 64
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.01
  • num_epochs: 1.1

Framework versions

  • Transformers 4.21.2
  • Pytorch 1.10.0+cu113
  • Datasets 2.4.0
  • Tokenizers 0.12.1