transformers pytorch bart text2text-generation en dataset:midas/krapivin dataset:midas/inspec dataset:midas/kptimes dataset:midas/duc2001 arxiv:1910.13461 arxiv:2312.10700 autotrain_compatible endpoints_compatible region:us
---
datasets:
- midas/krapivin
- midas/inspec
- midas/kptimes
- midas/duc2001
language:
- en
widget:
- text: 'Relevance has traditionally been linked with feature subset selection, but
    formalization of this link has not been attempted. In this paper, we propose two
    axioms for feature subset selection sufficiency axiom and necessity axiombased
    on which this link is formalized: The expected feature subset is the one which
    maximizes relevance. Finding the expected feature subset turns out to be NP-hard.
    We then devise a heuristic algorithm to find the expected subset which has a polynomial
    time complexity. The experimental results show that the algorithm finds good enough
    subset of features which, when presented to C4.5, results in better prediction
    accuracy.'
- text: In this paper, we investigate cross-domain limitations of keyphrase generation
    using the models for abstractive text summarization. We present an evaluation
    of BART fine-tuned for keyphrase generation across three types of texts, namely
    scientific texts from computer science and biomedical domains and news texts.
    We explore the role of transfer learning between different domains to improve
    the model performance on small text corpora.
---

# BART fine-tuned for keyphrase generation

<!-- Provide a quick summary of what the model is/does. -->

This is the <a href="https://huggingface.co/facebook/bart-base">bart-base</a> (<a href = "https://arxiv.org/abs/1910.13461">Lewis et al.. 2019</a>) model finetuned for the keyphrase generation task (<a href="https://arxiv.org/pdf/2312.10700.pdf">Glazkova & Morozov, 2023</a>) on the fragments of the following corpora: 

* Krapivin (<a href = "http://eprints.biblio.unitn.it/1671/1/disi09055%2Dkrapivin%2Dautayeu%2Dmarchese.pdf">Krapivin et al., 2009</a>)
* Inspec (<a href = "https://aclanthology.org/W03-1028.pdf">Hulth, 2003</a>)
* KPTimes (<a href = "https://aclanthology.org/W19-8617.pdf">Gallina, 2019</a>)
* DUC-2001 (<a href = "https://cdn.aaai.org/AAAI/2008/AAAI08-136.pdf">Wan, 2008</a>)
* PubMed (<a href = "https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=08b75d31a90f206b36e806a7ec372f6f0d12457e">Schutz, 2008</a>)
* NamedKeys (<a href = "https://joyceho.github.io/assets/pdf/paper/gero-bcb19.pdf">Gero & Ho, 2019</a>).

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("aglazkova/bart_finetuned_keyphrase_extraction")
model = AutoModelForSeq2SeqLM.from_pretrained("aglazkova/bart_finetuned_keyphrase_extraction")

text = "In this paper, we investigate cross-domain limitations of keyphrase generation using the models for abstractive text summarization.\
        We present an evaluation of BART fine-tuned for keyphrase generation across three types of texts, \
        namely scientific texts from computer science and biomedical domains and news texts. \
        We explore the role of transfer learning between different domains to improve the model performance on small text corpora."

tokenized_text = tokenizer.prepare_seq2seq_batch([text], return_tensors='pt')
translation = model.generate(**tokenized_text)
translated_text = tokenizer.batch_decode(translation, skip_special_tokens=True)[0]
print(translated_text)
```

#### Training Hyperparameters

The following hyperparameters were used during training:

* learning_rate: 4e-5
* train_batch_size: 8
* optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
* num_epochs: 6

**BibTeX:**

```
@InProceedings{10.1007/978-3-031-67826-4_19,
author="Glazkova, Anna
and Morozov, Dmitry",
title="Cross-Domain Robustness of Transformer-Based Keyphrase Generation",
booktitle="Data Analytics and Management in Data Intensive Domains",
year="2024",
publisher="Springer Nature Switzerland",
address="Cham",
pages="249--265"
}
```

