transformers pytorch led text2text-generation summarization dataset:ilc arxiv:2004.05150 license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
datasets: ilc
license: apache-2.0
metrics:
- rouge
tags:
- summarization
widget:
- text: 'IN THE HIGH COURT OF JUDICATURE AT PATNA CRIMINAL MISCELLANEOUS No. 229121
    Arising Out of PS. Case No. 127 Year 2020 Thana DUMRAON District Buxar 1. Ramlal
    Goswami aged about 44 years Male S o Late Gauri Shankar 2. Dharmshila Devi @ Savita
    Devi aged about 35 years wife of Ramlal Both resident of village Badka Dhakaich
    P.S. Krishna Brahm District ... Petitioner s ... Opposite Party s The State of
    Bihar Appearance : For the Petitioner s For the State CORAM: HONOURABLE MR. JUSTICE
    AHSANUDDIN AMANULLAH ORAL JUDGMENT Mr. Manoj Kumar with Mr. Anil Kumar Roy Advocates
    Mr. Ram Sumiran Roy APP The matter has been heard via video conferencing. 2. Heard
    Mr. Manoj Kumar learned counsel along with Mr. Anil Kumar Roy learned counsel
    for the petitioners and Mr. Ram Sumiran Roy learned Additional Public Prosecutorfor
    the State. 3. Learned counsel for the petitioners submitted that he may be permitted
    to add alias name of petitioner no. 2 which is Savita Devi. 4. Prayer allowed.
    5. Let necessary correction be made in the cause title Date : 03 08 2021 Patna
    High Court CR. MISC. No. 229121 dt.03 08 2021 2 4 by learned counsel for the petitioners
    through e mode by day after tomorrow. 6. The petitioners apprehend arrest in connection
    with Dumraon PS Case No. 1220 dated 15.04.2020 instituted under Sections 406 420
    467 468 471 448 506 34 of the Indian Penal Code. 7. The allegation against the
    petitioners is that the informant who is the cousin brother of petitioner no.
    1 had bought land through the petitioner no. 1 but he was cheated both with regard
    to the rates as also that the same piece of land being sold by the petitioners
    to two different persons. 8. Learned counsel for the petitioners submitted that
    in the FIR itself it has been stated that the informant had sold his land at a
    much higher price than the price he was paying for the land which he alleges to
    have been negotiated by the petitioner no. 1 for him. Further it was submitted
    that all such dispute relating to money is a purely civil in nature for which
    criminal case is an abuse of the process of the Court. Learned counsel submitted
    that the informant being the first cousin of the petitioner no. 1 and having sold
    his land was very well aware of the ground realities and cannot take a stand that
    he was ignorant of what was the actual position. Further it was submitted that
    Patna High Court CR. MISC. No. 229121 dt.03 08 2021 3 4 the petitioners have filed
    a supplementary affidavit in which a categorical stand has been taken on oath
    that the petitioners have not sold the same piece of land to two different persons.
    Learned counsel submitted that the petitioners are simple citizens being husband
    and wife and have no other criminal antecedent. It was submitted that had the
    allegation been correct the other person aggrieved would also have filed a case
    and most importantly neither any name of any person has been taken nor details
    of any document that the same piece of land was transferred to two persons has
    been either mentioned or brought on record. 9. Learned APP submitted that the
    petitioners are alleged to have cheated the informant and have got the same piece
    of land registered in favour of two persons. 10. Having considered the facts and
    circumstances of the case and submissions of learned counsel for the parties in
    the event of arrest or surrender before the Court below within six weeks from
    today the petitioners be released on bail upon furnishing bail bonds of Rs. 25
    000 each with two sureties of the like amount each to the satisfaction of the
    learned Chief Judicial Magistrate Buxar in Dumrao PS Case No. 127 of 2020 subject
    to the conditions laid down in Patna High Court CR. MISC. No. 229121 dt.03 08
    2021 4 4 Section 438(2) of the Code of Criminal Procedure 1973 and furtherthat
    one of the bailors shall be a close relative of the petitioners andthat the petitioners
    shall cooperate with the Court and the police prosecution. Failure to cooperate
    shall lead to cancellation of their bail bonds. 11. It shall also be open for
    the prosecution to bring any violation of the foregoing conditions of bail by
    the petitioners to the notice of the Court concerned which shall take immediate
    action on the same after giving opportunity of hearing to the aforementioned terms.
    12. The petition stands disposed of Anjani '
---




# Longformer Encoder-Decoder (LED) fine-tuned on ILC

This model is a fine-tuned version of  [led-base-16384](https://huggingface.co/allenai/led-base-16384) on the [ILC](https://huggingface.co/datasets/d0r1h/ILC) dataset.

As described in [Longformer: The Long-Document Transformer](https://arxiv.org/pdf/2004.05150.pdf) by Iz Beltagy, Matthew E. Peters, Arman Cohan, *led-base-16384* was initialized from [*bart-base*](https://huggingface.co/facebook/bart-base) since both models share the exact same architecture. To be able to process 16K tokens, *bart-base*'s position embedding matrix was simply copied 16 times.

```Python

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "CPU"

checkpoint = "d0r1h/led-base-ilc"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint, return_dict_in_generate=True).to(device)
case = "......."
input_ids = tokenizer(case, return_tensors="pt").input_ids.to(device)
global_attention_mask = torch.zeros_like(input_ids)
global_attention_mask[:, 0] = 1
sequences = model.generate(input_ids, 
                           global_attention_mask=global_attention_mask).sequences
summary = tokenizer.batch_decode(sequences, 
                                 skip_special_tokens=True)
                                 
```

## Evaluation results

When the model is used for summarizing ILC documents(10 samples), it achieves the following results:

| Model |            rouge1-f  |  rouge1-p  |  rouge2-f   |  rouge2-p  | rougeL-f   |  rougeL-p  |
|:-----------:|:-----:|:-----:|:------:|:-----:|:------:|:-----:|
| led-ilc      | **42** |  **47** | **22**  | **24** | **39**  | **44** | 
| led-base          | 3     |  39     | 1      | 21     | 3      | 37 |

[This notebook](https://colab.research.google.com/github/d0r1h/Notebooks/blob/main/NLP/Summarization/led_base_ilc_summarization.ipynb) shows how *led* can effectively be used for downstream tasks such as summarization.
