readme: ---
  language: en
  datasets:
   - scientific_papers
   
  license: apache-2.0
  ---
  
  ## Introduction
  
  [Allenai's Longformer Encoder-Decoder (LED)](https://github.com/allenai/longformer#longformer).
  
  This is the official *led-large-16384* checkpoint that is fine-tuned on the arXiv dataset.*led-large-16384-arxiv* is the official fine-tuned version of [led-large-16384](https://huggingface.co/allenai/led-large-16384). As presented in the [paper](https://arxiv.org/pdf/2004.05150.pdf), the checkpoint achieves state-of-the-art results on arxiv
  
  ![model image](https://raw.githubusercontent.com/patrickvonplaten/scientific_images/master/led_arxiv_result.png)
  
  ## Evaluation on downstream task
  
  [This notebook](https://colab.research.google.com/drive/12INTTR6n64TzS4RrXZxMSXfrOd9Xzamo?usp=sharing) shows how *led-large-16384-arxiv* can be evaluated on the [arxiv dataset](https://huggingface.co/datasets/scientific_papers)
  
  ## Usage
  
  The model can be used as follows. The input is taken from the test data of the [arxiv dataset](https://huggingface.co/datasets/scientific_papers).
  
  ```python 
  LONG_ARTICLE = """"for about 20 years the problem of properties of
  short - term changes of solar activity has been
  considered extensively . many investigators
  studied the short - term periodicities of the
  various indices of solar activity . several
  periodicities were detected , but the
  periodicities about 155 days and from the interval
  of @xmath3 $ ] days ( @xmath4 $ ] years ) are
  mentioned most often . first of them was
  discovered by @xcite in the occurence rate of
  gamma - ray flares detected by the gamma - ray
  spectrometer aboard the _ solar maximum mission (
  smm ) . this periodicity was confirmed for other
  solar flares data and for the same time period
  @xcite . it was also found in proton flares during
  solar cycles 19 and 20 @xcite , but it was not
  found in the solar flares data during solar cycles
  22 @xcite . _    several autors confirmed above
  results for the daily sunspot area data . @xcite
  studied the sunspot data from 18741984 . she found
  the 155-day periodicity in data records from 31
  years . this periodicity is always characteristic
  for one of the solar hemispheres ( the southern
  hemisphere for cycles 1215 and the northern
  hemisphere for cycles 1621 ) . moreover , it is
  only present during epochs of maximum activity (
  in episodes of 13 years ) .
  similarinvestigationswerecarriedoutby + @xcite .
  they applied the same power spectrum method as
  lean , but the daily sunspot area data ( cycles
  1221 ) were divided into 10 shorter time series .
  the periodicities were searched for the frequency
  interval 57115 nhz ( 100200 days ) and for each of
  10 time series . the authors showed that the
  periodicity between 150160 days is statistically
  significant during all cycles from 16 to 21 . the
  considered peaks were remained unaltered after
  removing the 11-year cycle and applying the power
  spectrum analysis . @xcite used the wavelet
  technique for the daily sunspot areas between 1874
  and 1993 . they determined the epochs of
  appearance of this periodicity and concluded that
  it presents around the maximum activity period in
  cycles 16 to 21 . moreover , the power of this
  periodicity started growing at cycle 19 ,
  decreased in cycles 20 and 21 and disappered after
  cycle 21 . similaranalyseswerepresentedby + @xcite
  , but for sunspot number , solar wind plasma ,
  interplanetary magnetic field and geomagnetic
  activity index @xmath5 . during 1964 - 2000 the
  sunspot number wavelet power of periods less than
  one year shows a cyclic evolution with the phase
  of the solar cycle.the 154-day period is prominent
  and its strenth is stronger around the 1982 - 1984
  interval in almost all solar wind parameters . the
  existence of the 156-day periodicity in sunspot
  data were confirmed by @xcite . they considered
  the possible relation between the 475-day (
  1.3-year ) and 156-day periodicities . the 475-day
  ( 1.3-year ) periodicity was also detected in
  variations of the interplanetary magnetic field ,
  geomagnetic activity helioseismic data and in the
  solar wind speed @xcite . @xcite concluded that
  the region of larger wavelet power shifts from
  475-day ( 1.3-year ) period to 620-day ( 1.7-year
  ) period and then back to 475-day ( 1.3-year ) .
  the periodicities from the interval @xmath6 $ ]
  days ( @xmath4 $ ] years ) have been considered
  from 1968 . @xcite mentioned a 16.3-month (
  490-day ) periodicity in the sunspot numbers and
  in the geomagnetic data . @xcite analysed the
  occurrence rate of major flares during solar
  cycles 19 . they found a 18-month ( 540-day )
  periodicity in flare rate of the norhern
  hemisphere . @xcite confirmed this result for the
  @xmath7 flare data for solar cycles 20 and 21 and
  found a peak in the power spectra near 510540 days
  . @xcite found a 17-month ( 510-day ) periodicity
  of sunspot groups and their areas from 1969 to
  1986 . these authors concluded that the length of
  this period is variable and the reason of this
  periodicity is still not understood . @xcite 