Answers.Body: <p>My guess is the reason for the stats on Huggingface Hub is in fact pretty random: something like that someone uses the particular dataset in a tutorial.</p>
  <p>Usually, it is considered a good idea to use the same data as previous work a paper is compared with. WTM16 was the first one where neural <a href="https://aclanthology.org/W16-2323" rel="nofollow noreferrer">MT models clearly dominated</a>, this might be part of the reason. There were bigger datasets in later years, but until the WMT16 data will be considered really small, some people will certainly prefer more direct comparability with previous work.</p>
  <p>The WMT14 dataset is also frequently used in papers. The <a href="https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf" rel="nofollow noreferrer">original Transformer paper</a> presented the results on WMT14 data and since that time people tend to compare their results with them.</p>
  <p>Also that the data is bigger does not necessarily have to be a good thing. University labs do not have as big computational resources as big companies and they need to trade-off the number of experiments and the training data size. Big, but not huge dataset such as WMT16 might be a good choice.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-20T10:13:01.943Z
Answers.LastActivityDate: 2022-01-20T10:13:01.943Z
Answers.UserDisplayName: Jindřich
Answers.Body: <p>Could it be also because in 2016 WMT became a conference and hence more &quot;trustworthy&quot; as a citation source? Until then WMT was a workshop. Maybe the older the papers, the more famous and studied they get :D</p>
  <p><a href="https://machinetranslate.org/wmt" rel="nofollow noreferrer">https://machinetranslate.org/wmt</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-03T16:59:04.483Z
Answers.LastActivityDate: 2022-05-03T16:59:04.483Z
Answers.UserDisplayName: Arrriba
Body: <p>The <a href="https://www.statmt.org/" rel="nofollow noreferrer"><strong>Workshop on Statistical Machine Translation</strong></a> has released translation challenges each year from 2004 on, which feature a dataset of sentence pairs in a variety of languages.</p>
  <p>Even though the conference has been taking place each year, with ever growing dataset sizes, the dataset from 2016 is still being used frequently as an evaluation dataset in scientific papers when <strong>evaluating machine translation models</strong> and it has by far <strong>the most downloads</strong> on the <a href="https://huggingface.co/datasets?sort=downloads&amp;search=wmt" rel="nofollow noreferrer">huggingface hub</a>.</p>
  <p>I would be grateful if someone could shed light on the reason for which the <strong>WMT16 dataset is so strongly favoured</strong> by the machine translation community.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-18T09:20:51.213Z
Comments.Text: That doesn’t answer the question. WMT16 featured seven languages, whereas e.g. WMT19 featured ten.
Comments.UserDisplayName: Zwiebak
Tags: <machine-translation><test-datasets>
Title: Why is the WMT16 dataset favoured for evaluating machine translation models?
UserDisplayName: Zwiebak
ThreadText: # Why is the WMT16 dataset favoured for evaluating machine translation models?
  
  (Asked by: username_0 on 2022-01-19)
  
  The [**Workshop on Statistical Machine Translation**](https://www.statmt.org/) has released translation challenges each year from 2004 on, which feature a dataset of sentence pairs in a variety of languages.
  
  Even though the conference has been taking place each year, with ever growing dataset sizes, the dataset from 2016 is still being used frequently as an evaluation dataset in scientific papers when **evaluating machine translation models** and it has by far **the most downloads** on the [huggingface hub](https://huggingface.co/datasets?sort=downloads&search=wmt).
  
  I would be grateful if someone could shed light on the reason for which the **WMT16 dataset is so strongly favoured** by the machine translation community.
  
  **Tags:** machine-translation, test-datasets
  > **Comments:**
  > **username_0**: That doesn’t answer the question. WMT16 featured seven languages, whereas e.g. WMT19 featured ten.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-01-20. Score: 0
  
  My guess is the reason for the stats on Huggingface Hub is in fact pretty random: something like that someone uses the particular dataset in a tutorial.
  
  Usually, it is considered a good idea to use the same data as previous work a paper is compared with. WTM16 was the first one where neural [MT models clearly dominated](https://aclanthology.org/W16-2323), this might be part of the reason. There were bigger datasets in later years, but until the WMT16 data will be considered really small, some people will certainly prefer more direct comparability with previous work.
  
  The WMT14 dataset is also frequently used in papers. The [original Transformer paper](https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf) presented the results on WMT14 data and since that time people tend to compare their results with them.
  
  Also that the data is bigger does not necessarily have to be a good thing. University labs do not have as big computational resources as big companies and they need to trade-off the number of experiments and the training data size. Big, but not huge dataset such as WMT16 might be a good choice.
  
  
  ---
  
  ### Answer by username_2 on 2022-05-03. Score: -1
  
  Could it be also because in 2016 WMT became a conference and hence more "trustworthy" as a citation source? Until then WMT was a workshop. Maybe the older the papers, the more famous and studied they get :D
  
  <https://machinetranslate.org/wmt>
  
  
  ---
  
  