Hi,
I’m trying to make sense out of the various versions of the bigpatent dataset.
In particular, the version in the huggingface’s dataset library is uncased and tokenized. Did you obtain that version out of the raw untokenized dataset here or it was provided by the authors?
My confusion comes from the fact that there are different versions of the dataset. For example, the Pegasus preserved casing for their experients. As far as I understood, their version of the dataset was obtained from the raw dataset by using this 1 preprocessing. However, I fail when trying to build the dataset*, even when considering a single code split only.
Also, the huggingface dataset seems to lack some (useless) sections (e.g. in relation to previous art) so I was wondering how you obtained the clean version of the dataset in the first place.
Can anybody help?
*I have seen from the TFDS issues that huggingface had the same problem when trying to replicate the results from Pegasus. Did you manage to solve that problem?