How to output labels for GLUE test set
## ❓ Questions & Help

Thanks for your great work.

I have done some modification based on your library. I'd like to test it on GLUE test set (not dev set). Is there any way I can do it? Do you have a script for output GLUE submission files?

Thanks.
Hi, if you're working on a clone/fork of this library, then you can freely change the following lines in `transformers/data/processors/glue.py` (lines 254-262 on the current master):

```py
    def get_train_examples(self, data_dir):
        """See base class."""
        return self._create_examples(
            self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")

    def get_dev_examples(self, data_dir):
        """See base class."""
        return self._create_examples(
            self._read_tsv(os.path.join(data_dir, "dev.tsv")), "dev")
```

to read the `test.tsv` instead of the `dev.tsv`.<|||||>Hi @LysandreJik,

Thanks for your reply!
However, the format of test set is different from the dev set. It has an extra `id` field and does not have ground truth labels.  Thus, the simple change you suggested wouldn't work in this case.   Besides, GLUE leaderboard also requires a specific output format for submission.

I have noticed that in Huggingface's workshop paper [DistilBERT](https://arxiv.org/abs/1910.01108), you also use DEV set instead of TEST result from GLUE leaderboard. I guess it may be somehow related to the lack of a submission script. To facilitate further research, I hope you can make a script for GLUE submission. As far as I know, many researchers are using huggingface/transformers for their research. I really appreciate it if you can provide a script for the community.

Thanks!<|||||>Hi @LysandreJik,
I've just realized that you are one of the authors of DistilBERT so I suppose you know exactly what I mean. It can truly benefit the research community if you do this. Thanks.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>hi did u get the script? <|||||>@yuchenlin My co-author wrote one himself. I added you on Wechat but you didn't respond. <|||||>Hey, I think this would be helpful for many in the community. Can you maybe share the script you wrote for this via a pull request?<|||||>@Breakend Sure, I will verify the script and open a pull request.<|||||>Hello @JetRunner , no news about this script?
Thank you in advance<|||||>> Hello @JetRunner , no news about this script?
> Thank you in advance

Sorry, I totally forgot about this stuff. I'll notify you as soon as we upload it.

cc @michaelzhouwang<|||||>> Hello @JetRunner , no news about this script?
> Thank you in advance

Hi I have uploaded the script for prediction on GLUE benchmarks at:
https://github.com/JetRunner/BERT-of-Theseus/tree/master/glue_script
You can first replace the glue.py in src/transformers/data/processor/ and then use run_prediction.py.<|||||>> > Hello @JetRunner , no news about this script?
> > Thank you in advance
> 
> Hi I have uploaded the script for prediction on GLUE benchmarks at:
> https://github.com/JetRunner/BERT-of-Theseus/tree/master/glue_script
> You can first replace the glue.py in src/transformers/data/processor/ and then use run_prediction.py.

cc @Breakend @jibay <|||||>@JetRunner  @MichaelZhouwang  Thank you for the quick answer, i will check asap :)