pytorch tf electra pretraining transformers
.gitattributes README.md config.json pytorch_model.bin special_tokens_map.json tf_model.h5 tokenizer_config.json vocab.txt
To produce BioELECTRA, we pretrain ELECTRA on a corpus of over 20 million abstracts from PubMed.

How to use the discriminator in transformers:

    from transformers import ElectraForPreTraining, ElectraTokenizerFast
    import torch
    discriminator = ElectraForPreTraining.from_pretrained("molly-hayward/bioelectra-small-discriminator")
    tokenizer = ElectraTokenizerFast.from_pretrained("molly-hayward/bioelectra-small-discriminator")