Hi,
I’ve seen that SpanBERT models are on the hub, but we haven’t added the model itself yet to the library.
This would be a great project actually:

contribute SpanBERT to HuggingFace Transformers, based on the modeling file. This will be relatively easy, as the authors already used HuggingFace’s implementation of BERT and tweaked it a little bit. The only difference is this class. We could then call the model SpanBertModel in the library, and add a SpanBertForPreTraining similar to BertForPreTraining that includes the heads necessary for pre-training.
add a script to the examples directory, which could be called run_span_mlm.py (similar to run_mlm.py). This can be based on the files defined here 1 (Facebook open-sourced everything!).

If anyone is interested in contributing, let me know!