AllenNLP SPECTER model
# 🌟 New model addition

SPECTER: Document-level Representation Learning using Citation-informed Transformers

## Model description

[SPECTER](https://arxiv.org/pdf/2004.07180.pdf) by AllenAI is a new method to generate document-level embedding of scientific documents based on pretraining a Transformer language model on a powerful signal of document-level relatedness: the citation graph. It is based on SciBERT.

This model is showing us promising results for scientific paper similarity tasks. However, the repo is quite rough around the edges and inference performance could be better. According to the AllenNLP repo issues, they do not plan to adapt the models to ONNX runtime.

It would be great to have a plug'n'play implementation for Transformers. Instructions on how to do this are also welcome 🤝 

## Open source status

* [x] the model implementation is available: https://github.com/allenai/specter 
* [x] the model weights are available: see repo above 
* [x] who are the authors: AllenAI, @armancohan, @sergeyf 

Wonderful if we can use embeddings of scientific documents in transformers.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>SPECTER is a great work. Is there any plan to have the Huggingface version of SPECTER?<|||||>This should be addressed now: https://huggingface.co/allenai/specter
How to use: https://github.com/allenai/specter