Hi @guoziyuan
We’ve since built on the previous work in the paper “Incorporating Count-Based Features into Pre-Trained Models for Improved Stance Detection” (https://arxiv.org/pdf/2010.09078.pdf 64). The code for this work is available at https://github.com/Anushka-Prakash/RumourEval-2019-Stance-Detection/ 100
This work outperforms a RoBERTa baseline and achieved state-of-the-art results in stance detection by solving these problems (from paper):

Pre-trained models, such as BERT, are often trained for between 2 and 5 epochs during fine-tuning whereas simpler feature based models need to be trained for much longer. Our experiments show that a simple ensemble of these models results in over-fitting
There are likely to be too many features to directly ensemble the raw features with pre-trained models (resulting in too much noise), a loss of important - task specific - information when using dimensionality reduction methods, and too few output classes to use only the outputs of a feature based model in an ensemble (lack of information).