  In this paper, we assess the ability of BERT and its derivative models
(RoBERTa, DistilBERT, and ALBERT) for short-edits based humor grading. We test
these models for humor grading and classification tasks on the Humicroedit and
the FunLines dataset. We perform extensive experiments with these models to
test their language modeling and generalization abilities via zero-shot
inference and cross-dataset inference based approaches. Further, we also
inspect the role of self-attention layers in humor-grading by performing a
qualitative analysis over the self-attention weights from the final layer of
the trained BERT model. Our experiments show that all the pre-trained BERT
derivative models show significant generalization capabilities for
humor-grading related tasks.

Computation and Language