The use of Bidirectional Encoder Representations from Transformers (BERT) models for different natural language processing (NLP) tasks, and for sentiment analysis in particular, has become very popular in recent years and not in vain. The use of social media is being constantly on the rise. Its impact on all areas of our lives is almost inconceivable. Researches show that social media nowadays serves as one of the main tools where people freely express their ideas, opinions, and emotions. During the current Covid-19 pandemic, the role of social media as a tool to resonate opinions and emotions, became even more prominent. This paper introduces HeBERT and HebEMO. HeBERT is a transformer-based model for modern Hebrew text . Hebrew is considered a Morphological Rich Language (MRL) , with unique characteristics that pose a great challenge in developing appropriate Hebrew NLP models . Analyzing multiple specifications of the BERT architecture, we come up with a language model that outperforms all existing Hebrew alternatives on multiple language tasks. HebEMO is a tool that uses HeBERT to detect polarity and extract emotions from Hebrew user-generated content (UGC), which was trained on a unique Covid-19 related dataset that we collected and annotated for this study. Data collection and annotation followed an innovative iterative semi-supervised process that aimed to maximize predictability. HebEMO yielded a high performance of weighted average F1-score = 0.96 for polarity classification. Emotion detection reached an F1-score of 0.78-0.97  , with the exception of \textit{surprise , which the model failed to capture (F1 = 0.41). These results are better than the best-reported performance, even when compared to the English language .
 This paper introduces HeBERT and HebEMO. HeBERT is a Transformer-based model for modern Hebrew text , which relies on a BERT (Bidirectional Encoder Representations for Transformers) architecture. BERT has been shown to outperform alternative architectures in sentiment analysis, and is suggested to be particularly appropriate for MRLs . Analyzing multiple BERT specifications, we find that while model complexity correlates with high performance on language tasks that aim to understand terms in a sentence, a more-parsimonious model better captures the sentiment of entire sentence. Either way, out BERT-based language model outperforms all existing Hebrew alternatives on all common language tasks. HebEMO is a tool that uses HeBERT to detect polarity and extract emotions from Hebrew UGC. HebEMO is trained on a unique Covid-19-related UGC dataset that we collected and annotated for this study. Data collection and annotation followed an active learning procedure that aimed to maximize predictability. We show that HebEMO yields a high F1-score of 0.96 for polarity classification. Emotion detection reaches F1-scores of 0.78-0.97 for various target emotions , with the exception of \textit{surprise , which the model failed to capture (F1 = 0.41). These results are better than the best-reported performance, even among English-language models of emotion detection .
D
The use of Bidirectional Encoder Representations from Transformers (BERT) models for different natural language processing (NLP) tasks, and for sentiment analysis in particular, has become very popular in recent years and not in vain. The use of social media is being constantly on the rise. Its impact on all areas of our lives is almost inconceivable. Researches show that social media nowadays serves as one of the main tools where people freely express their ideas, opinions, and emotions. During the current Covid-19 pandemic, the role of social media as a tool to resonate opinions and emotions, became even more prominent.
R
transformer-based
Transformer-based
R
. Hebrew is considered a Morphological Rich Language (MRL) , with unique characteristics that pose a great challenge in developing appropriate Hebrew NLP models
, which relies on a BERT (Bidirectional Encoder Representations for Transformers) architecture. BERT has been shown to outperform alternative architectures in sentiment analysis, and is suggested to be particularly appropriate for MRLs
R
specifications of the BERT architecture, we come up with a language model that
BERT specifications, we find that while model complexity correlates with high performance on language tasks that aim to understand terms in a sentence, a more-parsimonious model better captures the sentiment of entire sentence. Either way, out BERT-based language model
R
multiple
all common
R
user-generated content (UGC), which was
UGC. HebEMO is
R
Covid-19 related
Covid-19-related UGC
R
innovative iterative semi-supervised process
active learning procedure
R
HebEMO yielded a high performance of weighted average
We show that HebEMO yields a high
R
=
of
R
reached an F1-score
reaches F1-scores
A
for various target emotions
R
surprise
surprise
R
when compared to the English language
among English-language models of emotion detection