AnasAlokla/multilingual_go_emotions_V1.1

Overview

Model trained from bert-base-multilingual-cased on the multilingual_go_emotions dataset for multi-label classification.

Multilingual GoEmotions Chatbot Task Report

Author: Anas Hamid Alokla

This repository/space contains details about a multilingual emotion classification model and chatbot based on the GoEmotions dataset.

Links

• Live Demo: https://huggingface.co/spaces/AnasAlokla/test_emotion_chatbot
• Dataset (Supports 6 Languages): https://huggingface.co/datasets/AnasAlokla/multilingual_go_emotions
• Model Used: https://huggingface.co/AnasAlokla/multilingual_go_emotions
• GitHub Code: https://github.com/anasAloklah/emotion_chatbot

Test Set Performance

The following table shows the performance metrics of the fine-tuned model on the test set, broken down by emotion category.

The table below shows the performance of the test model:

Performance of Test Model (using class weight)

Labels	accuracy	precision	recall	f1	mcc	support	threshold
admiration	0.933	0.598	0.668	0.631	0.596	2790	0.15
amusement	0.967	0.682	0.793	0.733	0.718	1866	0.10
anger	0.952	0.327	0.356	0.341	0.317	1128	0.15
annoyance	0.908	0.223	0.301	0.256	0.211	1704	0.10
approval	0.920	0.351	0.288	0.317	0.276	2094	0.15
caring	0.970	0.381	0.303	0.337	0.325	816	0.20
confusion	0.959	0.359	0.390	0.374	0.353	1020	0.25
curiosity	0.933	0.405	0.552	0.467	0.438	1734	0.10
desire	0.984	0.385	0.420	0.402	0.394	414	0.30
disappointment	0.958	0.278	0.216	0.243	0.224	1014	0.40
disapproval	0.920	0.221	0.343	0.269	0.235	1398	0.10
disgust	0.972	0.302	0.383	0.338	0.326	600	0.15
embarrassment	0.991	0.388	0.346	0.366	0.362	240	0.45
excitement	0.968	0.248	0.333	0.285	0.272	624	0.10
fear	0.985	0.501	0.526	0.513	0.506	498	0.20
gratitude	0.988	0.913	0.894	0.903	0.897	2004	0.35
grief	0.999	0.529	0.250	0.340	0.363	36	0.85
joy	0.959	0.381	0.472	0.422	0.403	1032	0.15
love	0.971	0.715	0.789	0.750	0.736	1812	0.25
nervousness	0.996	0.430	0.283	0.342	0.347	120	0.70
optimism	0.971	0.573	0.423	0.487	0.478	1062	0.45
pride	0.997	0.468	0.262	0.336	0.349	84	0.25
realization	0.967	0.220	0.146	0.176	0.163	792	0.25
relief	0.993	0.117	0.094	0.104	0.102	138	0.10
remorse	0.987	0.586	0.638	0.611	0.605	516	0.20
sadness	0.960	0.415	0.519	0.461	0.444	1062	0.15
surprise	0.975	0.518	0.425	0.467	0.457	828	0.60
neutral	0.733	0.582	0.621	0.601	0.401	10524	0.10

Test Model Performance (Threshold = 0.5)

The table below shows the performance of the test model with a threshold of 0.5:

Labels	accuracy	precision	recall	f1	mcc	support	threshold
admiration	0.939	0.673	0.570	0.617	0.587	2790	0.5
amusement	0.967	0.735	0.666	0.699	0.682	1866	0.5
anger	0.961	0.400	0.264	0.318	0.306	1128	0.5
annoyance	0.940	0.328	0.137	0.194	0.185	1704	0.5
approval	0.931	0.432	0.211	0.283	0.269	2094	0.5
caring	0.973	0.431	0.246	0.314	0.313	816	0.5
confusion	0.963	0.401	0.337	0.366	0.349	1020	0.5
curiosity	0.944	0.463	0.361	0.406	0.380	1734	0.5
desire	0.985	0.409	0.384	0.396	0.389	414	0.5
disappointment	0.961	0.300	0.198	0.239	0.224	1014	0.5
disapproval	0.945	0.293	0.195	0.234	0.212	1398	0.5
disgust	0.978	0.376	0.267	0.312	0.306	600	0.5
embarrassment	0.991	0.392	0.333	0.360	0.357	240	0.5
excitement	0.977	0.348	0.204	0.257	0.255	624	0.5
fear	0.986	0.547	0.468	0.504	0.499	498	0.5
gratitude	0.988	0.925	0.879	0.902	0.896	2004	0.5
grief	0.999	0.400	0.278	0.328	0.333	36	0.5
joy	0.966	0.451	0.367	0.405	0.389	1032	0.5
love	0.971	0.742	0.747	0.744	0.729	1812	0.5
nervousness	0.996	0.382	0.283	0.325	0.327	120	0.5
optimism	0.971	0.583	0.413	0.484	0.477	1062	0.5
pride	0.997	0.500	0.190	0.276	0.308	84	0.5
realization	0.971	0.270	0.124	0.170	0.169	792	0.5
relief	0.995	0.125	0.029	0.047	0.058	138	0.5
remorse	0.988	0.644	0.560	0.599	0.594	516	0.5
sadness	0.968	0.512	0.408	0.454	0.441	1062	0.5
surprise	0.974	0.492	0.430	0.459	0.447	828	0.5
neutral	0.742	0.648	0.440	0.524	0.368	10524	0.5