Hey @Winstead,
Thanks for your post here! I think it would be a good idea to use Wav2Vec2 for emotion classification. I won’t find time to fine-tune the model myself any time soon, but it should be rather straightforward to do so. Things that need to be done before being able to fine-tune Wav2Vec2 on emotion classification.



Add a Wav2Vec2ForSpeechClassification model that would be very similar to how BertForSequenceClassification is implemented.




It would be probably much easier to train the model if the two datasets you linked above would be added to datasets. It should be rather straight-forward to do this yourself or else you can put up a dataset request issue in the library and maybe someone in the open-source community is interested in tackling the issue. See those issues for example: Issues · huggingface/datasets · GitHub 15




Having added those things it should be rather straight-forward to train the model. You can look for “transformers” sentiment analysis online to get a feeling for how it should be done. See this article for example: https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/ 36