Sorry that I don’t have a proper answer and only more of an opinion piece on sentiment analysis

Traditional sentiment analysis is often just about the polarity/valence, e.g., three classes: positive, neutral, negative. And even then, the results of state-of-the art solutions might still be not much higher than 60% for some datasets. So having 12 classes makes it much more challenging particularly given that you treat each class as independent which they are not here.
Without seeing the data, I would assume that lyrics using often flowery/poetic language, a lot of subtext, stylistic devices, etc, where the correct sentiment is not necessary explicit given the text but more implied by the listener. For example, in “This morning I was happy. Then I woke up.” we kind of know that the singer was only dreaming about being happy. A sentiment classifier would probably say that is positive, though. This is why a song about stalking has become one of the most popular wedding songs (Police - Every Breath You Take)
Sentiment is highly subjective. Given a text, different people often assign different polarities based on their interpretation. For example, “I wish I would be with you” is for some positive (singer is in love) and for others negative (singer hates to be separated), and both can are right for their point of view. When you introduce arousal, things get only more complicated.
Where does the dataset come from? Who annotated it? Multiple people, and if so, what was the interannotator agreement? These are all important questions for sentiment analysis.
Be careful with pretrained word embeddings. For example, in Word2Vec two words are similar if they are used in the same context. But this is often true for antonyms. For example, both “ugly” and “beautiful” are commonly used to describe appearances, so they are kind of similar w.r.t. to Word2Vec but obviously have different polarities.
You don’t mention it, but in case you do remove stopwords, don’t remove “not” “n’t” and stuff. They are pretty important for sentiment analysis :).
Given that you want to do regression (or 12 classes), 11k samples might not be that much. Can you at least overtrain your networks, maybe even with just 100 samples? That is, can you get your training accuracy to almost 1 for a small dataset. That is the first and easiest test to see if the network is training anything meaningful.
Have you tried more traditional ML approaches for regression or classification. For text (and smaller datasets), an SVM might perform much better then a neural network. And you can better interpret any errors the classifier is making
What dies “I am using as input 4 lyrics for each song”? Do you mean 4 lines of each song?

Some things that I would do to see what’s going on:

Double-check that the preprocessing does not remove anything important for sentiment analysis (particularly anything related to negation).
Simplify the annotations to a 3-class problem and check if the results make sense
Try to overtrain the model(s) with a small dataset to see if the training accuracy goes to 1.
In case there annotation from multiple users, only use those where all or most annotators agree on to avoid highly subjective/controversial lyrics.

Well, I hope there was anything useful in my rambling.