Very interesting @nielsr @BramVanroy . Thanks. Its great to know what huggingface has a consistent linear classification head in all cases.
If my understanding is correct, then the linear classification head actually learns a big matrix whose dimensions are rows * columns = vocabulary size * classes.
Is that right? Thanks!