When is the input of a transformer a matrix? I have a question about the format of the input of the transformers, depicted in the image below:
(taken from the [page](https://
blogs.oracle.com/datascience/multi-head-self-attention-in-nlp)).

When is the input X a matrix?
I think in the question answering problem area X is just a vector and in other tasks like recommendation X is a matrix, but I am not sure...

X is a matrix as every token in the input sentence is encoded in a vector.
For example, if the input sentence is "I like trains" each token (assume each word is a token) gets transformed to a vector.
So, for example, "I" --> [0.2, 0.1, 0, 0], "like" --> [0.1, 1, 0, 1], "trains" --> ...
Now you have a 3x4 matrix.
Than, positional encoding is added (without changing the matrix shape) and you get your X input matrix
