It should already give the embedding for each token. For example, “This is a test” has 4 tokens with the gpt2 tokenizer
tokenizer("This is a test")
And if you see the output of the pipeline, you get a list with 4 lists, each containing the embedding for each individual token.
data = pipe("This is a test")
print(len(data[0]))
>>> 4
print(len(len(data[0][0]))
>>> 768