Okay, that makes a little sense to me. Let me see if I can put what you said in my own words, and you tell me if I understood correctly:
In the linear sense, an embedding is just PCA…
Then there are two cases… Let’s say a sequence of numerical data (like stock prices) and a sequence of characters in a sentence.
In the first case y = Ax is easy. But with the string of characters you have to one-hot-encode each character, so then you get a sequence of sparse vectors. This isn’t so easy to transform to a lower dimension because it’s not a simple column vector. So nn.Embedding is the solution to this problem?
So in my personal use case, I want to do anomaly detection through reconstruction error. I’m trying to train an autoencoder to reconstruct timeseries data. This, I think, falls into the first category where the sequence is a dense vector.
How does that affect my use of nn.Embedding?