Ok, but I’m not quite figuring out how can I ‘train’ without having some fixed representation to feed the neural network… I mean in such a case, should I use nn.Embedding(num_samples, size) . Could you provide a little example of code please?
Also, I guess that the problem in using the first option, is that we won’t allow any ‘inductive’ ability on the model, as the predictions will be constrained to the specific trainable embeddings we have provided at the beginning, i.e. if I provide a new input, we won’t know which features to assign it. Is that correct?