For such thing a python dict will be perfect for the following reasons:

It is easy to use and make clean code
It will hold references to Tensors, so nothing will be copied.
If you save Tensors for inspection and don’t need gradient through these new computations, you can save feature(x_i).detach() to avoid keeping the autograd informations.
If you need gradients (for example if you save these in a dict to later add an l2 penalty to your loss) then don’t detach.

nn.Parameter is definitely not what you want as it should only be used to define new parameters in an nn.Module.
nn.Embedding is used more to learn the features associated with a given label. So even though you could use it, it would be cumbersome to fit to your use.