How to use Huggingface model for continuous values directly?
Hi, I have a dataset which contains continuous values [ batch_size, features ] 

Features look like this :

`[0.49221584, -0.021571456, -0.0920076, -0.14408934, -0.62306774]`

I want to apply transformer model on these values and pass it to the final layer, something like this 

`batch_data ==> Transformer ==> output_layer ==> classification`

Currently, I am using hand-coded multi-head attention and norm with the feed-forward network to pass these values to the transformer block. 

I Gone through hugging face models, but all the models accept tokens and sequences, Is there any way/hack How I can use hugging face transformer models on direct continuous values?
you could input the `input_embeds` tokens instead of the `input_ids` to the forward pass and adjust the `hidden_size` accordingly. Or you just tweak the model files yourself and you remove `nn.Embeddings` and replace it by a dense layer. Btw, normally you get much better answers for these kind of questions when you post it on https://discuss.huggingface.co/ . We try to move "non-bug" questions to this forum :-) <|||||>@patrickvonplaten Thank you for the reply, Sure from onwards,  I'll post my doubts there.
If you could provide any quick template to start with continuous values, that’d be helpful.<|||||>Hi, @patrickvonplaten  I asked this question on discussion forum but didn't get any update yet. Can you provide any starter code/ Template where I can feed continuous values directly? 

https://discuss.huggingface.co/t/how-to-use-huggingface-model-for-continuous-values-directly/816<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
