I think @patrickvonplaten’s suggestion in https://github.com/huggingface/transformers/issues/6608 27 to use input_embeds would be the way to go here.
Or you could also try simply bucketizing your continuous values into buckets and embedding them, like tokens.