If your task is closely related to those numbers, and all examples follow the same pattern, perhaps you should just use the price and surface as raw features in your model, along with the text. You could concatenate the price and surface values to the Transformer embedding for the text at the last hidden-layer before classification. I am not sure if that is what you meant by



 bone:

Would it make sense to manually convert these numbers and pass in its own vector?


Maybe also normalize the surface values to one standard.
I don’t know if there are better ways to deal with values at tokenizing, but this would be my suggestion.