I’m glad to help you. Embedding layer use LongInt, if you want float type you need another solution
, but Transformers usually used with sequence of tokens that repeating like DNA, RNA, text, image, i not suggest use case for transformers other than text, image or biology.
Quick update:
You can look this post on reddit.

  
      
      reddit
  
  
    

r/MachineLearning - [D] Transformers for time series data 71

8 votes and 17 comments so far on Reddit