I think you could consider something similar to this line of work to create a stock embeddings:

  
      

      ACL Anthology
  

  
    

Stock Embeddings Acquired from News Articles and Price History, and an... 5

  Xin Du, Kumiko Tanaka-Ishii. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020.


  

  
    
    
  

  


Probably you want to ignore the news in your problem set but I think they are still quite useful.