Hello, I am Kazuma Takaoka.
I am a developer of Japanese morphological analyzers, Sudachi and SudachiPy.
We have recently started a project to use SudachiPy as a tokenizer for transformers, and we are planning to release models using Sudachi.

  
      

      GitHub
  

  
    

WorksApplications/SudachiTra 4

  Japanese tokenizer for Transformers. Contribute to WorksApplications/SudachiTra development by creating an account on GitHub.


  

  
    
    
  

  


We welcome your comments on our project.