readme: ---
  license: mit
  ---
  # Bangla FastText Model
  This is a FastText pre-trained model for the Bengali language.
  
  This model is build for [bnlp](https://github.com/sagorbrur/bnlp) package.
  
  ## Datasets
  - [Wikipedia dump datasets](https://dumps.wikimedia.org/bnwiki/latest/)
  
  ## Training Details
  - Fasttext trained with total words = 20M, vocab size = 1171011, epoch=50, embedding dimension = 300
  
  ## Evaluation Details
  - training loss = 0.318668
  
  ## Usage
  - `pip install -U bnlp_toolkit`
  - `pip install fasttext==0.9.2` 
  - Generate Vector Using Pretrained Model
   ```py
   from bnlp.embedding.fasttext import BengaliFasttext
  
   bft = BengaliFasttext()
   word = "গ্রাম"
   model_path = "bengali_fasttext_wiki.bin"
   word_vector = bft.generate_word_vector(model_path, word)
   print(word_vector.shape)
   print(word_vector)
   ```
        
  - Train Bengali FastText Model
  
  ```py
  from bnlp.embedding.fasttext import BengaliFasttext
  
  bft = BengaliFasttext()
  data = "raw_text.txt"
  model_name = "saved_model.bin"
  epoch = 50
  bft.train(data, model_name, epoch)
  ```
  
  - Generate Vector File from Fasttext Binary Model
  ```py
  from bnlp.embedding.fasttext import BengaliFasttext
  
  bft = BengaliFasttext()
  
  model_path = "mymodel.bin"
  out_vector_name = "myvector.txt"
  bft.bin2vec(model_path, out_vector_name)
  ```