readme: ---
  language: pt-br
  license: mit
  tags:
  - LegalNLP
  - NLP
  - legal field
  - python
  - word2vec
  - doc2vec
  ---
  
  
  # ***LegalNLP*** - Natural Language Processing Methods for the Brazilian Legal Language ⚖️
  
  ### The library of Natural Language Processing for Brazilian legal language, *LegalNLP*, was born in a partnership between Brazilian researchers and the legal tech [Tikal Tech](https://www.tikal.tech) based in São Paulo, Brazil. Besides containing pre-trained language models for the Brazilian legal language, ***LegalNLP*** provides functions that can facilitate the manipulation of legal texts in Portuguese and demonstration/tutorials to help people in their own work.
  
  You can access our paper by clicking [**here**](https://arxiv.org/abs/2110.15709). 
  
  If you use our library in your academic work, please cite us in the following way
  
      @article{polo2021legalnlp,
        title={LegalNLP--Natural Language Processing methods for the Brazilian Legal Language},
        author={Polo, Felipe Maia and Mendon{\c{c}}a, Gabriel Caiaffa Floriano and Parreira, Kau{\^e} Capellato J and Gianvechio, Lucka and Cordeiro, Peterson and Ferreira, Jonathan Batista and de Lima, Leticia Maria Paz and Maia, Ant{\^o}nio Carlos do Amaral and Vicente, Renato},
        journal={arXiv preprint arXiv:2110.15709},
        year={2021}
      }
  
  --------------
  
  ## Summary
  
  0. [Accessing the Language Models](#0)
  1. [ Introduction / Installing package](#1)
  2. [ Language Models (Details / How to use)](#2)
      1.  [ Word2Vec/Doc2Vec ](#2.1)
  3. [ Demonstrations / Tutorials](#3)
  4. [ References](#4)
  
  --------------
  
  <a name="0"></a>
  ## 0\. Accessing the Language Models
  
  
  All our models can be found [here](https://drive.google.com/drive/folders/1tCccOXPLSEAEUQtcWXvED3YaNJi3p7la?usp=sharing).
  
  Please contact *felipemaiapolo@gmail.com* if you have any problem accessing the language models. 
  
  --------------
  
  <a name="1"></a>
  ## 1\. Introduction / Installing package
  *LegalNLP* is promising given the scarcity of Natural Language Processing resources focused on the Brazilian legal language. It is worth mentioning that our library was made for Python, one of the most well-known programming languages for machine learning.
  
  
  You first need to install the HuggingFaceHub library running the following command on terminal
  ``` :sh
  $ pip install huggingface_hub
  ```
  
  Import `hf_hub_download`:
  
  ```python
  from huggingface_hub import hf_hub_download
  ```
  
  And then you can download our Word2Vec(SG)/Doc2Vec(DBOW) and Word2Vec(CBOW)/Doc2Vec(DM) by the following commands:
  
  ```python
  w2v_sg_d2v_dbow = hf_hub_download(repo_id = "Projeto/LegalNLP", filename = "w2v_d2v_dbow_size_100_window_15_epochs_20")
  w2v_cbow_d2v_dm = hf_hub_download(repo_id = "Projeto/LegalNLP", filename = "w2v_d2v_dm_size_100_window_15_epochs_20")
  ```
  
  --------------
  
  
  
  <a name="2"></a>
  ## 2\. Model Languages
  
  <a name="3.2"></a>
  ### 3.2\. Word2Vec/Doc2Vec
  
  Our first models for generating vector representation for tokens and
  texts (embeddings) are variations of the Word2Vec [1,
  2] and Doc2Vec [3] methods. In short, the
  Word2Vec methods generate embeddings for tokens5 and that somehow capture
  the meaning of the various textual elements, based on the contexts in which these
  elements appear. Doc2Vec methods are extensions/modifications of Word2Vec
  for generating whole text representations.
  
  Remember to at least make all letters lowercase. Please check our paper or [Gensim page](https://radimrehurek.com/gensim_3.8.3/models/doc2vec.html) for more details. Preferably use Gensim version 3.8.3.
  
  
  Below we have a summary table with some important information about the trained models:
  
  
  
  | Filenames       |  Doc2Vec | Word2Vec   | Size | Windows
  |:-------------------:|:--------------:|:--------------:|:--------------:|:--------------:|
  | ```w2v_d2v_dm*```     | Distributed Memory       (DM)             | Continuous Bag-of-Words (CBOW)          | 100, 200, 300 | 15 
  | ```w2v_d2v_dbow*``` | Distributed Bag-of-Words (DBOW)               | Skip-Gram (SG)                   | 100, 200, 300      | 15 
  
  
  Here we made available both models with 100 size and 15 window.
  
  #### Using *Word2Vec*
  
  Installing Gensim
  
  
  ```python
  !pip install gensim=='3.8.3' 
  ```
  
  Loading W2V:
  
  
  ```python
  from gensim.models import KeyedVectors
  
  #Loading a W2V model
  w2v=KeyedVectors.load(w2v_cbow_d2v_dm)
  w2v=w2v.wv
  ```
  Viewing the first 10 entries of 'juiz' vector
  
  
  ```python
  w2v['juiz'][:10]
  ```
  
  
  
  
      array([ 6.570131  , -1.262787  ,  5.156106  , -8.943866  , -5.884408  ,
             -7.717058  ,  1.8819941 , -8.02803   , -0.66901577,  6.7223144 ],
            dtype=float32)
  
  
  
  
  Viewing closest tokens to 'juiz'
  
  ```python
  w2v.most_similar('juiz')
  ```
  
  
  
  
      [('juíza', 0.8210258483886719),
       ('juiza', 0.7306275367736816),
       ('juíz', 0.691645085811615),
       ('juízo', 0.6605231165885925),
       ('magistrado', 0.6213295459747314),
       ('mmª_juíza', 0.5510469675064087),
       ('juizo', 0.5494943261146545),
       ('desembargador', 0.5313084721565247),
       ('mmjui