Data Science, Machine Learning.
between significant and insignificant tokens in the documents. A significant token in a document is a token that not only appears in the document often, but that is also relatively rare in the entire corpus. If a term appears many times in the corpus then it becomes less significant for this