Naive Bayes Classifiers and One-hot Encoding of Categorical Variables
<|reference_start|>Naive Bayes Classifiers and One-hot Encoding of Categorical Variables: This paper investigates the consequences of encoding a $K$-valued categorical variable incorrectly as $K$ bits via one-hot encoding, when using a Na\"{\i}ve Bayes classifier. This gives rise to a product-of-Bernoullis (PoB) assumption, rather than the correct categorical Na\"{\i}ve Bayes classifier. The differences between the two classifiers are analysed mathematically and experimentally. In our experiments using probability vectors drawn from a Dirichlet distribution, the two classifiers are found to agree on the maximum a posteriori class label for most cases, although the posterior probabilities are usually greater for the PoB case.<|reference_end|>
@article{williams2024naive,
    title={Naive Bayes Classifiers and One-hot Encoding of Categorical Variables},
    author={Christopher K. I. Williams},
    journal={arXiv preprint arXiv:2404.18190},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.18190},
    primaryClass={cs.LG stat.ML}
}