Is it common practice to use the most frequent words as features ? It looks like they don&#x27;t carry much information, by definition. As a first naive approach, I&#x27;d rank the words by the inverse of how many categories they appear in, factor in the overall frequency, with a weight or something more clever and then take the top N.<p>Well, thinking more about it leads me to tf-idf and naive bayes (of course), at which point you pretty much already have a classifier. So it seems feature selection <i>is</i> learning in itself and defines the maximum accuracy you&#x27;ll be able to reach ? This is border philosophical but I&#x27;d love to read more about these matters. Pointers welcome !