In my experience it&#x27;s usually best to start with all the words. If you use a decent implementation that supports sparse vectors it&#x27;s no problem, certainly not for these sorts of data sizes.<p>Usually you&#x27;ll end up with a frequency threshold, but it&#x27;s usually best to trim at the very low end --- like, words occurring 5 or fewer times. Further over-fitting can be controlled with regularization and parameter averaging.