How to extract numerical features that can separate well documents belonging to two different classes? I have a group of texts belonging to two different classes. I would like to extract numerical features that can separate well the two classes.
Right now I implemented a classic TF-IDF with a document for each text. This method may be not particularly useful in my opinion since it does not take into account the different classes but just looks at the important words in the texts.
Therefore I also implemented a TF-IDF fitted with a document for each class (so basically I combined all the texts of the same class together). I have never read about this, it is just my intuition, but I'm not sure it's correct.
I wonder if there are approaches similar to TF-IDF that extract the most important words for each document and also for each class.
Thank you!

I would argue that your approach should not consider the class to which the document belongs.
Your approach would go something like this:

*

*Determine the class to which the document belongs.


*Do the class-specific feature extraction.


*Use the extracted features to predict the class to which the document belongs.
If you already know the class that lets you do class-specific feature extraction in step #2, you're done. You don't need to extract features. You know the answer with $100\%$ certainty. And if you don't know the class, then you can't do step #2.
For a concrete example, consider having news articles about politics and news articles about the Olympics. Based on the category to which each document belongs, you do category-specific feature extraction.
Along comes a new article, and you don't know if it is about politics or the Olympics. Do you do the feature extraction for political articles or articles about the Olympics?
