title: What to do first: Feature Selection or Model Parameters Setting?
body: <p>This is more of a theoretical question. I'm working with the scikit-learn package to perform some NLP task. Sklearn provides many methods to perform both feature selection and setting of a model parameters. I'm wondering what I should do first.</p>
  <p>If I use <a href="http://scikit-learn.org/stable/modules/feature_selection.html#univariate-feature-selection" rel="nofollow noreferrer">univariate feature selection</a>, it's pretty obvious that I should do feature selection first and, with the selected features, I then tunne the parameters of the estimator.</p>
  <p>But what if I want to use <a href="http://scikit-learn.org/stable/modules/feature_selection.html#recursive-feature-elimination" rel="nofollow noreferrer">recursive feature elimination</a>? Should I first set the parameters with <a href="http://scikit-learn.org/stable/modules/grid_search.html" rel="nofollow noreferrer">grid search</a> using ALL the original features and just then perform feature selection? Or perhaps I should select the features first (with the estimator's default parameters) and then set the parameters with the selected features?</p>
  <p><strong>EDIT</strong></p>
  <p>I'm having pretty much the same problem stated <a href="http://www.mail-archive.com/scikit-learn-general@lists.sourceforge.net/msg02355.html" rel="nofollow noreferrer">here</a>. By that time, there wasn't a solution to it. Does anyone know if it exists one now?</p>