How does one fine-tune parameters and weights at the same time?
I have been having my hands full with training a model to classify web pages. This is the first time ever that I am doing this, so I know very little about ML. I'm here to learn. :-) Currently, I am using a Pipeline with FeatureUnion to run a a GridSearchCV to tune my parameters. However, I am slightly confused about how this process ought to go. For the whole process of creating a predictive model I read this answer, which was helpful, but I still have some questions.
First of all: are parameters independent of your model? The answer linked above suggests first testing different models, and then do parameter tuning. However, I started with tuning my parameters on a Naive Bayes multinomial model, with the idea to test the models (e.g. svm, kNN, NBmn) afterwards with the 'optimal' parameters. But the answer above seems to suggest otherwise. 650
Does this mean that the parameter tuning depends on the best model you choose, which makes it useless to first tune the parameters?
Apart from that, I am also not sure how to correctly tune my parameters per feature. I have 13 features, and put them all in the Pipeline. For things like TfidfVectorizer for eight features, I'd like to tune max_features, ngram_range, and max_df. However, because this takes quite a long time to run I thought I'd start with two features, make sure I get the optimal value, set them in the Pipeline (not in the parameter tuning), and add a feature. However, I feel that this is not the way at all. Is it required to the complete tuning as a single step, or is it possible to do it in steps?
Lastly, I noticed that you can also pass the feature weights of FeatureUnion as a parameter, and tune it. But as far as I have found you can only add specific values to test (example below) and not a range (e.g. 0-1 with steps of 0.1), which makes it hard to set the parameter values as you've got to guess which could be a good value. Is there a better way?
parameters = {
'union__transformer_weights': [
{'f1':1, 'f2':0.4, 'f3': 0.2,'f4': 1,'f5': 0.4,'f6': 0.1},
{'f1':0.4, 'f2':0.9, 'f3': 0.2,'f4': 0.4,'f5': 0.4,'f6': 0.2},
{'f1':0.8, 'f2':0.7, 'f3': 0.2,'f4': 0.7,'f5': 0.7,'f6': 0.4}
]
}
Thank you so much. 458