Creating a complex featureset for regression modeling
I am currently working a on project that requires me to convert all of the categorical variable to continuous (or binary) variables to build a regression model. The problem is that I am more than 100 categorical values and all of them have different measures. For example, let's say that users who have different types of cars and all of those car models have associated continuous variables such as the price of the cars, size of gas tank, avg price of the gas to fill the gas tank and etc. Here is the example of this dataset below:
| user | car_make| car_model| price| gas_tank| full_gas_tank_price| | -------- | -------------- |----|--|--| --| | 123_a | Honda | civic| 17000| 12| 19| | 345_b | Lexus | RX| 45000| 16| 35|
One way to rebuild this featureset to only contain continuous variables is have the car_make and car_models as columns that each could contain the price, gas_tank and full_gas_tank_price values. For example:
| user | Honda_price| honda_gas_tank| honda_full_gas_tank_oprice| Lexus_price| Lexus_gas_tank| Lexus_full_gas_tank_price| | -------- | -------------- |----|--|--| --|--| | 123_a | 17000 | 12| 19| 0| 0|0| | 345_b | 0 | 0| 35| 45000| 16|35|
(I have not included the columns for civic model and RX model because the table is going to be wide). So, if I create the table like above, the table is going to include too many columns. 856
Becuse for each car_make and also car_model, there are 3 different features. So, I decided to create my featureset in a way that all of the car_make and car_models are binary variables and then have other continuous features as they are. Here is the example of the table:
| user | Honda| Lexus| civic| RX| price| gas_tank| full_gas_tank_price| | -------- | -------------- |----|--|--| --| --|--| | 123_a | 1 | 0| 1| 0| 17000| 12|19| | 345_b | 0 | 1| 0| 1| 45000| 16|35|
This sounded reasonable to me because I would have less features, hence, better performance. However, I was a bit worried that it would affect the model's accuracy because this table does not specifically show the price of honda. Is just shows that the user has honda and then the price is on another column. What do you think?
(I understand that I can potentially remove features that don't correlated with the output, but let's assume that all of the existing features correlate with the output but there are just too many of them) 946