Question: I ran an OLS model on a dataset with 2 categorical variables. One of them was gender. The other one had 3 different categories. I used one-hot encoding for it during pre-processing before running my model. 

Variables in the image: Embarked\_C and Embarked\_Q.
The results showed a p-value for Embarked\_Q as 0.785. In this case, should I remove both Embarked\_Q and C or just Q?

[![Regression Results](https://i.stack.imgur.com/83Hl5.png)](https://i.stack.imgur.com/83Hl5.png)

Answer: 
You should keep all of levels as they collectively describe the feature. Removing the insignificant ones will bias your coefficients and distort your interpretation (e.i. change the reference level). 

Here are some stats.exchange references:

* [Is it advisable to drop certain levels of a categorical variable?](https://stats.stackexchange.com/questions/141063/is-it-advisable-to-drop-certain-levels-of-a-categorical-variable)
* [Can I ignore coefficients for non-significant levels of factors in a linear model?](https://stats.stackexchange.com/questions/24298/can-i-ignore-coefficients-for-non-significant-levels-of-factors-in-a-linear-mode)
By all means keep Embarked\_C. Consider the following example:
You're predicting whether or not someone's favorite color is blue. You know the color of their favorite shirt- it's either blue, yellow, or red. Color\_Blue is going to be significant, the other two one hot encoded variables would not be. You'd still want to keep Color\_Blue as a feature.