Controlling for non-linear variable in non-linear modeling of response I need to model a continuous response variable $y$ based on continuous features $x_1, ..., x_n$ while controlling for another continuous feature $x_c$. The intent is to understand how much an increase of 1 in one of the features (e.g. $x_1$) would on average impact $y$ while controlling for $x_c$.
If the relationships were linear, then including $x_c$ in a normal linear regression would be enough to control for $x_c$, but that's not the case. The data has the following patterns that makes it much more difficult to model:

*

*The relationship between $y$ and $x_c$ is highly non-linear

*The relationship between $y$ and $x_1, ..., x_n$ is highly non-linear

*Some of the features in $x_1, ..., x_n$ are highly correlated with $x_c$
How would I best model $y$ using features $x_1, ..., x_n$ while controlling for $x_c$?

You could train a non-linear model using only $x_c$ to predict $y$, i.e. $f(x_c) = \hat{y}$. Then compute the residual $\epsilon = \hat{y} - y$. The residual $\epsilon$ will have been controlled for with regards to $x_c$ as $f$ has captured the relationship between $x_c$ and $y$ as well as it possibly could.
You can now use the residual $\epsilon$ as input for all further analysis such as any modeling or feature importance algorithms. You’ll want to include $x_c$ as a feature in your further analysis in case of any interaction effects between $x_c$ and $x_1, …, x_n$.
In terms of the choice of the non-linear model for $f$, random forest is a good and easy-to-use method, but many other non-linear models (e.g. neural networks) could be used as well.
Warning: I think this approach works, but would appreciate references to confirm it!
