Controlling for non-linear variable in non-linear modeling of response I need to model a continuous response variable $y$ based on continuous features $x_1, ..., x_n$ while controlling for another continuous feature $x_c$. The intent is to understand how much an increase of 1 in one of the features (e.g. $x_1$) would on average impact $y$ while controlling for $x_c$.
If the relationships were linear, then including $x_c$ in a normal linear regression would be enough to control for $x_c$, but that's not the case. The data has the following patterns that makes it much more difficult to model:

*

*The relationship between $y$ and $x_c$ is highly non-linear

*The relationship between $y$ and $x_1, ..., x_n$ is highly non-linear

*Some of the features in $x_1, ..., x_n$ are highly correlated with $x_c$
How would I best model $y$ using features $x_1, ..., x_n$ while controlling for $x_c$?

All you've really told us is that the relationships are nonlinear, and you are asking us to tell you how to get the best model.  It's not really possible to do that, since "nonlinear" is a huge class of relationships.  Nevertheless, here are some things to think about:

*

*Ideally, there would be some aspect of the variables that would suggest a class of nonlinear functions that might be plausible for the regression relationships.  Sometimes the nature of the variables and the context from which they are obtained give us some hints on the types of nonlinear relationships they might have.


*If this is not the case, you might need to proceed on a purely empirical basis.  For non-periodic functions you can usually approximate them locally by polynomials (based on the theory of Taylor series), and for periodic functions you can usually approximate them locally by sums of sinusoidal functions (based on the theory of Fourier series).  These approximations work fairly well on a wide class of nonlinear functions, so they are commonly used as a starting point in nonlinear regression in cases where there is no prior information about the nature of the nonlinear relationship.


*Once you have formulated a reasonable starting point for your model, fit the model and produce its diagnostic plots, paying particular attention to the added variable plots.  These latter plots will allow you to scrutinise the chosen regression relationshops in your model based on the data.  This can alert you to erroneous assumptions in your model, and allow you to choose a more appropriate regression function to fit your data.  (But there is a limit to this; beware of the phenomenon of over-fitting.)


*Finally, the issue of multicollinearity of the explanatory variables is a common issue in prediction.  Unless you are conducting a controlled trial and you have control over your explanatory variables, there is nothing much you can do about this, except to be aware of the effect that multicollinearity has on the accuracy of your regression estimates, and to make sure that you are clear on what type of predictive inferences you want to make (i.e., exactly what do you want to condition on).
