Linear regression: highly-correlated features but not redundant I have been working on a problem whose goal is to find the most accurate coefficient (from a physical point of view) that expresses the relationship between a feature and an output variable. The model is linear, has multiple features, and I am using the normal equation to solve this.
My concern here is that some features are highly correlated, and in my understanding this can lead to some questions about how to find the correct coefficients. I have read that removing these features is sometimes recommended, but in this problem these features are not redundant, i.e. they all contribute to explaining the output variable.
So, how should I proceed to find the correct coefficients without discarding any features?
Edit: I think I should have provided a simple example of the real problem to make this more clear and interesting:

*

*Consider a small village where electricity consumers are connected to the same grid.

*I would like to know how the voltage at my home is affected by the consumption of consumer A, consumer B, ....

*Part of these consumers have PV panels, which inject power into the grid (it's like having a negative consumption).

*Injecting power affects the voltages of the grid, particularly near that injection point. For example, if my neighbor is injecting power into the grid, the voltage at my home will increase more than if that injection occurred farther.

*During the day, electricity consumption may be low because people are not at home but all the PV panels are injecting power in a very similar way. As you can see bellow, when consumers install PV panels, their consumption patterns become highly correlated. However, I cannot discard any of them because each one is influencing my voltage. Closer to me, higher the influence.




The correlation affects the standard errors on coefficients, but, at least under nice conditions that are typical to assume, the OLS solution is unbiased. Regress away, knowing that there is uncertainty in your estimates. If that uncertainty is small enough (that's a decision for you to make with your knowledge of the material being studied), then you're done. If you are concerned about lacking the sample size to have small standard errors that result in less uncertainty, then you can do a sample size calculation to determine how many observations you need to get acceptable results. An alternative could be to design your experiment to keep the features from getting correlated. How to execute such an experiment will come down to your knowledge of the subject matter.
Alternatives like regularized regression might be problematic in your case where interpreting the coefficients is of paramount importance. In particular, there is not a straightforward way to estimate uncertainty in the coefficient estimates, especially if you use cross validation to tune the regularization amount.
