(Consistent) Feature Selection Over Time Let us assume the following simple scenario: Your goal is to forecast the sales figures of different companies. You assume that the sales figures are determined by company-specific characteristics (size, industry, etc.) and economic covariates (GDP growth, etc.).
Assume further that each company occurs in only one particular year of your database (cross-sectional data). When calibrating the model, you use a rolling-window approach. In particular, you calibrate your model with data from 2010-2014 and intend to make predictions for 2015. In a next step, you move the window to 2011-2015 for calibration and 2016 for prediction and so on. This approach ensures that you maintain the time dependence of the observations.
Since you can draw on a wide set of features, you use a selection method, such as Lasso. However, this may result in completely non-overlapping characteristics in each period, which does not make sense from a business perspective. Due to the time dependence, it is also not possible to identify the features in a first step across all observations. Ultimately, you want to identify a set of features that work well over time.
The question now is: Are there any proven methods or ways to deal with such a situation?


Since you can draw on a wide set of features, you use a selection method, such as Lasso

Alternatively, you could consider a regularized framework with an extensive collection of features (explanatory variables, "filter bank" in image modeling terminology).  Different features matter at different times.  Commercial fundamental factor models (models based on company attributes) tend to have extensive sets of factors, not all of which matter in a particular period.

this may result in completely non-overlapping characteristics in each period, which does not make sense from a business perspective

It may make sense.  The salient features of a particular period may vary.  Perhaps sensitivity to inflation matters in one period and sensitivity to energy prices matters in another period. The issue is not that a factor is absent or a factor loading is "wrong" in one period, but the latent common factor is dormant or nearly so.
You may want to decompose your problem into identifying common factor returns (to use financial risk modeling terminology) and common factor loadings (regression coefficients).  This process is discussed in

*

*Connor, Gregory, and Oliver Linton. "Semiparametric estimation of a characteristic-based factor model of common stock returns." Journal of Empirical Finance 2007

*Grinold, Richard C., and Ronald N. Kahn. Active portfolio management 2000

For an online framework for your factor loading estimation, you could consider implementing a Bayesian dynamic linear model (regression coefficients are permitted to change slowly with time).

*

*West, Mike, and Jeff Harrison. Bayesian forecasting and dynamic
models 2006

*Kalaba, Robert,
and Leigh Tesfatsion. "Time-varying linear regression via flexible
least squares." Computers & Mathematics with Applications 1989

To summarize salient features of a historical period, you could iteratively seek to replicate the return distributions, adding features until your synthetic distribution ("model") matches the observed distribution.

*

*(self cite) Keane, Kevin R. "Ch. 6 Portfolio Variance Constraints." Implementing high dimensional Gaussian models for financial applications State University of New York at Buffalo thesis, 2015.

