  Neural networks are rapidly gaining interest in nonlinear system
identification due to the model's ability to capture complex input-output
relations directly from data. However, despite the flexibility of the approach,
there are still concerns about the safety of these models in this context, as
well as the need for large amounts of potentially expensive data. Aluminum
electrolysis is a highly nonlinear production process, and most of the data
must be sampled manually, making the sampling process expensive and infrequent.
In the case of infrequent measurements of state variables, the accuracy and
open-loop stability of the long-term predictions become highly important.
Standard neural networks struggle to provide stable long-term predictions with
limited training data. In this work, we investigate the effect of combining
concatenated skip-connections and the sparsity-promoting $\ell_1$
regularization on the open-loop stability and accuracy of forecasts with short,
medium, and long prediction horizons. The case study is conducted on a
high-dimensional and nonlinear simulator representing an aluminum electrolysis
cell's mass and energy balance. The proposed model structure contains
concatenated skip connections from the input layer and all intermittent layers
to the output layer, referred to as InputSkip. $\ell_1$ regularized InputSkip
is called sparse InputSkip. The results show that sparse InputSkip outperforms
dense and sparse standard feedforward neural networks and dense InputSkip
regarding open-loop stability and long-term predictive accuracy. The results
are significant when models are trained on datasets of all sizes (small,
medium, and large training sets) and for all prediction horizons (short,
medium, and long prediction horizons.)

Systems and Control;Machine Learning;Systems and Control