Time Series - Models seem to not learn
I am doing my undergrad Dissertation on time series prediction, and use various models (linear /ridge regression, AR(2), Random Forest, SVR, and 4 variations of Neural Networks) to try and 'predict' (for academic only reasons) daily return data, using as input lagged returns and SMA - RSI features (using TA - Lib) built based on those returns. However, I have noticed that my NNs do not learn anything, and upon inspecting the loss graph and the vector of predictions, I noticed it only predicts a single value, with the same applying for the Ridge and AR regressions.
Also, when I try to calculate the correlation between the labels and the predictions (of the NNs) I get 'nan' as a result, no matter what I try, which I suspect has to do with the predictions. 458
I also get wildly varying r2 scores on each re-run (even though I have set multiple seeds, both on Tensorflow backend as well as numpy) and always negative, which I cannot understand as even though my search on the internet and the sklearn's docs say it can be negative, my professor insists it cannot be, and I truly am bewildered.
What can I do about it? Isn't it obviously wrong for an entire NN to predict only a single value? Below I include the code for the ridge / AR regressions as well as the 'Vanilla NN' and a couple of useful graphs. The data itself is quite large, so I don't know if there's much of a point to include it if not asked specifically, given there are no algorithmic errors below.
```python def vanillaNN(X_train, y_train,X_test,y_test): n_cols = X_train.shape1 model = Sequential() model.add(Dense(100,activation='relu', input_shape=(n_cols, ))) model.add(Dropout(0.3)) model.add(Dense(150, activation='relu')) model.add(Dense(50, activation='relu')) model.add(Dropout(0.1)) model.add(Dense(1))
model.compile(optimizer='adam', loss='mse', metrics=['mse'])
history = model.fit(X_train,y_train,epochs=100,verbose=0,
shuffle=False, validation_split=0.1)
# Use the last loss as the title
plt.plot(history.history['loss'])
plt.title('last loss:' + str(round(history.history['loss'][-1], 6)))
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.show()
# Calculate R^2 score and MSE
# .... Omitted Code ......
# it returns those for testing purposes in the IPython shell
return (train_scores, test_scores, y_pred_train, y_pred_test, y_train, y_test)
VNN_results = vanillaNN(
train_features,train_targets,
test_features,test_targets)
```
```python def AR(X_train, order=2): arma_train = np.array(X_train['returns']) armodel = ARMA(arma_train, order=(order,0)) armodel_results = armodel.fit() print(armodel_results.summary()) armodel_results.plot_predict(start=8670, end=8698) plt.show() ar_pred = armodel_results.predict(start=8699, end=9665)
# ...r2 and MSE scores omitted code...
return [mse_ar2, r2_ar2, ar_pred]
```
```python def Elastic(X_train, y_train, X_test, y_test): elastic = ElasticNet() param_grid_elastic = {'alpha': [0.001, 0.01, 0.1, 0.5], 'l1_ratio': [0.001, 0.01, 0.1, 0.5] } grid_elastic = GridSearchCV(elastic, param_grid_elastic, cv=tscv.split(X_train),scoring='neg_mean_squared_error')
grid_elastic.fit(X_train, y_train)
y_pred_train = grid_elastic. 392