Issue while predicting multiple values which possess different order of magnitude (regression)
I have a system that take electric signal and output parameters (regression). However I run into an issue. The parameters I want to predict are not on the same magnitude. I would like also to use only one neural networks if possible. ``` Parameter 1: Mean = 12.53673 Minimum = 10.00461 Maximum = 14.98899
Parameter 2: Mean = 148656955394038 Minimum = 75029133522564 Maximum = 224934092847235
Parameter 3: Mean = 1.475720134278e+17 Minimum = 7.506184799345e+16 Maximum = 2.249190781380e+17
... ``` If I use any loss function such as RMSE, MSE, MAE, MSLE the losses of the big parameters takeover all the rest. 650
The network therefore randomly guess all the smaller parameters
I am searching a good way to find those parameters using one network. I tried using "mean absolute percentage error". But the learning is exceptionally slow, and it often get stuck into extremely low value and extremely high value.
I am maybe thinking about creating one loss per target. Freeze all the network except the output layer and each epoch let one of the loss train the whole network. But I am not sure it is a good idea.
Currently I am normalizing the target before predicting it, but I don't like this solution very much. I would prefer having a slightly more complicated architecture than normalizing and denormalizing (afraid of data leakage) and it seems more robust to directly output the right value.
Moreover In the future I will have an issue similar except I will need to predict series that possesses different order of magnitude (and not only some distinct value). So I hope being able to reuse the technique I am learning
Any help would be very appreciated! Thanks :)
EDIT: For those who happens to have the same issue, this is the code I went with (keras) ``` class Model: def init(self, shape_features, shape_targets, mean_targets, std_targets): self.shape_features = shape_features self.shape_targets = shape_targets self.mean_targets = mean_targets self.std_targets = std_targets
def build(self):
inputs = Input(shape=self.shape_features, name='Input')
x = Conv1D(filters=16, kernel_size=3, activation='relu', name='Convolution1D_1')(inputs)
x = MaxPooling1D(pool_size=2, strides=2, padding='valid', name='MaxPooling1D_1')(x)
x = Conv1D(filters=32, kernel_size=3, activation='relu', name='Convolution1D_2')(x)
x = MaxPooling1D(pool_size=2, strides=2, padding='valid', name='MaxPooling1D_2')(x)
x = Conv1D(filters=64, kernel_size=3, activation='relu', name='Convolution1D_3')(x)
x = MaxPooling1D(pool_size=2, strides=2, padding='valid', name='MaxPooling1D_3')(x)
x = Flatten(name='Flatten')(x)
x = Dense(128, activation='relu', name='Dense_1')(x)
x = Dense(64, activation='relu', name='Dense_2')(x)
x = Dropout(0.2, name='Dropout')(x)
outputs = Dense(self.shape_targets, activation='linear')(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
def de_normalizing(tensor):
return tensor * self.std_targets + self.mean_targets
predictions = Lambda(de_normalizing)(outputs)
model_prediction = tf.keras.Model(inputs=inputs, outputs=predictions)
def custom_loss(y_true, y_pred):
y_true = (y_true-self.mean_targets)/self.std_targets
return K.mean(K. 458