<p>It is a difficult task to identify the areas to be corrected to improve accuracy unless that area is looking into your face. By that I mean unless the regularizing parameter has unusually large or small values it is difficult to pin down and identify which regularizer to tweak. You need to do a grid search or random search over the hyper-parameter space to come up with an optimal combination of hyper parameters. </p>

<p>There could be a lot of things that you could consider to reduce over fitting. Some of them are <span class="math-container">$L_2$</span> regularizers, Dropout, depth of the network, number of neurons in the layers, the optimizer,  etc. </p>

CC BY-SA 4.0
2019-04-20T07:45:52.087Z
2019-04-20T10:01:54.327Z
2019-04-20T10:01:54.327Z
naive
<p>I am trying classify CIFAR10. The CNN that I generated over fits when the accuracy reaches ~77%. The code and the plot is given below. I tried DropOut, Batch Normalization and L2 Regularization. But the accuracy does not go beyond ~77.  </p>

<p>How can I identify the areas to be corrected to reduce over fitting? </p>

<pre><code>convolutional_model = Sequential()

# 32
convolutional_model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3), kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

# 64
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(128, (3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

convolutional_model.add(Flatten())
convolutional_model.add(Dropout(0.5))

convolutional_model.add(Dense(128, activation='relu'))
convolutional_model.add(Dense(10, activation='softmax'))

print(convolutional_model.summary())
convolutional_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor='val_loss', mode='min', verbose=2, patience=8)
history = convolutional_model.fit(X_Train_Part, Y_Train_Part, epochs=200, verbose=2,validation_data=(X_Train_Validate, Y_Train_Validate), callbacks=[es])

scores = convolutional_model.evaluate(X_Test, Y_Test, verbose=2)
</code></pre>

<p><a href="https://i.sstatic.net/TAOQ5.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TAOQ5.png" alt="enter image description here"></a></p>

CC BY-SA 4.0
2019-04-19T18:25:08.950Z
Please, ask just one question per post!
nbro
CC BY-SA 4.0
2019-04-19T19:32:35.267Z
Welcome to SE:AI!  I'm leaving this question open pending more closevotes, but please consider reducing it to a single question.  (You can always ask additional question in separate posts.)
DukeZhou
CC BY-SA 4.0
2019-04-20T01:40:47.927Z
Edited the post to reduce number of questions
Maanu
<deep-learning><convolutional-neural-networks><overfitting>
How to identify the areas to reduce over fitting?
Maanu
# How to identify the areas to reduce over fitting?

(Asked by: username_0 on 2019-04-19)

I am trying classify CIFAR10. The CNN that I generated over fits when the accuracy reaches ~77%. The code and the plot is given below. I tried DropOut, Batch Normalization and L2 Regularization. But the accuracy does not go beyond ~77.  
How can I identify the areas to be corrected to reduce over fitting? 
convolutional_model = Sequential()

# 32
convolutional_model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3), kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

# 64
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(128, (3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

convolutional_model.add(Flatten())
convolutional_model.add(Dropout(0.5))

convolutional_model.add(Dense(128, activation='relu'))
convolutional_model.add(Dense(10, activation='softmax'))

print(convolutional_model.summary())
convolutional_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor='val_loss', mode='min', verbose=2, patience=8)
history = convolutional_model.fit(X_Train_Part, Y_Train_Part, epochs=200, verbose=2,validation_data=(X_Train_Validate, Y_Train_Validate), callbacks=[es])

scores = convolutional_model.evaluate(X_Test, Y_Test, verbose=2)

**Tags:** deep-learning, convolutional-neural-networks, overfitting

> **Comments:**
> **username_1**: Please, ask just one question per post!  
> **username_2**: Welcome to SE:AI!  I'm leaving this question open pending more closevotes, but please consider reducing it to a single question.  (You can always ask additional question in separate posts.)  
> **username_0**: Edited the post to reduce number of questions  

---

## 1 answers

### Answer by username_3 on 2019-04-20. Score: 1

It is a difficult task to identify the areas to be corrected to improve accuracy unless that area is looking into your face. By that I mean unless the regularizing parameter has unusually large or small values it is difficult to pin down and identify which regularizer to tweak. You need to do a grid search or random search over the hyper-parameter space to come up with an optimal combination of hyper parameters.

There could be a lot of things that you could consider to reduce over fitting. Some of them are $L\_2$ regularizers, Dropout, depth of the network, number of neurons in the layers, the optimizer, etc.


---

