I am using a deep neural network for a semantic segmentation task. The training data comprise 17,000 images that have been interpreted manually to generate two classes: MMD, and non-MMDs.
While I have good confidence in the accuracy of interpreted MMDs (high accuracy for true positives), there is still a chance that we misclassified some MMDs as non-MMD in the training data (even though its not very frequent). Further, the ratio of MMD to non-MMD samples is not balanced; and only ~20% of the training images are covered by MMDs. As the result, the training step can get biased toward the non-MMD class and show high accuracy for identifying the non-MMDs, which cover 80% of the image. I am currently using the following parameters:
• softmax as the activation function for the last layer
• Adam as the optimizer in Keras
• soft dice as the loss function to account for the imbalanced training data
• precision (tf.keras.metrics.Precision) as the metric to find the optimum model during training.
Obviously, my objective is to focus the training on maximizing the precision for the MMD class, and punish the training for generating false positives specifically for the MMD class. However, the above configuration does not seem to produce the desired results.
My question is, what would you do differently to achieve this objective? all suggestions are welcome!