Why is my artificial neural networks almost always predicting positive elements?
I've been working for a long time with an artificial neural network algorithm (specifically, a siamese neural network) that I implemented in Torch & Lua.
I've been studying and playing with many details of this algorithm (momentum alpha, number of minibatches, learning rate, gradient update iterations, dropout, 10-fold cross validation, etc), but I'm still facing the same old error: after training and during testing, my artificial neural network predicts almost every test element as positive.
I train my model with a training set, and then test it on a test set. Both sets contain 2,000 elements. This is a typical confusion matrix result I get:
false negatives FN: 21
true positives TP: 179
false positives FP: 1,747
true negatives TN: 53
And these category values lead to the following rates:
f1_score = <PHONE_NUMBER> = 2*tp/(2*tp+fp+fn) [1: best] [0: worst]
accuracy = 0.116 = (tp+tn)/(tp+fn+fp+tn) [1: best] [0: worst]
recall = 0.9 = tp/(tp+fn) [1: best] [0: worst]
precision = 0.09 = tp/(fp+tp) [1: best] [0: worst]
specificity = 0.03 = tn/(fp+tn) [1: best] [0: worst]
fallout = 0.97 = fp/(fp+tn) [0: best] [1: worst]
false discovery rate = 0.91 = fp/(fp+tp) [0: best] [1: worst]
miss rate = 0.105 = fn/(fn+tp) [0: best] [1: worst]
MatthewsCC = -0.12008 = ((tp*tn)-(fp*fn))/math.sqrt((tp+fp)*(tp+fn)*(tn+fp)*(tn+fn)) [+1: perfect correlation] [0: no better than random prediction] [-1: total disagreement]
As you can notice, the FN and TP results are very good, but the FP and TN results are very bad. 458
False positives are too many.
The artificial neural network thinks almost everything it processes is a positive element: it predicts the 96% of the elements as positives.
And the weird thing is that the training set is artificially balanced towards negative values: among the 2,000 elements, I'm choosing 90% of negatives and only 10% of positives. I also am using the same proportion for the test set (90% of negatives and only 10% of positives).
Does anyone have any idea about what is going on? Why does my neural network predict so many positives?
EDIT: Thanks to all for the help. Here are some data and the main core of my code.
Here are some toy data (you can copy them into your Torch terminal):
first_datasetTrain={};
first_datasetTrain[1]=torch.Tensor{4, 5, 8, 10, 36, 0, 11, 22, 23, 44, 49, 35, 22, 6, 12, 16, 16, 4, 8, 10, 12, 14, 3, 30, 11, 12, 1, 3, 12, 16, 3, 4, 3, 6, 8, 0, 5, 12, 5, 4, 18, 10, 6, 8, 4, 7, 12, 7, 13, 3, 66, 61, 9, 51, 28, 7, 24, 8, 43, 40, 115, 2, 5, 11, 11, 41, 8, 2, 2, 2, 11, 6, 4, 4, 5, 6, 6, 9, 25, 16, 19, 13 };
first_datasetTrain[2]=torch. 458