Solved by suraj.pt in post #9 
              
              
                AFAIK f-score is ill-suited as a loss function for training a network. F-score is better suited to judge a classifier’s calibration, but does not hold enough information for the neural network to improve its predictions. 
Loss functions are differentiable so that they can propagate gradients through…