The problem is when using fbeta_score, I get an error:

"ValueError: Classification metrics can’t handle a mix of multilabel-indicator and continuous-multioutput targets

I take it to mean that I need to binarize the predictions (so going from probabilities to 0/1).
To do that I need to decide on a threshold. So, if I’m going to choose one why not choose the one that gives me the best fbeta_score?