Machine Learning, Crowdsourcing, Innovation, Innovation Management.
are of equal size such that there is a 50% probability that a randomly selected instance should be either positive or negative (Bradley & P., 1997; DeLong, DeLong, & Clarke-Pearson, 1988; Fawcett, 2005; Hanley & McNeil, 1982). 3. DATA COLLECTION Data from eight crowdsourcing competitions were