ML is useful for many things, but not for predicting scientific replicability
From the OG paper:<p>&gt;Our  machine  learning  model  used  an ensemble of random forest and logistic regression models to predict a paper’s likelihood of replication based on the paper’s text.<p>&gt;We trained a model using word2vec on a corpus of 2 million social science publication abstracts published between 2000 and 2017<p>&gt;converting publications into vectors. To do this, we multiplied the normalized frequency of each word in each paper in the training sample by its corresponding 200-dimension word  vector,  which  produced  a  paper-level  vector  representing  the textual content of the paper<p>If you took a paper and rearranged the words to have a completely different meaning, their method would produce the same prediction. It also has no understanding of, or the ability to differentiate between, quotation and references within the paper and content written by the authors themselves. Good luck with that! It&#x27;s basically just learning some known shitty combinations of keywords.