Data Science, Machine Learning, Design Patterns, Bigquery, Sql.
np.random.rand(len(df)) train = df[ rnd < 0.8 ] valid = df[ rnd >= 0.8 & rnd < 0.9 ] test = df[ rnd >= 0.9 ] The problem is that this fails in many real-world situations. The reason is that it is rare that the rows are independent. For example, if you are training a model to predict flight delays,