Ways Data Projects Fail
Good to see that data cleaning was #1 on that list. Whenever I do work on a side project, it takes way way more time to get and structure the data than it does running the algorithms. Granted, that's because I have to go out and get the data in the first place, and then make sure it's useable and in the correct format.<p>Like the recent project I'm doing trying to classify country music songs based on their topic on the data blog I write on (<a href="https://bigishdata.com" rel="nofollow">https://bigishdata.com</a>), the amount of time it's taking to scrape lyrics, remove duplicate / incorrect songs, and then do manual classification for training data is taking far longer than running the ml algorithms in the end one I've gone through that process.<p>I've been looking for jobs recently, and I've seen only one job posting that mentions data cleaning as a necessity, whereas the rest only talk about data science and algorithm knowledge, or overall ETL design on the data engineering side. Seems like data set knowledge should be emphasized more.