End-to-end implementation of a machine learning pipeline (2017)
Great explanation and I love the fact that the entire presentation is a Jupyter Notebook!A non-academic observation - the 'real-world' challenge of ML pipelines is what I call the 'last-mile' problem of ML - operationalizing your model. You begin to run into problems of:1. How often do you 'score' live data? How will this affect latency, data ingestion etc?2. How often do you have to update your weights, if you want your model's performance to be consistent?3. Integration with source systems4. If you build your final scoring model on library-dependent languages like Python, how do you ensure no breakages? (Docker solves this to a large extent though)