End-to-end implementation of a machine learning pipeline (2017)
Great explanation and I love the fact that the entire presentation <i>is</i> a Jupyter Notebook!<p>A non-academic observation - the 'real-world' challenge of ML pipelines is what I call the 'last-mile' problem of ML - operationalizing your model. You begin to run into problems of:<p>1. How often do you 'score' live data? How will this affect latency, data ingestion etc?<p>2. How often do you have to update your weights, if you want your model's performance to be consistent?<p>3. Integration with source systems<p>4. If you build your final scoring model on library-dependent languages like Python, how do you ensure no breakages? (Docker solves this to a large extent though)