JupyterHub in Kubernetes
We were in almost exactly the same situation. Workstations at the office. GCP account. We started pushing for more remote work about two years ago, and we built our internal machine learning platform (<a href="https:&#x2F;&#x2F;iko.ai" rel="nofollow">https:&#x2F;&#x2F;iko.ai</a>) around our workflow solving one bit after the other, precisely because we didn&#x27;t want to be tied to one library&#x2F;framework, and without having to pollute our notebooks.<p>Here&#x27;s what we&#x27;re doing:<p>- No-setup, fresh, notebook environments with the most popular libraries pre-installed: this saves a lot of time, avoids having environments that break, etc. We start afresh with large Docker images, and then people can install libraries.<p>- Real-time collaboration on notebooks: so we can troubleshoot and pair-program on the same notebook at the same time, we can see cursors, selections, changes, etc. We also use that for our weekly calls where we have the agenda, code snippets, etc. all in the same place. We can talk through agenda items, edit in real-time, add snippets of code, and brainstorm.<p>- Multiple versions of your notebooks: there&#x27;s a regression in JupyterLab that we fixed. We&#x27;re working to make it available in upstream JupyterLab[0].<p>- Long-running notebook scheduling with output that survives closed tabs and network disruptions: we select the Docker image and the output file. This solves the common problem of launching a long-running notebook, and having to keep the browser tab open and hope the network connection doesn&#x27;t break. Some people circumvent that porblem by scheduling the notebook and saving the artifacts right from the notebook, but we wanted to be able to have the output streamed whether the tab was closed or not. Bonus: we can watch the same notebook running on multiple laptops.<p>- Automatic experiment tracking: automatically detects your models, parameters, and metrics and saves them without you remembering to do so or polluting your notebook with tracking code.<p>- Easily deploy your model and get a &quot;REST endpoint&quot; so data scientists don&#x27;t tap on anyone&#x27;s shoulder to deploy their model, and developers don&#x27;t need to worry about ML dependencies to <i>use</i> the models, and be dragged into the &quot;ML realm&quot;. The models also have a page where you can invoke the model by entering JSON and get predictions, or uploading a CSV file and get predictions.<p>- Build Docker images for your model and push it to a registry to use it wherever you want: currently we push to DockerHub and GitLab<p>- Monitor your models&#x27; performance on a live dashboard: requests, latency, performance, etc.<p>- Publish notebooks as AppBooks: automatically parametrize a notebook to enable clients to interact with it without exporting PDFs or having to build an application or mutate the notebook. This is very useful when you want to expose some parameters that are very domain specific to a domain expert. For example, some features in a nuclear engineering problem are very domain specific, and a nuclear engineer can bring in a lot of value tweaking the parameters themselves. It also spares our people from spinning up yet another VM on GCP, loading the model, creating a Flask application, setting authentication, etc. The AppBook runs are also tracked, of course.<p>Much more on our roadmap. We&#x27;re only focusing on actual problems we have faced serving our clients, and problems we are facing now.<p>The notebook servers are run on an arbitrary cluster that happens to be ours for now, but we&#x27;re pretty much approaching it with the following mindset: we are a company that uses this platform, and the platform just happens to be ours.<p>- [0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;jupyterlab&#x2F;jupyterlab&#x2F;issues&#x2F;5923" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jupyterlab&#x2F;jupyterlab&#x2F;issues&#x2F;5923</a>