**Leads:** errordeveloper, r2d4  
**Slides:** n/a  
**Thanks to our notetakers:** mrbobbytales, onyiny-ang

What APIs should we target, what parts of the developer workflow haven't been covered yet?

* Do you think the Developer tools for Kubernetes is a solved problem?
  * A: No

### Long form responses from SIG Apps survey
* Need to talk about developer experience
* Kubernetes Community can do a lot more in helping evangelize Software development workflow, including CI/CD. Just expecting some guidelines on the more productive ways to write software that runs in k8s.
* Although my sentiment is neutral on kube, it is getting better as more tools are emerging to allow my devs to stick to app development and not get distracted by kube items. There is a lot of tooling available which is a dual edge sword, these tools range greatly in usability robustness and security. So it takes a lot of effort to...

### Current State of Developer Experience
* Many Tools
* Mostly incompatible
* Few end-to-end workflows

### Comments and Questions
* Idea from scaffold to normalize the interface for builders, be able to swap them out behind the scenes.
* Possible to formalize these as CRDs?
* Lots of choices, helm, other templating, kompose etc..
* So much flexibility in the Kubernetes API that it can become complicated for new developers coming up.
  * Debug containers might make things easier for developers to work through building and troubleshooting their app.
* Domains and workflow are so different from companies that everyone has their own opinionated solution.
* Lots of work being done in the app def working group to define what an app is.
* app CRD work should make things easier for developers.
* Break out developer workflow into stages and try and work through expanding them, e.g. develop/debug
* debug containers are looking to be used both in prod and developer workflows
* Tool in sig-cli called kustomize, was previously 'konflate'?
* Hard to talk about all these topics as there isn't the language to talk about these classes of tools.
* @jacob investigation into application definition: re: phases, its not just build, deploy, debug, its build, deploy, lifecycle, debug. Managing lifecycle is still a problem, '1-click deploy' doesn't handle lifecycle.
* @Bryan Liles: thoughts about why this is hard:
    * kubectl helm apply objects in different orders
    * objects vs abstractions
    * some people love [ksonnet](https://ksonnet.io/), some hate it. Kubernetes concepts are introduced differently to different people so not everyone is starting with the same base. Thus, some tools are harder for some people to grasp than others. Shout out to everyone who's trying to work through it * Being tied to one tool breaks compatibility across providers.
* Debug containers are great for break-glass scenarios
* CoreOS had an operator that handled the entire stack, additional objects could be created and certain metrics attached.
  * Everything is open source now, etcd, prometheus operator
* Tools are applying things in different orders, and this can be a problem across tooling
* People who depend on startup order also tend to have reliability problems as they have their own operational problems, should try and engineer around it.
* Can be hard if going crazy on high-level abstractions, can make things overly complicated and there are a slew of constraints in play.
* Ordering constraints are needed for certain garbage collection tasks, having ordering may actually be useful.
* Some groups have avoided high-level DSLs because people should understand readiness/livelness probes etc. Developers may have a learning curve, but worthwhile when troubleshooting and getting into the weeds.
* Lots of people don't want to get into it at all, they want to put in a few details on a db etc and get it.
* Maybe standardize on a set of labels to on things that should be managed as a group. Helm is one implementation, it should go beyond helm.
  * There is a PR that is out there that might take care of some of this.
* Everyone has their own "style" when it comes to this space.
* Break the phases and components in the development and deployment workflow into sub-problems and they may be able to actually be tackled. Right now the community seems to tackling everything at once and developing different tools to do the same thing.
* build UI that displays the whole thing as a list and allows easy creation/destruction of cluster
    * avoid tools that would prevent portability
* objects rendered to file somehow: happens at runtime, additional operator that takes care of the sack
    * 3, 4 minor upgrades without breakage
* @Daniel Smith: start up order problems = probably bigger problems, order shouldn't need to matter but in the real world sometimes it does
* platform team, internal paths team (TSL like theme), etc. In some cases it's best to go crazy focusing on the abstractions--whole lot of plumbing that needs to happen to get everything working properly
* Well defined order of creation may not be a bad thing. ie. ensure objects aren't created that are immediately garbage collected.
* Taking a step back from being contributors and put on developer hats to consider the tool sprawl that exists and is not necessarily compatible across different aspects of kubernetes. Is there anyway to consolidate them and make them more standardized?
* Split into sub-problems

## How can we get involved?
- SIG-Apps - join the conversation on slack, mailing list, or weekly Monday meeting
