Question: I'm a senior developer in a mid-sized business. We are split into multiple sub-teams with specific roles for each. My allocated team's responsibility is for the day to day running of services and bug fixing issues that arise from our product.

I have recently had a bad case of burnout related to the current COVID situation. Amongst the several concerns that I have are that of an individual team member. This team member has a habit of making breaking changes, without much thought or consideration given to others in the team, let alone across sub-teams. It is almost the bane of my life, multiple system outages have been caused by this person, but they continue at an astonishing pace of development. This is obviously viewed by management as a "positive" thing as they're getting the work done.

The catch here is that each time there is an outage, the investigation follows. This involves multiple meetings, bug fixing (my team) and slowdown in our work while we put out the inevitable fires that have been started. Worst of all, it is really hard to get them to own up to their mistakes. Talking with them, it's never "their" code that's at fault. The individual even tried to blame the hardware ECC memory for memory corruption that originated from this individual's code.

I am all for people learning from mistakes, but we barely get to put out the last fire before the next one is started. Most recently, and while I have been under a lot of pressure to deliver, I went to get the final task of the day done. Lo and behold, they are at it again and a code commit broke my work.

I quite publicly flipped out on our team channels about it because I was pretty fed-up with the direct approach. This has got management attention, in that their manager got involved. After talking with management time and again, little seems to have changed with this individual.

How do I raise this to management’s attention, get them to act and, more importantly, help prevent this from continuing?

Answer: 
**Perform retrospectives and iterate your process to be resilient in the face of one person's error.**

Performing [blameless incident retrospectives](https://landing.google.com/sre/sre-book/chapters/postmortem-culture/) and regular development team retrospectives where you focus on the issues, all learn about your system, and devise ways to make your system resistant to errors is the best first step.

Identify the issues, figure out frequency and cost metrics around them, and figure out how to make it better. Does there need to be more testing and higher code coverage? Do people need to be committing more frequently so the PRs are smaller? Do more people need to be on a PR? Are build failure notifications being sent in actionable ways (in chat, in email, etc.) to the right party?

A developer should be free to make mistakes. There should be a framework in place that helps all the devs find errors, quickly, before passing them down the line.

**Maybe there still needs to be personnel action - maybe.**

Some folks believe "everything" is the system's issue. In reality there are some people who are slackers or screwups and no reasonable amount of development guardrails will correct that.

However, that's rarer than people think. If you have a very prolific developer that creates 5x more code than their colleagues, then if they create 5x the bugs/incidents of their colleagues that's normal, and I would prefer that employee to other employees because I'm getting 5x the output. It's only if they are causing 10x the problems that calculus starts to cross back over the line.

Even if you need to take this to their manager, be focused on solutions. "Talking to" the manager and their manager "talking to" them hasn't worked and usually doesn't. Engage him in fixing the problem. "How can flaws in your team's work not affect ours? What agreements or technical safeguards can we put in place to make this best for everyone?" Keep in mind YOU should not be doing this, but your manager should be talking with the other team's manager.
I agree with others who've advocated using a CI system and tests. In my startup, we run tests against each pull request, and only those that pass can be merged. After the merge, tests must pass again before the latest merge is deployed to production. And if a pull request breaks production, it is *immediately* reverted. Then it's up to the developer to clean it up, get it through code review again (usually with new tests to demonstrate that the problem is fixed), and ensure it passes tests. We still have problems that take down production, but only for a matter of minutes at most. If in doubt, we revert first and ask questions later.