AWS Service Disruption Post Mortem
tldr: ""The trigger for this event was a network configuration change. We will audit our change process and increase the automation to prevent this mistake from happening in the future."<p>AMZN has gotten a lot of flack over this outage, and rightly so.  But I do want to dissuade anyone from thinking anybody else could do much better.  I worked there 10 years ago, when they were closer to 200 engineers, and the caliber of people there at that point was insane.  By far the smartest bunch I've ever worked with, and a place where I learned habits that serve me well to this day.<p>I know the guys that started the AWS group and they were the best of that already insanely selective group.  It is easy to be an arm chair coach and scream that the network changes should have been automated in the first place, or that they should have predicted this storm, but that ignores just how fantastically hard what they are doing is and how fantastically well it works 99(how many 9's now?)% of the time.<p>In short, take my word for it, the people working on this are smarter than you and me, by an order of magnitude.  There is no way you could do better, and it is unlikely that if you are building anything that needs more than a handful of servers you could build anything more reliable.