The ominous opacity of the AWS bill – a cautionary tale
Here's my $700 surprise bill story. There are many like it, but this one is mine.<p>* An example CloudFormation from a Re:Invent (AWS conference) session silently failed to tear down some resources.<p>* Not trusting CloudFormation, I looked through each (known service, region) manually to make sure resources had been torn down. This failed to identify the running resources because a tutorial div opened in regions with no running resources and remained open if you switched to a region with running resources, hiding them.<p>* Not trusting my manual service tour, I kept a close eye on my daily costs until I saw several days pass with $0 spend. This failed because free tier credits were hiding substantial service usage.<p>* Not trusting any of the above, I had billing alerts set as a catch-all. They correctly triggered on an unrelated usage surge, but with such high latency that I incorrectly attributed their failure to reset to high latency rather than to a genuine underlying charge.<p>Bam, $700 charge next month. Amazon was quick to refund half of it. I was eventually able to get them to refund the other half by making waves in the support system of a high-spend business account.<p>At the last re:invent session I went to, I surveyed a table of 6 people. After sharing my $700 figure, 3 of the 6 came forward with even bigger numbers, 1 of the 6 with a smaller number, and the remaining person was a newbie.