An administrator accidentally deleted the production database
My very first job - ~25 years ago.Destroyed the production payroll database for a customer with a bug in a shell script.No problem - they had 3 backup tapes.First tape - read fails.Second tape - read fails.Third tape - worked.... (very nervous at this point).I think most people have an equivalent educational experience at some point in their careers.Edit: Had a project cancelled for one customer because they lost the database of test results..... 4 months work! Their COO (quite a large company) actually apologised to me in person!Edit: Also had someone from Oracle break a financial consolidation system for a billion dollar company - his last words were "you need to restore from tape" and then he disappeared. I was not happy as it was his attempts at "improving" things were the cause of the incident! Wouldn't have been angry if he had admitted he had made a mistake and worked with us to fix it - simply saying "restore from tape" and running away was not a good approach.