Answers: <p>I was wondering whether anyone in this forum had new arguments they have heard or tried in person to convince executives of the need to set service level objectives at a level below 100%, hence the original question.</p>
  
  <p>Here are some approaches I've seen or taken in the past to tackling this problem.  <em>The Site Reliability Workbook</em> has a few suggestions <a href="https://landing.google.com/sre/workbook/chapters/implementing-slos/#reliability-targets-and-error-budgets" rel="nofollow noreferrer">here</a>, which include:</p>
  
  <ul>
  <li>100% is not a reasonable target since every component of any system has a
  non-zero probability of failure.</li>
  <li>External components (such as an ISP) sitting between the customer and the target system aren't 100% reliable.</li>
  <li>100% reliability implies you can never change the system, since all change introduces risk.</li>
  <li>100% reliability implies you spend all your time reacting to reliability problems and have no time for anything else.</li>
  </ul>
  
  <p>In addition to the above, I've also gathered the following talking points from various in-person discussions that might be useful to persuade organizational leaders:</p>
  
  <ul>
  <li>If an IT exec sets a target of 100% reliability, they are basically encouraging their engineers to lie to them and hide problems, which means they will discover those problems in the most expensive possible way.</li>
  <li>Nature has never evolved a 100% reliable system. Gene replication is unreliable. The human heart is not 100% reliable.  Life continues and thrives in the face of imperfect reliability, and so can technology systems and their users.</li>
  <li>No group of humans has ever engineered a system that's 100% reliable in the long run.</li>
  </ul>
  
  <p>We also have to acknowledge that for certain dimensions of reliability, such as data loss prevention, some regulated industries require levels of reliability so high and over long periods of time that they are virtually indistinguishable from 100%, like 99.999999%. For other dimensions like availability and latency, three or four nines of reliability are often more than adequate, and some of the arguments above may help persuade IT leadership that this is true.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-04-02T22:47:19.910Z
Answers: 2019-04-02T22:47:19.910Z
Answers: Vivek Rau
Answers: <p>If your company/manager/etc is arguing for 100% reliability, please ask them to name <strong>just one</strong> human-engineered system that has achieved that mark over any non-trivial period of time. I'm not aware of any -- and I've looked <em>a lot</em>.  Pacemakers don't even have 100% reliability. </p>
  
  <p>Remove the humans. You'd be hard pressed to find anything in nature that achieved that goal, either.</p>
  
  <p>A goal of perfection is deeply counterproductive and will eventually slow your innovation asymptotically towards 0. </p>
  
  <p><a href="https://www.youtube.com/watch?v=0zqBlRW_6jA" rel="nofollow noreferrer">Here's</a> a fuller explanation.</p>
  
  <p>You will ship much greater reliability to your customers if you concentrate less on reducing errors, and focus more on detecting them and mitigating them quickly. The smaller the blast radius, the faster you can go. </p>
  
  <p>The sin is not in the failing, but in the failing to notice.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-10-27T01:22:34.197Z
Answers: 2019-10-27T01:22:34.197Z
Answers: Dave Rensin
Answers: <p>Why do you think 100% reliability is not a good idea? Can you support your beliefs with numbers, facts and logical explanations?</p>
  
  <p>Is the needed hardware too expensive? Development time too long? Maybe you can show that the probability of success of a chain of components is the multiplication of the individual parts, some of which are not under your control?</p>
  
  <p>If you can support your claims then present those to your managers, if not then your managers might be right. </p>
  
Answers: CC BY-SA 4.0
Answers: 2019-03-26T22:50:08.843Z
Answers: I'm wondering whether there's an example of a software or hardware system that is 100% reliable.  What metric was used to quantify reliability, and over how long a period was it measured?
Answers: Vivek Rau
Answers: CC BY-SA 4.0
Answers: 2019-03-25T19:34:03.390Z
Answers: 2019-03-25T19:34:03.390Z
Answers: Rsf
Answers: <p>This is actually quite easy, if you know how much lost means the system being down.</p>
  
  <p>That is if you take eBay or Amazon they would have considerable loss even for one second of downtime in a year. So they strive to have a 99.9999... reliability/uptime.</p>
  
  <p>So this is not about 100% but how many nines does it make sense to invest for you to come close to 100%. </p>
  
Answers: CC BY-SA 4.0
Answers: 2019-03-27T05:46:08.510Z
Answers: 2019-03-28T17:05:08.210Z
Answers: 2019-03-28T17:05:08.210Z
Answers: Ta Mu
Body: <p>It's a fundamental principle of DevOps and SRE that failure is normal and setting a goal of perfect reliability is misguided. But sometimes IT execs and business leaders push back on the idea. They believe the business needs 100% reliability. What are some good way to persuade them to change their thinking?</p>
  
Tags: <sre>
Title: How can I convince someone that 100% reliability is not the right target for anything?
UserDisplayName: Vivek Rau
ThreadText: # How can I convince someone that 100% reliability is not the right target for anything?
  
  (Asked by: username_0 on 2019-03-25)
  
  It's a fundamental principle of DevOps and SRE that failure is normal and setting a goal of perfect reliability is misguided. But sometimes IT execs and business leaders push back on the idea. They believe the business needs 100% reliability. What are some good way to persuade them to change their thinking?
  
  **Tags:** sre
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_0 on 2019-04-02. Score: 4
  
  I was wondering whether anyone in this forum had new arguments they have heard or tried in person to convince executives of the need to set service level objectives at a level below 100%, hence the original question.
  
  Here are some approaches I've seen or taken in the past to tackling this problem. *The Site Reliability Workbook* has a few suggestions [here](https://landing.google.com/sre/workbook/chapters/implementing-slos/#reliability-targets-and-error-budgets), which include:
  
  * 100% is not a reasonable target since every component of any system has a
    non-zero probability of failure.
  * External components (such as an ISP) sitting between the customer and the target system aren't 100% reliable.
  * 100% reliability implies you can never change the system, since all change introduces risk.
  * 100% reliability implies you spend all your time reacting to reliability problems and have no time for anything else.
  
  In addition to the above, I've also gathered the following talking points from various in-person discussions that might be useful to persuade organizational leaders:
  
  * If an IT exec sets a target of 100% reliability, they are basically encouraging their engineers to lie to them and hide problems, which means they will discover those problems in the most expensive possible way.
  * Nature has never evolved a 100% reliable system. Gene replication is unreliable. The human heart is not 100% reliable. Life continues and thrives in the face of imperfect reliability, and so can technology systems and their users.
  * No group of humans has ever engineered a system that's 100% reliable in the long run.
  
  We also have to acknowledge that for certain dimensions of reliability, such as data loss prevention, some regulated industries require levels of reliability so high and over long periods of time that they are virtually indistinguishable from 100%, like 99.999999%. For other dimensions like availability and latency, three or four nines of reliability are often more than adequate, and some of the arguments above may help persuade IT leadership that this is true.
  
  
  ---
  
  ### Answer by username_1 on 2019-10-27. Score: 2
  
  If your company/manager/etc is arguing for 100% reliability, please ask them to name **just one** human-engineered system that has achieved that mark over any non-trivial period of time. I'm not aware of any -- and I've looked *a lot*. Pacemakers don't even have 100% reliability.
  
  Remove the humans. You'd be hard pressed to find anything in nature that achieved that goal, either.
  
  A goal of perfection is deeply counterproductive and will eventually slow your innovation asymptotically towards 0.
  
  [Here's](https://www.youtube.com/watch?v=0zqBlRW_6jA) a fuller explanation.
  
  You will ship much greater reliability to your customers if you concentrate less on reducing errors, and focus more on detecting them and mitigating them quickly. The smaller the blast radius, the faster you can go.
  
  The sin is not in the failing, but in the failing to notice.
  
  
  ---
  
  ### Answer by username_2 on 2019-03-25. Score: 1
  
  Why do you think 100% reliability is not a good idea? Can you support your beliefs with numbers, facts and logical explanations?
  
  Is the needed hardware too expensive? Development time too long? Maybe you can show that the probability of success of a chain of components is the multiplication of the individual parts, some of which are not under your control?
  
  If you can support your claims then present those to your managers, if not then your managers might be right.
  
  > **Comments:**
  > **username_0**: I'm wondering whether there's an example of a software or hardware system that is 100% reliable.  What metric was used to quantify reliability, and over how long a period was it measured?  
  
  ---
  
  ### Answer by username_3 on 2019-03-27. Score: 1
  
  This is actually quite easy, if you know how much lost means the system being down.
  
  That is if you take eBay or Amazon they would have considerable loss even for one second of downtime in a year. So they strive to have a 99.9999... reliability/uptime.
  
  So this is not about 100% but how many nines does it make sense to invest for you to come close to 100%.
  
  
  ---
  
  