Answers: <p><strong>DevOps</strong> does not have any <strong>KPI</strong>. It would be like asking what are the KPI of Love. But some of the things you mentioned (<em>Problem and Incident Management</em>, <em>Capacity Management</em>, <em>Change and Release Management</em>) do have good KPI, some of which are based on the theory behind DevOps.</p>
  
  <p>In general, for any business process, you can create a <strong>Value Chain Map</strong> describing how value flows from <strong>Customer</strong>, through the enterprise back to <strong>Customer</strong>. The entire loop always has to start and end with Customer, but sometimes, for a service organization, the Customer can be internal. The <strong>throughput of value</strong> through such chain can be a good way to design your KPI in a tamper-proof way. Measuring any KPI in any individual link of the value chain <em>only makes sense as long as that particular link is the bottleneck</em> of the process and you try to exploit or elevate the bottleneck.</p>
  
  <p>A common problem with KPI is when it starts halfway through the chain. For example, a Change and Release process often starts with developers and ends with deployment. This process excluded: </p>
  
  <ul>
  <li>Customer having a problem</li>
  <li>Support team identifying the problems</li>
  <li>Product team defining the problem for backlog</li>
  <li>Solutions team customizing the deployment for the customer</li>
  <li>Customer realizing value from the solution</li>
  </ul>
  
  <p>The problem is that measuring a cycle time alone could lead to two major problems:</p>
  
  <ol>
  <li><p>The bottleneck is in any of the excluded parts mentioned above and your customers are not realizing the value and you are not realizing the revenue proportional to the speed of your cycle time. So while your engineering is excellent, your business suffers.</p></li>
  <li><p>The disconnect from Customers will make your release cycle spin on empty - not producing any value, despite change being done - or even counter the needs of your Customers and the work being done could have negative business value. </p></li>
  </ol>
  
  <p>Another problem with KPI is that while starting and ending with a customer it might not actually measure the value to the customer. A good example would be a Problem and Incident Response process and MTTR (Mean Time To Repair) as KPI. Does the problem even affect anyone? What is the value to customers? You could have excellent MTTR of 3 hours over 100 incidents. But if most of them were internal, with no or minimal impact to customers and resolved in minutes, while the one large incident with huge customer impact took 3 days to handle, the business value is lower than if you had 1 day MTTR, because you ignore most of the internal incidents, but you responded to the huge customer impact incident within 1 hour.</p>
  
  <p><strong>NOTE:</strong> For an internal customer, in case of the support team business process, the derived value is not the value of the work to the internal customer, but the value gained by the business in unblocking the internal customer in their own business process. Unblocking a team that is a bottleneck in their own process derives higher value than unblocking a non-bottleneck team or individual. All KPI of such support team need to include the business value in their calculation.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-12-26T09:04:35.630Z
Answers: 2017-12-26T09:21:59.777Z
Answers: 2017-12-26T09:21:59.777Z
Answers: Jiri Klouda
Answers: <ol>
  <li>Deployment frequency  </li>
  <li>Speed of deployment                                                     </li>
  <li>Deployment success rate              </li>
  <li>How quickly service can be restored after a failed deployment<br>
  And finally,   </li>
  <li>DevOps Culture, which actually can’t be measured</li>
  </ol>
  
Answers: CC BY-SA 3.0
Answers: 2017-11-23T14:02:12.210Z
Answers: `5.DevOpsCulture, which actually can’t be measured` => make an anonymous questionaire for everyone involved even slightly and ask them how they feel about all of this. This surely will tell you if you have a process that is lived by your people, or if many people are in truth half-way out of the door...
Answers: AnoE
Answers: CC BY-SA 3.0
Answers: 2017-11-22T08:38:23.723Z
Answers: 2017-12-22T08:54:12.670Z
Answers: 2017-12-22T08:54:12.670Z
Answers: Bibin Joseph
Answers: <p>I don't think there are any "universal" DevOps KPIs.  For example, velocity is great, unless it's not a key driver for your business.  Amazon cares a lot about velocity because they have a massive retail operation.  That's less important for a small app with 100 users.</p>
  
  <p>This begs the question: how do <em>you</em> select the best KPIs relevant to <em>your</em> business?  That's a research and discovery process that involves your entire Enterprise.</p>
  
  <p>What do you care about?</p>
  
  <ul>
  <li>Quality</li>
  <li>Reliability</li>
  <li>Maintainability</li>
  <li>Velocity</li>
  <li>Process Improvement</li>
  <li>Service Levels</li>
  </ul>
  
  <p>What keeps your business stakeholders up at night?  What determines whether you make money this quarter or not?  The list above might include some of those things, or it might not.  Make your list, then figure out how to align incentives <em>across every department</em> to achieve them.  </p>
  
  <p>Incentives drive behavior, so decide collaboratively on SMART goals.  Pick two or three items off your brainstormed list, and start a measure/fix feedback cycle for each.  Don't pick too many at once- you're more likely to succeed by focusing hard on two or three things.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-31T14:35:13.213Z
Answers: 2017-03-31T14:35:13.213Z
Answers: Dave Swersky
Body: <p>I am trying to drive good behaviors within a DevOps transformation programme, to support this I am looking at identifying actionable metrics around the operations disciplines:</p>
  
  <ul>
  <li>Problem and Incident Management</li>
  <li>Capacity Management</li>
  <li>Change and Release Management</li>
  </ul>
  
  <p>To be absolutely clear, these are functions that used to belong to the operations organization and are now owned by the Agile/DevOps organization.  There are existing KPIs that drive bad behaviors are:</p>
  
  <ul>
  <li>Time to Root Cause Analysis completed:
  
  <ul>
  <li><em>Drives incomplete RCAs just to get them into the system on time.</em></li>
  </ul></li>
  <li>Test execution duration:
  
  <ul>
  <li><em>Disables long running tests, regardless of their business value.</em></li>
  </ul></li>
  <li>Average utilization of cloud services:
  
  <ul>
  <li><em>Encourages over-commitment of compute resources, resulting in slow response times</em></li>
  </ul></li>
  </ul>
  
  <p><strong>What Key Performance Indicators can be used to encourage good behaviour in a DevOps Programme?</strong></p>
  
Comments: CC BY-SA 3.0
Comments: 2017-03-31T20:30:06.677Z
Comments: You've found the inherent problem with *all* KPIs. People will work to maximize the performance *indicators* instead of maximizing *actual performance*. Metrics give people a score to run up, and they will, even at the cost of doing their jobs well.
Comments: Adrian
Comments: CC BY-SA 3.0
Comments: 2017-03-31T20:34:24.280Z
Comments: @Adrian I agree, however there are certain KPIs, such as cycle time, that are inherently difficult to game.
Comments: Richard Slater
Comments: CC BY-SA 3.0
Comments: 2017-03-31T20:38:11.657Z
Comments: True. Still, even those that are hard to game will lead to optimizing for the KPI, which may be sub-optimal in general, or simply favoring those KPIs that can be gamed. I've found very few ways to measure DevOps performance automatically with metrics; it's mostly subjective and requires personal observation and engagement.
Comments: Adrian
Comments: CC BY-SA 3.0
Comments: 2017-07-17T09:47:43.013Z
Comments: That's not DevOps, it's ITIL haha
Comments: Gaius
Tags: <culture><metrics><kpi>
Title: What Key Performance Indicators (KPIs) are used to measure DevOps?
UserDisplayName: Richard Slater
ThreadText: # What Key Performance Indicators (KPIs) are used to measure DevOps?
  
  (Asked by: username_0 on 2017-03-31)
  
  I am trying to drive good behaviors within a DevOps transformation programme, to support this I am looking at identifying actionable metrics around the operations disciplines:
  
  * Problem and Incident Management
  * Capacity Management
  * Change and Release Management
  
  To be absolutely clear, these are functions that used to belong to the operations organization and are now owned by the Agile/DevOps organization. There are existing KPIs that drive bad behaviors are:
  
  * Time to Root Cause Analysis completed:
    + *Drives incomplete RCAs just to get them into the system on time.*
  * Test execution duration:
    + *Disables long running tests, regardless of their business value.*
  * Average utilization of cloud services:
    + *Encourages over-commitment of compute resources, resulting in slow response times*
  
  **What Key Performance Indicators can be used to encourage good behaviour in a DevOps Programme?**
  
  **Tags:** culture, metrics, kpi
  
  > **Comments:**
  > **username_1**: You've found the inherent problem with *all* KPIs. People will work to maximize the performance *indicators* instead of maximizing *actual performance*. Metrics give people a score to run up, and they will, even at the cost of doing their jobs well.  
  > **username_0**: @username_1 I agree, however there are certain KPIs, such as cycle time, that are inherently difficult to game.  
  > **username_1**: True. Still, even those that are hard to game will lead to optimizing for the KPI, which may be sub-optimal in general, or simply favoring those KPIs that can be gamed. I've found very few ways to measure DevOps performance automatically with metrics; it's mostly subjective and requires personal observation and engagement.  
  > **username_2**: That's not DevOps, it's ITIL haha  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_6 on 2017-03-31. Score: 12 | Accepted answer
  
  I don't think there are any "universal" DevOps KPIs. For example, velocity is great, unless it's not a key driver for your business. Amazon cares a lot about velocity because they have a massive retail operation. That's less important for a small app with 100 users.
  
  This begs the question: how do *you* select the best KPIs relevant to *your* business? That's a research and discovery process that involves your entire Enterprise.
  
  What do you care about?
  
  * Quality
  * Reliability
  * Maintainability
  * Velocity
  * Process Improvement
  * Service Levels
  
  What keeps your business stakeholders up at night? What determines whether you make money this quarter or not? The list above might include some of those things, or it might not. Make your list, then figure out how to align incentives *across every department* to achieve them.
  
  Incentives drive behavior, so decide collaboratively on SMART goals. Pick two or three items off your brainstormed list, and start a measure/fix feedback cycle for each. Don't pick too many at once- you're more likely to succeed by focusing hard on two or three things.
  
  
  ---
  
  ### Answer by username_3 on 2017-12-26. Score: 3
  
  **DevOps** does not have any **KPI**. It would be like asking what are the KPI of Love. But some of the things you mentioned (*Problem and Incident Management*, *Capacity Management*, *Change and Release Management*) do have good KPI, some of which are based on the theory behind DevOps.
  
  In general, for any business process, you can create a **Value Chain Map** describing how value flows from **Customer**, through the enterprise back to **Customer**. The entire loop always has to start and end with Customer, but sometimes, for a service organization, the Customer can be internal. The **throughput of value** through such chain can be a good way to design your KPI in a tamper-proof way. Measuring any KPI in any individual link of the value chain *only makes sense as long as that particular link is the bottleneck* of the process and you try to exploit or elevate the bottleneck.
  
  A common problem with KPI is when it starts halfway through the chain. For example, a Change and Release process often starts with developers and ends with deployment. This process excluded:
  
  * Customer having a problem
  * Support team identifying the problems
  * Product team defining the problem for backlog
  * Solutions team customizing the deployment for the customer
  * Customer realizing value from the solution
  
  The problem is that measuring a cycle time alone could lead to two major problems:
  
  1. The bottleneck is in any of the excluded parts mentioned above and your customers are not realizing the value and you are not realizing the revenue proportional to the speed of your cycle time. So while your engineering is excellent, your business suffers.
  2. The disconnect from Customers will make your release cycle spin on empty - not producing any value, despite change being done - or even counter the needs of your Customers and the work being done could have negative business value.
  
  Another problem with KPI is that while starting and ending with a customer it might not actually measure the value to the customer. A good example would be a Problem and Incident Response process and MTTR (Mean Time To Repair) as KPI. Does the problem even affect anyone? What is the value to customers? You could have excellent MTTR of 3 hours over 100 incidents. But if most of them were internal, with no or minimal impact to customers and resolved in minutes, while the one large incident with huge customer impact took 3 days to handle, the business value is lower than if you had 1 day MTTR, because you ignore most of the internal incidents, but you responded to the huge customer impact incident within 1 hour.
  
  **NOTE:** For an internal customer, in case of the support team business process, the derived value is not the value of the work to the internal customer, but the value gained by the business in unblocking the internal customer in their own business process. Unblocking a team that is a bottleneck in their own process derives higher value than unblocking a non-bottleneck team or individual. All KPI of such support team need to include the business value in their calculation.
  
  
  ---
  
  ### Answer by username_4 on 2017-11-22. Score: 1
  
  1. Deployment frequency
  2. Speed of deployment
  3. Deployment success rate
  4. How quickly service can be restored after a failed deployment  
     And finally,
  5. DevOps Culture, which actually can’t be measured
  
  > **Comments:**
  > **username_5**: `5.DevOpsCulture, which actually can’t be measured` => make an anonymous questionaire for everyone involved even slightly and ask them how they feel about all of this. This surely will tell you if you have a process that is lived by your people, or if many people are in truth half-way out of the door...  
  
  ---
  
  