It's enough to keep IT managers awake at night. The recent media storms over software glitches are causing reputation and financial damage, and have meant that software quality issues have never been more relevant than today.
Poor quality in software can hit an organisation financially. Reputation damage is not always as tangible but nonetheless can hit as hard. If you consider the actual cost of correcting a software glitch and what it can do to the brand, the impact is huge.
Unfortunately, most projects that involve software are inevitably prone to glitches or bugs. When a project is behind schedule or over budget, corners are often cut. So what are the tell-tale signs of a project in trouble? These include business stakeholders forced to accept a reduced scope; project teams under pressure to deliver more rapidly; and external stakeholders demanding results from senior management and the project board.
As stress levels rise, software defects may remain undiscovered. In many cases, critical defects in the basic design or operation of the system are only found mid-way - or even later - during the software development lifecycle.
Quality issues are generally identified as software or application defects and integration problems during the various testing phases of the project. Of course, there are ways to mitigate these issues and some agile approaches can improve quality. However, for the vast majority there are still basic errors in how quality is built into development projects.
In the traditional model, developers build code and pass it on to testers who execute a series of pre-defined tests against it. This approach can only ever discover defects retrospectively - so this traditional process only 'adds quality' to the solution.
That means project managers have to budget for rework time after defects are discovered. However, the later a technical issue is discovered, the higher the cost to resolve it.
The groundwork for the most difficult-to-resolve defects should be laid early on in requirements and design stages. Instead, those defects typically tend to be discovered late (during integration, performance or security testing) when many components are finished or close to completion.
A more effective way to tackle the problem is to build in quality from the outset, seeking to prevent expensive defects from infiltrating the design or requirements phases.
Planning for three cycles of testing is quite common. Unfortunately, discovering that re-tests and rework cause testing to 'spill over' into later test stages is also quite common too. As rework is discovered, quality assurance costs rise because much of the test team's time is spent testing or re-testing reworked code. Accordingly, the proportion of time that testers spend focussing on new functionality or system behaviour can drop dramatically.
Treating all products delivered through the project lifecycle as testable (assurable) items is a better approach. Just like the code, documents and other artefacts are part of the overall solution, improving quality by addressing technical and business risk factors early in the lifecycle delivers higher quality software and projects.
The earlier a defect is introduced, the higher the ultimate cost of removing that defect (e.g. removing defects introduced at the requirements stage can cost 100 times more than defects introduced during integration).
Starting assurance earlier in the lifecycle may mean that quality assurance resource is engaged on projects for longer, but the reduction in workload towards the end of the project compensates for this (because fewer defects are discovered and re-tested). Catching just one major issue early on could make the difference between on time and overrun or smooth end-customer experience compared to a complete system meltdown.
It seems ironic that one of the best ways to reduce spend on testing effort is to do more testing, albeit of a different sort than most test and project managers are accustomed to. That means software quality assurance as an on-going activity, not a bolt-on after software has been built.
Mindset is critical. We often hear of the SDLC - Software Development Lifecycle, which tends to focus the attention on the software, instead, we should think in terms of a solution delivery lifecycle, in which all project/programme artefacts form part of the solution and are subject to scrutiny by testers.
Identify the key artefacts produced during the Solution Delivery Lifecycle, and define the business and technical risk based acceptance criteria for them.
Coach the testers in the review process that you follow - these can range from full blown Fagan type inspections that appeared in the late 1970s to informal reviews. Exercise caution here: it is all too easy for this kind of assurance activity to turn into a box-ticking exercise and assurers need to have people skills and objectivity.
Record the outcomes of inspections or reviews as you would for a software defect, assigning priorities to the defects; if using a defect tracking tool, record the fact that they were raised following and inspection and identifying the artefact that they were found in. Independence from the project is important to ensure objectivity and assurers should not normally be part of the project delivery team.