Challenge

I was the lead engineer for a software solution that scientists in our  company used in order to prepare clinical reports for their clients. Their  clients were oncologists who used the report to make clinical decisions  regarding treatment for their patients. The input toward the creation of a  report was a patient diagnosis and a list of genetic observations driving that  patient’s cancer. The output was a list of individualized treatment   recommendations given knowledge stored in the organization’s knowledge  base.

Due to technical difficulties, the scientists were observing that the software  was returning the wrong treatment recommendations. As a result, the  scientists were being forced to manually fix the same errors over and over  while under pressure from their oncology clients who needed the report  urgently.

Selection

To solve the reporting error problem, I used storytelling. In specific, I  worked with scientists and engineers to put together ‘patient report  journeys’, which is a type of ‘journey mapping’. Journey mapping is a tool to  capture the steps that a client goes through in order to accomplish a goal.  Since the intent of the software was to automate the steps that a scientist  would go through to produce the final report based on a set of decision  rules, journey mapping was the appropriate tool as it could provide a way to  visually contrast the steps and decision points that a scientist would take in  order to put together a high quality report vs how the software was  implementing them.

Application

My team and I needed to understand and map the journey of a patient  report through several decision points in order to understand where the  wrong information was being added or omitted given the inputs and  expected outputs.

Since the errors were only happening in some reports, we needed to create  several ‘patient report journeys’ (including inputs and expected outputs).

For example, if the patient had been diagnosed with melanoma and the  genetic observations revealed a mutation in the BRAF gene, we would map  out every decision point where a treatment was added or removed from the  report given the information provided in the previous step—we called these  ‘decision gates’. We would then compare the software generated output  against the expected output in order to identify the decision gate at which  the wrong output was being created. For example “does the knowledge- base have a recommended treatment given this specific mutation in the  BRAF gene in melanoma? If yes, output the recommended treatment(s),  otherwise output ‘no recommendation’ text”.

By comparing the desired patient report journeys against those produced  by the scientists, we learned that the treatment recommendation was being  manually removed from the knowledge-base by the scientists in some  cases - whereas their intent had been to remove the recommendation for  one specific patient report (eg. because the patient had already received  that therapy in the past), the result was that the recommendation was being  removed from all subsequent reports, thus causing the missing  recommendations that needed to be fixed.

The journey mapping approach was effective in creating a shared  understanding between the engineers and the scientists on the decision  points and their sequence - once these decision steps were mapped out,  the scientists could finally understand exactly why the software was  producing a given output. As a result, the combined team agreed on new  decision rules to be created to handle special cases (eg. the scientist would  mark a recommendation removal as “exception” and the software would  respond by treating that report differently). Through several iterations of  focusing on revising the decision gates, the combined team was able to  obtain a long term shared understanding that avoided future errors and  empowered the scientists with a tool to communicate with the engineers.

This approach could have been even more effective if, instead of trying to  put together the journey maps after the software was in production, these  had been prepared and aligned with the scientists before the software was  created.

Insight & Approach

As a engineering lead, when I am trying to understand what end-users  need to accomplish with a given tool, it’s hard to keep track of all the edge  cases where the tool has to respond to the user’s expectations. As such, it  is tempting to ignore the edge cases and instead to create software  specifications that address the majority of cases using the 80/20 rule (i.e.  automate for 80% of the problems, solve the other 20% manually). The  journey mapping tool helped me understand how important it is to collect  not only the use case majority as a journey map but also pinpoint where the  edge cases deviate from the norm. This way, even if the software team  cannot automate every single edge case, a journey map is an extremely  powerful communication tool to align on what the software can and cannot  do.

The journey mapping approach was useful to identify points of  misalignment between the expectation and the product but it was used  retrospectively once problems emerged as opposed to proactively before  the implementation began. If it had been put in place ahead of any software   implementation, it would have avoided significant frustration, delays  and  re-work.


Challenge I was the lead engineer for a software solution that scientists in our company used in order to prepare clinical reports for their clients . Their clients were oncologists who used the report to make clinical decisions regarding treatment for their patients . The input toward the creation of a report was a patient diagnosis and a list of genetic observations driving that patient ’s cancer . The output was a list of individualized treatment recommendations given knowledge stored in the organization ’s knowledge base . Due to technical difficulties , the scientists were observing that the software was returning the wrong treatment recommendations . As a result , the scientists were being forced to manually fix the same errors over and over while under pressure from their oncology clients who needed the report urgently . Selection To solve the reporting error problem , I used storytelling . In specific , I worked with scientists and engineers to put together ‘ patient report journeys ’ , which is a type of ‘ journey mapping ’ . Journey mapping is a tool to capture the steps that a client goes through in order to accomplish a goal . Since the intent of the software was to automate the steps that a scientist would go through to produce the final report based on a set of decision rules , journey mapping was the appropriate tool as it could provide a way to visually contrast the steps and decision points that a scientist would take in order to put together a high quality report vs how the software was implementing them . Application My team and I needed to understand and map the journey of a patient report through several decision points in order to understand where the wrong information was being added or omitted given the inputs and expected outputs . Since the errors were only happening in some reports , we needed to create several ‘ patient report journeys ’ ( including inputs and expected outputs ) . For example , if the patient had been diagnosed with melanoma and the genetic observations revealed a mutation in the BRAF gene , we would map out every decision point where a treatment was added or removed from the report given the information provided in the previous step — we called these ‘ decision gates ’ . We would then compare the software generated output against the expected output in order to identify the decision gate at which the wrong output was being created . For example “ does the knowledge- base have a recommended treatment given this specific mutation in the BRAF gene in melanoma ? If yes , output the recommended treatment(s ) , otherwise output ‘ no recommendation ’ text ” . By comparing the desired patient report journeys against those produced by the scientists , we learned that the treatment recommendation was being manually removed from the knowledge - base by the scientists in some cases - whereas their intent had been to remove the recommendation for one specific patient report ( eg . because the patient had already received that therapy in the past ) , the result was that the recommendation was being removed from all subsequent reports , thus causing the missing recommendations that needed to be fixed . The journey mapping approach was effective in creating a shared understanding between the engineers and the scientists on the decision points and their sequence - once these decision steps were mapped out , the scientists could finally understand exactly why the software was producing a given output . As a result , the combined team agreed on new decision rules to be created to handle special cases ( eg . the scientist would mark a recommendation removal as “ exception ” and the software would respond by treating that report differently ) . Through several iterations of focusing on revising the decision gates , the combined team was able to obtain a long term shared understanding that avoided future errors and empowered the scientists with a tool to communicate with the engineers . This approach could have been even more effective if , instead of trying to put together the journey maps after the software was in production , these had been prepared and aligned with the scientists before the software was created . Insight & Approach As a engineering lead , when I am trying to understand what end - users need to accomplish with a given tool , it ’s hard to keep track of all the edge cases where the tool has to respond to the user ’s expectations . As such , it is tempting to ignore the edge cases and instead to create software specifications that address the majority of cases using the 80/20 rule ( i.e. automate for 80 % of the problems , solve the other 20 % manually ) . The journey mapping tool helped me understand how important it is to collect not only the use case majority as a journey map but also pinpoint where the edge cases deviate from the norm . This way , even if the software team can not automate every single edge case , a journey map is an extremely powerful communication tool to align on what the software can and can not do . The journey mapping approach was useful to identify points of misalignment between the expectation and the product but it was used retrospectively once problems emerged as opposed to proactively before the implementation began . If it had been put in place ahead of any software implementation , it would have avoided significant frustration , delays and re - work .