<http://www.w3.org/2009/08/skos-reference/skos.html#note>
"THE DEVELOPMENT OF A DISINVESTMENT FRAMEWORK TO GUIDE RESOURCE ALLOCATION DECISIONS IN HEALTH SERVICE DELIVERY ORGANIZATIONS  by Diane E. Schmidt  B.A., Simon Fraser University, 2010  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF SCIENCE in (Health Care and Epidemiology)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  October 2012  \u00A9 Diane E. Schmidt, 2012  \u000CABSTRACT Objective: Within publicly funded health care, there is an urgent need to reduce spending while maximizing benefits; however, processes to guide so-called disinvestment decisions are lacking. The purpose of this research is to develop a framework that will provide decision-makers with a more equitable approach to meeting budgetary constraints than current practices.  Methods: Through a systematic review of the health care literature and a scoping review of the public sector and business literatures, a knowledge synthesis of disinvestment approaches was created, including analyses of current strategies and the modeling of appropriate processes. From this synthesis, a disinvestment framework has been developed. In collaboration with Chief Financial Officers from across Western Canada and an external reference group comprised of international researchers, the framework has been critiqued in keeping with current resource allocation practices.  Results: Evidence from the two reviews revealed that while budgetary cutbacks are experienced across government, non-profit and the private sector, very few processes have been developed to identify and implement disinvestment options. In cases of budget reallocation, program budgeting and marginal analysis (PBMA) was the most relevant framework described. However, PBMA fails to address stand-alone disinvestment requirements. Within the public sector and business literatures, cutback management and policy termination research offered strategies to mitigate barriers and facilitate implementation, however, details were absent. Drawing elements from the approaches ii  \u000Cidentified in the reviews, and in collaboration with decision-makers and other researchers, a seven-step disinvestment framework was developed that can be incorporated into on-going priority setting practices or applied as a stand-alone activity.  Conclusion: This work addresses a critical knowledge gap in how health service organizations approach disinvestment activities. The proposed framework provides detailed steps to equip health care decision makers with a clear and defined disinvestment process. Such a process will help to ensure limited funds are allocated based on evidence rather than across-the-board cuts or historical practices.  iii  \u000CPREFACE  This research represents the results of a knowledge synthesis project, led by Dr. Craig Mitton (PhD, University of Calgary).  The one year study, Synthesizing knowledge on  disinvestment processes in health care, was funded by the Canadian Institutes for Health Research. To help guide the work, the research was supported by Dr. Iestyn Williams (PhD, University of Birmingham) and Mr. Duncan Campbell (MBA, University of Witwatersrand) and by the External Reference Group comprised of Dr. Marion Danis (MD, University of Chicago), Dr. Adam Elshaug (PhD, the University of Adelaide), Dr. Marthe Gold (M.D., Tufts University School of Medicine; M.P.H., Columbia University School of Public Health) and Dr. Suzanne Robinson (PhD, University of Birmingham).  Under the direct supervision of Dr. Craig Mitton (PhD, University of Calgary) and co-supervision of Dr. Stirling Bryan (PhD, Brunel University) and Dr. Stuart Peacock (PhD, University of York), Ms. Schmidt conducted the following research activities:  1. Systematic review and scoping review: Schmidt led the research team in determining inclusion criteria, databases searched, key words used and search protocols followed; she read and reviewed 80% of the abstracts and all of the full articles; Lisa Mascucci (MSc, McMaster University) was the second reviewer. 2. Data extraction and analyses: Schmidt developed the data extraction tool and she was responsible for the data extraction and synthesis. Schmidt conducted the crossiv  \u000Ctab analysis and performed a thematic analysis to determine components and attributes of a disinvestment process. Feedback from Drs. Mitton, Bryan, Peacock, Williams, Danis, Elshaug, Gold, Robinson and Mr. Campbell were subsequently incorporated into the thesis.  v  \u000CTABLE OF CONTENTS ABSTRACT ............................................................................................................................. ii PREFACE ............................................................................................................................... iv TABLE OF CONTENTS ...................................................................................................... vi LIST OF TABLES ................................................................................................................. ix LIST OF FIGURES ................................................................................................................ x ACKNOWLEDGEMENTS .................................................................................................. xi DEDICATION....................................................................................................................... xii 1.  INTRODUCTION........................................................................................................... 1  1.1  Resource allocation and the rationing of health care .......................................... 1  1.2  How to allocate scarce resources ........................................................................... 2  1.2.1  Economic evaluation ................................................................................................. 2  1.2.2  Decision making theory ............................................................................................ 4  1.2.3  Types of decision making models............................................................................. 5  1.3  Health care decision making during fiscal constraint ......................................... 9  1.4  Definitions of disinvestment ................................................................................. 12  1.5  Relevance of the Research .................................................................................... 13  2.  STUDY OBJECTIVES................................................................................................. 16  3.  METHODS .................................................................................................................... 17  3.1  Project planning .................................................................................................... 18  3.2  Search terms .......................................................................................................... 19  3.3  Article eligibility .................................................................................................... 21  3.4  Assessment and analysis ....................................................................................... 23  vi  \u000C4.  RESULTS ...................................................................................................................... 26  4.1  Disinvestment definitions ..................................................................................... 30  4.1.1  Comparison of definition use and its application ................................................... 32  4.1.2  Definition by time period ........................................................................................ 34  4.1.3  Definition by region ................................................................................................ 34  4.2  Disinvestment processes ....................................................................................... 35  4.2.1  Process attributes .................................................................................................... 41  4.2.2  Phases and components........................................................................................... 45  4.3  Challenges to disinvestment ................................................................................. 52  4.4  Facilitators to disinvestment ................................................................................ 55  4.5  Framework design ................................................................................................ 56  4.5.1 4.6  Defining the key attributes ...................................................................................... 57 Development of the disinvestment framework ................................................... 59  4.6.1  Process participants ................................................................................................. 61  4.6.2  Identification ........................................................................................................... 62  4.6.3  Selection .................................................................................................................. 63  4.6.4  Implementation ....................................................................................................... 64  4.6.5  Evaluation ............................................................................................................... 65  5.  DISCUSSION ................................................................................................................ 70  5.1 5.1.1  Definition of disinvestment................................................................................... 70 Proposal of a new definition ................................................................................... 72  5.2  Disinvestment processes ....................................................................................... 74  5.3  Development of a disinvestment framework ...................................................... 77  vii  \u000C6.  LIMITATIONS ............................................................................................................. 80  7.  CONCLUSIONS ........................................................................................................... 81  REFERENCES ...................................................................................................................... 84 APPENDIX A ........................................................................................................................ 93 APPENDIX B ........................................................................................................................ 95  viii  \u000CLIST OF TABLES Table 1: Description of project support teams ........................................................................ 18 Table 2: Key health care articles identified to guide literature search .................................... 20 Table 3: Disinvestment definitions identified from key papers .............................................. 24 Table 4: Articles identified from a systematic review of the health care literature ................ 27 Table 5: Articles identified from a scoping review of the public sector/business literature ... 29 Table 6: Characteristics of a disinvestment activity by sector ................................................ 31 Table 7: Processes identified from the health care and public sector/business literatures .... 36 Table 8: Attributes of a disinvestment process ....................................................................... 40 Table 9: Phases and key components of a disinvestment process .......................................... 46 Table 10: Key features identified from the literature and used to develop a framework ....... 57 Table 11: Detailed description of the disinvestment framework ............................................ 67  ix  \u000CLIST OF FIGURES Figure 1: Flow charts to show assessment of identified peer-reviewed papers ...................... 26 Figure 2: Phases and components of a disinvestment process ................................................ 39 Figure 3: Disinvestment framework ....................................................................................... 61  x  \u000CACKNOWLEDGEMENTS  I want to thank the members of my thesis committee, Craig Mitton, Stirling Bryan and Stuart Peacock for providing feedback and guidance and for asking pertinent questions that always moved my work forward. I would also like to extend my thanks to Iestyn Williams and Duncan Campbell, both of whom were active members of the Disinvestment Group and whose wisdom and humor supported the research far beyond its humble beginnings. To members of the External Reference Group \u2013 Marion Danis, Suzanne Robinson, Adam Elshaug and Marthe Gold \u2013 for their on-going support and invaluable feedback throughout the research process, many thanks. Finally, I would like to express my deep gratitude to Craig Mitton, my thesis supervisor, who gave me the freedom to pursue this project in my own way, yet was always there to answer questions, respond to my emails and provide the necessary guidance needed to complete this project. Thank you for everything.  I would also like to thank my funder, the Canadian Institutes for Health Research (CIHR), who supported this project through a graduate award and for providing travel funds that allowed me to present this research at a national conference.  This acknowledgement would not be complete without thanking Garth, who has supported me in every possible way since we met that cold, snowy February night. Thanks for picking up the slack, for never complaining whenever I uttered the words \u201CI don\u2019t have time\u201D, and for always believing in me no matter how many curve balls I might throw. You\u2019re my heart.  xi  \u000CDEDICATION  For my mother, who would have excelled in school had she been given the chance.  xii  \u000C1. INTRODUCTION  1.1  Resource allocation and the rationing of health care  When referring to the rationing of health care, there is an underlying assumption that health care, like most societal goods, is limited in quantity, yet needs to extend far beyond what the system can provide. Even if one devoted an entire country\u2019s Gross Domestic Product to providing health care, there would still not be enough resources to meet the needs of what individuals require in order to be healthy (Mariner, 1997; Evans, 2007). Part of the issue is, of course, that good health, which is the goal of health care, is tied not only to the supply of health care resources, but to factors outside of the health system: individual behavior, societal structures and genetic makeup also play a role in determining one\u2019s health (Mariner, 1997). In this way health is unlike other societal goods, such as clean water, public transportation and to some extent, public education, all of which can be distributed more easily with outcomes that tend to be fixed rather than dynamic.  If supply cannot meet demand, then a good is considered scarce. When a good is scarce, rationing is necessary. In Canada, such rationing occurs first at the Federal and provincial government level where the size of the health care funding pot \u2013 how many dollars are allocated to health care services as opposed to other societal goods such as education, job creation or defense \u2013 is decided. The next level of rationing occurs at the regional level wherein service provider organizations decide which combination of services, programs and interventions will be supported. At each phase as the process moves from macro-allocation to micro-allocation decisions are implemented as to what to fund and what not to fund and trade1  \u000Coffs are made. It is clear that when funds are allocated to one area, another area \u2018loses out\u2019. For example, if funding is allocated to increase the number of hip replacement surgeries, then perhaps healthy eating programs will need to be reduced. Essentially, how decision makers choose to spend money is the true determinant of which health care services are available to whom.  1.2  How to allocate scarce resources  1.2.1 Economic evaluation  Once the amount of funding dollars is determined, health care service providers must then decide how those dollars are to be spent. Economic evaluation is one of the most popular ways of determining how resources should be allocated, offering techniques that can be used to measure the cost of providing a service or intervention against the benefit gained. Techniques used to conduct economic evaluation include cost-benefit analysis (CBA), costeffectiveness analysis (CEA) and cost-utility analysis (CUA).  In health economics, cost-benefit analysis requires that all costs and all benefits accrued are measured monetarily (Donaldson, Currie &amp; Mitton, 2002).  The goal is to  determine which of the options being assessed provides the greatest net benefit. The decision rule lies where the net benefit is positive, that is, the benefits when tallied are greater than the costs required to provide that benefit.  2  \u000CUnlike CBA, cost-effectiveness analysis takes into account the efficiency of the option over an alternative and thereby determines if one option provides greater benefit than the other and at a lower cost (Banta &amp; de Wit, 2008). Cost-utility analysis was developed as a further step to CEA whereby health benefits are measured using Quality-Adjusted-Life-Years (QALYs) (Poetz and Hoch, 2005). QALYs are a composite measure that accounts for life expectancy and health-related quality of life gains which each additional QALY measured by the incremental cost-effectiveness ratio (ICER). However, problems arise when attempting to assess outcomes other than health gains. Further arguments contend that the use of the QALY does not reflect what society values and cannot measure (Schwappach, 2002).  While economic evaluation approaches enable the decision maker to assess resource allocation options through the quantification of specific objectives such as increases in healthrelated quality of life, such assessments become problematic when decisions must be made based on more than one objective. Health-related quality of life and quantifiable benefits are important factors when determining where funds should be allocated; however, within complex systems such as a health service delivery organization additional objectives may be equally important - objectives such as access, equity, and appropriateness.  To ensure  decisions are closely aligned to an organization\u2019s mandate, strategic direction and goals, a multifaceted process is required, one that permits the inclusion of multiple objectives to assess and evaluate a set of diverse services and programs within the context of a dynamic system.  3  \u000C1.2.2 Decision making theory  If it is agreed that health care is a scarce resource requiring necessary trade-offs, resource allocation processes need to be responsive to the complex environments within which such difficult decisions are made. As such, decision making processes should be analyzed in order to determine context-specific factors that influence how decisions are made and whether such decision making efforts can be improved upon.  There are several types of decision making process models that have been developed in other sectors such as education and public management research, yet are applied to health care organizations regardless of their ability to work within such a complex system. In this section, resource allocation in public organizations is examined followed by a description and critique of decision making models to determine whether these models can be applied within the context of a health service delivery organization.  Decision making in public versus private organizations  While public organizations may receive funding from private sources, for the most part it can be said that the government \u2018owns\u2019 the organizations in that they are managed by government authority with the majority of funding generated from public sources, such as taxation legislation (Rainey, Ronquillo &amp; Avellaneda, 2010). For private organizations, since their funding is derived primarily from sales, they are in most respects ruled by the market, forcing companies to produce what customers want.  4  \u000CPublic organizations, while needing to be responsive to public wants and needs, are operated under the authority of government officials who determine how dollars are to be allocated, deciding what is produced, how it is produced and for whom. In this respect, as Nutt and Backoff suggest (1992), the relationship with the consumer is more contentious and coercive than for private companies, with public members having less power to control the production and distribution of goods. Since elected and appointed officials are responsible for decisions that affect a large number of people, decision makers must consider the implications of such a decision across a broad spectrum of the population.  It is because of these differences between public and private organizations that scholars suggest that public organizations should have distinct decision making processes (Rainey et al, 2010). This is not to say that public organizations cannot learn from how decisions are made in the private realm; rather that public decision making processes must accommodate more dynamic and complex circumstances and be responsible and reportable to a broader population base.  1.2.3 Types of decision making models  Rational model  The rational decision making model is considered to be normative in that it is prescriptive rather than descriptive: it prescribes the conditions under which decisions should be made wherein the problem is identified, an objective is determined, options are weighed and a choice is made that will maximize the outcome (Tarter &amp; Hoy, 1998). The model rests  5  \u000Con facts and therefore discards any variable that cannot be quantified. As such, decisions are made with a single, fixed outcome and a fixed number of variables (Harrison, 1993). It is highly structured and tends to focus on short-term goals. Some of the problems with implementing a rational model within a health care decision making context are: organizational goals are often dynamic; the ability to assess multiple alternatives is limited and variables cannot always be fixed nor are they necessarily quantifiable. Further, all the facts that are needed to make a \u2018rational\u2019 decision under this model are not always available; evidence may be lacking or of poor quality.  The response to these types of limitations and their effect on decision making processes is referred to as the strategy of satisficing (Simon, 1993, as cited in Tarter &amp; Hoy, 1998). That is, within such complex systems, where options are difficult to determine and are too many to evaluate fully, decision makers are overwhelmed by the plethora of information they must assess. Further, because decision makers enter the decision making process with their own biases, their own understanding of the process\u2019s purpose, they are not able to make fully rational decisions. Instead, they attempt to satisfy and in doing so, they seek solutions that will be satisfactory, rather than seeking the best possible outcome.  Incremental model  The incremental model of decision making, which was developed to improve upon the rational model and its need for \u2018all the facts\u2019, is often referred to as simply, \u2018muddling through\u2019 (Rainey et al, 2010). Unlike the rational model, the incremental model assumes the decision maker does not have an ideal goal; rather that programs, services or policies are 6  \u000Calready in place and decisions need to be made based on incremental changes (Tarter &amp; Hoy, 1998). To ensure decision makers are not overwhelmed by options, only a few are considered and their potential consequences evaluated to determine where incremental changes can occur. Critics of the model argue that this approach supports only short-term changes and deters the decision making process (Rainey et al, 2010). Further, it tends to assume that current services, programs and policies are satisfactory since it does not promote their full review and assessment. Clear objectives are not used to make choices which may result in decision maker biases playing a large role in how programs, services or policies are modified (Smith &amp; May, 1980).  Mixed scanning model  With the limitations of an incremental model leading decision makers to adopt a muddling-through strategy, Etizioni (1986) suggested combining incremental with rational decision making models to form what he termed a mixed scanning model. In this model, a well-defined objective guides the decision making and only options that map to the objectives are considered and implemented. Information required to make decisions is sparse rather than complete, which makes it easier for decision makers to examine and evaluate the evidence compared to the requirements of the rational model.  According to Thomas (as cited in Tarter &amp; Hoy, 1998), the mixed scanning model has its origins in medicine wherein physicians must make incremental decisions when choosing treatment for their patients: the objective is to move the patient from illness to health. Yet, in treating the patient, a physician does not necessarily wait until the test results are available, or 7  \u000Cin cases where no test is available, treatment is often recommended based on symptoms. If treatment fails, then an alternative intervention is suggested.  Even though a mixed-scanning approach enables decision makers to focus on fewer options thus simplifying the process, in health care organizations where common goals of effective and efficient care are supported, the assessment of a select number of services and programs can be problematic when considering how funds should be distributed. Further, if services or programs are \u2018tried\u2019 and \u2018fail\u2019 precious resource dollars are lost. While the model itself may not be applicable to a health care setting, the idea that good decisions can be made with less information is one to bring forward into the decision making process.  Garbage can model  In this model, the garbage can is used as a metaphor to describe the disorganized yet diverse components that go into decision making processes, understanding that organizational features such as preferences, technology and process participants are never fixed (Rainey et al, 2010). At the same time, the complexity of environments such as the one found within health service delivery organizations tend to be more dynamic than static, with issues arising then abating, while evidence points to one solution this month and rapidly changes the next. In this regard, the garbage can model posits that decisions should be made when the right problem, the right solution and the ideal political situation arises. For health care organizations such ideal conditions under which the garbage can model can be applied are few, with decisions such as the allocation of scarce resources occurring on a time-sensitive basis. It is difficult to  8  \u000Censure that all three requirements as described will ever be perfectly aligned when resources are being assigned.  If economic evaluation is insufficient as a mechanism for selection of alternatives in health care resource distribution, and decision making models as previously described are of limited effectiveness when determining how to spend funding dollars, then the allocation of resources is even more problematic in an era of fiscal decline. In such a fiscal context, with less resources to spend than previously, decision makers must not only decide how scarce resources will be spent, but also which services, programs or interventions will be scaled back or removed, that is disinvested from, in order to meet budgetary shortfalls. Since it is more difficult to disinvest than invest, processes to aid such decision making are essential to ensure a fair and effective distribution of scarce health care resources.  1.3  Health care decision making during fiscal constraint  Currently, with the economic downturn still affecting global economies, governments are in a precarious position: there is an urgent need to reduce spending while attempting to maximize benefits from a more limited pool of resources. In an era of government cutbacks, it is critical that decision makers understand how to invest and equally how to disinvest to ensure health care systems remain fiscally viable (Peacock, Ruta, Mitton, Donaldson, Bate &amp; Murtagh, 2006)  While the goal of health care disinvestment is evident, the processes used to achieve this goal are complex and fraught with political and ethical challenges (Pearson &amp; Littlejohns, 9  \u000C2007).  All too often there is no formal process and very little guidance as to how  disinvestment decisions should be made. Without such guidance decision makers often defer to historical practices or implement \u2018across the board\u2019 cuts in an attempt to adopt a more systematic approach (Donaldson, Bate, Mitton, Dionne &amp; Ruta, 2010).  While applying  uniform reductions across programs and services may appear to be equitable, such practices often lead to increasing inequities or a deepening of historical injustices under the rationale of \u2018sharing the pain\u2019 equally.  The most common response to budget restrictions is to identify areas where costs can be trimmed often expressed through cost savings efforts such as reducing services or finding ways to curb spending. Some researchers have described this process as limiting the water flow rather than doing what is required: stopping the water altogether (Elshaug, Hiller, Tunis &amp; Moss, 2007). When an organization engages in cost savings activities the goal is to lower expenses within a program, service or intervention compared to historical costs while maintaining or improving health outcomes; it is also referred to as cost containment (Dionne, Mitton, Shoveller, Peacock &amp; Barer, 2009). Common cost savings tools identified in the literature include service outsourcing, re-engineering, and clinical or non-clinical integration, consolidation or standardization of service areas. Such engagement may assist in reducing the overall budget of an organization, but it is often not sufficient to meet budgetary needs.  Disinvestment, on the other hand, does not seek to decrease the cost of treatments or services in terms of the dollars needed to provide said treatment or service; rather, its goal is to reduce the overall budget. To meet budgetary constraints, an organization may first decide to engage in cost savings measures, and if such measures are not enough to meet budgetary 10  \u000Cneeds, then disinvestment activities could be introduced. However, disinvestment may also be applied as a first response to budget cuts independent of cost savings efforts or be used to freeup resources that can then be re-invested into existing programs or new initiatives.  One of the greatest challenges to implementing disinvestment strategies recognized by researchers and decision makers alike is the strong opposition to the removal of long-running health care services and the political fall-out that ensues (Elshaug et al, 2007; Pearson &amp; Littlejohns, 2007; Robinson, Dickinson, Freeman &amp; Williams, 2011). Patients, caregivers, health care providers and stakeholder groups, all of whom may be impacted by disinvestment decisions are strong advocates against such changes. While opposition will always be part of health care priority setting, having disinvestment strategies in place, with clear guidelines, criteria and decision making pathways, should assist in fostering a fair and transparent process. Such transparency will facilitate civic engagement and promote public support while also increasing political accountability.  With increasing recognition that disinvestment needs to be included in health care priority setting, both as a means to ensure that ineffective services are no longer funded and that services found to be of lower value are scaled back, the inclusion of an evidence-based disinvestment mechanism is essential (Peacock et al, 2006; Pearson &amp; Littlejohns, 2007). It appears that very few disinvestment tools are available that encompass all the complexities of priority setting from the purview of a central decision making body.  11  \u000C1.4  Definitions of disinvestment  In 2002, a report from the UK\u2019s Health Select Committee described the need to remove ineffective interventions and maximize efficiency in the delivery of health care services (National Institute of Clinical Excellence, 2002). This call to action was taken forward by the Chief Medical Officer for England in the 2005 Annual Report that recommended the National Institute for Clinical Excellence (NICE) provide guidance to the National Health Service (NHS) \u201C\u2026on disinvestment, away from established interventions that are no longer appropriate or effective, or do not provide value for money\u201D (Department of Health, 2006, p. 15). In response, NICE developed a program to aid the NHS in disinvesting from areas that do not provide good value in order to free up resources to invest elsewhere (National Institute for Health and Clinical Excellence, 2006).  Since its introduction, other countries have embraced this definition of disinvestment with activities focused primarily on health technologies. In Europe, for example, the European Network for Health Technology Assessment (EUnetHTA) acknowledged that \u201C\u2026new health technologies\u2026be adopted and obsolete technologies abandoned in a well-informed and robust manner, hence bringing about high quality, safe, accessible, sustainable, ethical and efficient health care for citizens across Europe\u201D (EUnetHTA, 2008, p.6). Elshaug et al provide an Australian perspective and have developed proposals to disinvest by \u201C\u2026withdrawing health resources from existing healthcare practices, procedures, technologies, or pharmaceuticals that are deemed to deliver little or no health gain for their cost, and thus do not represent efficient health resource allocation\u201D (Elshaug, Moss, Littlejohns, Karnon, Merlin &amp; Hiller, 2009a, p. 2). Similar approaches have been adopted to delist services in Canada and to identify obsolete 12  \u000Ctechnologies in Spain and Italy (Elshaug, Watt, Moss &amp; Hiller, 2009b; Ibargoyen-Roteta, Gutierrez-Iborluzea &amp; Asua, 2010; Nuti, Vainieri &amp; Bonini, 2010).  Several researchers have suggested that disinvestment activity which focuses on tradeoffs and scaling back of effective services is important (Donaldson et al, 2010; Rich, Leonard, Zalmanovitch &amp; Vashdi, 2010). The focus here moves from \u2018ineffective\u2019 to \u2018effective, but less effective than other services\u2019. It is the relative value, and thus the need for making trade-offs that becomes paramount in the priority setting endeavor. Pearson and Littlejohns (2007) provide a definition of disinvestment in line with this strategy by describing the process as \u201C\u2026taking resources from one service in order to use them for other purposes that are believed to be of better value\u201D (p.160). Such practice is a critical step in any resource allocation decision, in particular when resources are scarce (Donaldson et al, 2010).  However,  acceptance and inclusion of lower value service reduction within health care organizations is often overlooked, particularly for clinicians who might give greater emphasis to service effectiveness rather than cost.  1.5  Relevance of the Research  It is evident that disinvestment decisions require a more thorough analysis to ensure limited resources are allocated to services and programs that will achieve the best possible health outcomes. Such analysis must involve a firm understanding of what disinvestment means, including concrete evidence of effective strategies and the adoption of appropriate tools to help guide the process.  13  \u000CThrough the analysis of how disinvestment is described and used within the health care research, a common definition is developed and adopted to ensure that when disinvestment practices are applied they are done so with a common understanding of what it means to \u2018disinvest\u2019. Without such understanding appropriate measures may not be undertaken. From a thematic analysis of current strategies for health care disinvestment and the subsequent modeling of appropriate processes, this research also establishes stronger practices for evaluation and assessment of health care services. Further, it provides evidence-based support for policy development around disinvestment initiatives.  With increasing recognition that disinvestment needs to be included in health care priority setting, both as a means of meeting budget constraints and to ensure that ineffective services are no longer funded, the inclusion of an evidence-based disinvestment mechanism in the decision making process is essential. Tools have been developed for specific areas of health care services, in particular to assess health care technologies, pharmaceuticals and clinical practices (Hughes &amp; Ferner, 2010; Mortimer, 2010). However, it appears that no disinvestment process encompasses all the complexities of resource allocation in times of fiscal constraint.  Perhaps the most significant contribution of this research is the development of a disinvestment framework to guide health care decision makers. This framework seeks to mediate the challenges faced when adopting disinvestment practices and supports the inclusion of disinvestment activities into annual budgetary cycles to ensure that health service systems are fiscally viable and remain sustainable. Equipping decision makers with an  14  \u000Ceffective tool will ensure that limited resources are allocated appropriately based on evidence rather than historical or political considerations.  15  \u000C2. STUDY OBJECTIVES  There are three objectives this research will undertake:  1) Determine how disinvestment is understood and used by researchers and decisionmakers in priority setting contexts; create a classification of approaches.  2) Identify examples of disinvestment practices used in resource allocation decisions; catalogue contexts and mechanisms involved; challenges and facilitators will also be collected to determine the applicability of such practices.  3) Develop a disinvestment framework to facilitate decision making processes, including key attributes and components as drawn from the identified practices.  16  \u000C3. METHODS  The objectives as previously described were met through a systematic review of the health care literature and supplemented by a scoping review of the public sector and business literatures. A systematic review was undertaken to ensure all relevant disinvestment research from the health care sector was captured. Research from the public sector and business literature was thought to contain a greater range of tools and processes to get at disinvestment activities, and as such, it may better inform disinvestment practices within health care organizations. Working from this assumption and because of unfamiliarity with the research from these sectors, the project team decided to approach the public sector and business literatures through a scoping review from which a broad range of relevant information could be identified and applied herein.  Assessing how disinvestment is defined and applied in the literature is the first objective of this research project. In order to clarify and refine the concept of disinvestment, the research underwent an iterative process involving review and revision of search protocols at each step. In order to capture how disinvestment is defined, discussed and used in the context of health care, business and the public sector, information was extracted using predetermined questions and cross-tab analyses were conducted to synthesize the data. These analyses facilitated the identification and comparison of themes and disinvestment approaches across sectors (Pope, Mays &amp; Popay, 2007).  17  \u000C3.1  Project planning  To ensure the review process was informed by expert opinion, two committees were established outside of the core research team: the first consisted of chief financial officers from health care authorities extending across Western Canada, which formed the Decision maker Advisory Committee (DMAC). The second, the External Reference Group (ERG), was comprised of international researchers with expertise in resource allocation and decisionmaking processes. For a complete description of the project team and committees, including roles and meeting timelines see Table 1.  Table 1: Description of project support teams  Title Disinvestment Group (DG)  External Reference Group (ERG)  Decision maker Advisory Committee (DMAC)  Membership  Role  Meeting Timeline  Seven-member team with expertise in health economics, health service research, priority setting and disinvestment; included a Chief Financial Officer (decision maker) from a local health authority. Four-member team representing researchers and experts from the National Institutes of Health (NIH), Harvard Medical School/University of Adelaide, the University of Birmingham, and City University of New York (CUNY). Committee comprised of Chief Financial Officers from Western Canadian health authorities, including British Columbia, Alberta, Saskatchewan and Manitoba.  Core research team responsible for research questions, search strategy and protocol, data extraction tool, analysis and synthesis.  All-team meetings held every second month; ad hoc meetings conducted when needed.  Support team responsible for assessing and critiquing search protocol, data extraction tool, analysis and results.  Feedback and discussion received and conducted via email; formal allteam meeting held once every six months via telephone conferencing.  Decision maker group responsible for critiquing the results and aiding in their practical application.  Two in-person meetings held: one to discuss initial results and one following project completion.  18  \u000COne primary search protocol was developed to apply across both reviews with modifications made based on key informant feedback; the research team agreed to the final search protocol. Search terms were reviewed and revised to reflect the sector under consideration with discussions held prior to initiating a database search. For example, while health care speaks to disinvestment, business engages in \u2018divestment\u2019 activities and the public sector manages \u2018budgetary cutbacks\u2019. However, each sector is addressing the same issue, namely a means of addressing budgetary shortfalls.  3.2  Search terms  Working in partnership with an experienced research librarian, two health care databases were searched. Prior to launching the primary search, project team members identified key articles from the health care literature that were thought to be seminal pieces in moving the disinvestment discussion forward (Table 2). From these papers keywords were extracted to guide the health care search. For the scoping review, two public sector databases and two business databases were searched; keywords were identified from an initial scan of the literature.  19  \u000CTable 2: Key health care articles identified to guide literature search Number  Author(s)  Title  Source, Year  1  Donaldson C., et al  Rational disinvestment  QJM, 2010  2  Elshaug, A.G., et al  Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices  Australia and New Zealand Health Policy, 2007  3  Elshaug, A.G., et al  Exploring policy-makers\u2019 perspectives on disinvestment from ineffective healthcare practices  International Journal of Technology Assessment in Health Care, 2008  4  Elshaug, A.G., et al  Identifying existing health care services that do not provide value for money  Medical Journal of Australia, 2009  5  Nuti, S., et al  Disinvestment for reallocation: a process to identify priorities in health care  Health Policy, 2010  6  Pearson, S., &amp; Littlejohns, P.  Reallocating resources: how should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service?  Journal of Health Services Research &amp; Policy, 2007  7  Robinson, S., Dickinson, H., Freeman, T., Williams, I.  Disinvestment after the new White Paper: the challenges facing GP commissioners  Public Money and Management, 2010  Before approaching each database, the project team reviewed and reassessed the protocol and sought input from members of the ERG who provided feedback and input on the search strategy. Such revisions were necessary because of the difficulty in identifying key descriptors that could be applied to the health care search strategy with similar issues arising when searching the public sector and business databases. The term \u201Cdisinvestment\u201D was not a subject heading and so could not be searched directly. In response, and to ensure an exhaustive search was undertaken, the type of search term applied was broadened to include more general concepts such as cost cutting, cutbacks, and retrenchment. Terms used in the  20  \u000Chealth care search included deinvestment, discontinuation, decommissioning, termination and priority setting. Terms used to search the public sector and business literature included cutback management, divestment, budget cutback, and policy termination.  The final search strategy applied to the health care literature was implemented in late fall 2011 (Appendix A). For both the systematic review and scoping review, duplicate references were removed prior to searching subsequent databases. From June 2011 to November 2012, a search of the health care literature was conducted in Medline and Embase databases. In order to capture papers from the business and public sector literature, Business Source Complete, ABI-Inform, PAIS and ERIC databases were searched between October 2011 and January 2012.  3.3  Article eligibility  Both reviews covered the period of 1970 to 2011; in the mid-1970s a widespread economic recession began that resulted in limited government funding and business slowdown. Articles that self-identified as having a disinvestment focus, with abstracts published in English, were eligible for inclusion. These included case studies that described a process, activity or the management of budgetary shortfalls. To identify grey literature from the health care sector, the core research team, in collaboration with members of the ERG, compiled a list of health care organizations, agencies and government bodies thought to be engaged in disinvestment activities. Corresponding websites for each organization were searched applying the same inclusion criteria as described below.  21  \u000CEligible documents were not restricted to case studies and included commentaries as well as opinion and descriptive pieces. Eligible articles had to self-identify as having a disinvestment focus, which included case studies that described a process, activity or the management of budgetary shortfalls.  From the outset, an ambiguity in the literature was identified in determining what actually constitutes a disinvestment activity. In response, a distinction was made between the adoption of \u2018cost saving\u2019 measures and \u2018disinvestment\u2019. For the purpose of this research, cost saving activities are those activities that are implemented in order to lower the overall expense of the intervention without affecting the quality or quantity of care. For example, this research considered re-engineering, outsourcing, and non-clinical and clinical integration as cost saving activities. Such approaches were not included in this review. In addition, when assessing the health care literature articles that described a resource allocation process that included opportunities for disinvestment, but were investment-focused, were identified, but set aside; only papers with a clear disinvestment objective were retained for analysis. Abstracts in languages other than English were also excluded. Titles and abstracts were retrieved, read and assessed based on one broad criterion:  Is the article about disinvestment processes, and/or describes disinvestment theories or conceptual approaches?  22  \u000C3.4  Assessment and analysis  Two reviewers [DS, LM] screened the records for eligibility by examining titles, abstracts and keywords. Records identified were retrieved and full texts were read. Data was extracted using an instrument consisting of 21 questions (Appendix B). The questions were descriptive and based on key issues identified in the literature. Their wording was refined by the research team in collaboration with the external reference group through an iterative process that included trial coding, team discussion and coding refinement.  To determine disinvestment definitions applied in the literatures, categories were drawn from the key articles previously identified (Table 2). Within the key papers, two distinct disinvestment definitions emerged and were used in the extraction process (Table 3), and a third category captured combined approaches as follows:  Category 1: Reallocation of resources from services that are providing positive benefit to areas where greater benefit may be gained. Category 2: Removal of interventions or services that are ineffective or provide little or no value for money spent. Category 3: Combines categories 1 and 2.  To distinguish between the two definitions Category one is referred to as \u201Crelative\u201D disinvestment, which describes the comparison of an intervention or service to an alternative in order to determine which has greater value. Category two, on the other hand, measures the  23  \u000Cinherent value of an individual service or intervention; this type of activity is referred to as an \u201Cabsolute\u201D disinvestment.  Table 3: Disinvestment definitions identified from key papers Table 2 Paper number 2, 3, 4, 5, 7  1, 3, 6  Definition The processes of (partially or completely) withdrawing health resources from any existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain for their cost and thus are not efficient health resource allocations. An explicit process of taking resources from one service in order to use them for other purposes that are believed to be of better value.  For each search, a tabulation and thematic analysis were conducted to inform a preliminary synthesis. Studies that met the inclusion criteria were recorded in a matrix. The quantitative values generated from the 21 questions were imported into SPSS version 17 with the results of the cross-tab analysis reported herein. Articles were grouped according to type with comparisons made in order to identify common themes (Pope et al, 2011). Further, across the two reviews definition use, approaches and concepts by sector were compared and from this assessment gaps in the health care literature were identified.  In order to determine patterns and recurring themes, a thematic analysis was conducted to assess the processes identified from the eligible articles (Braun, Clarke, 2006). Familiarization with the data generated the initial codes that were then put into groups to reflect common features considered pertinent to the synthesis. Repeated patterns were noted and recorded. Papers were read and reread to ensure all features were captured.  24  \u000CThe relevant features were organized into two tables to distinguish between attributes and components of a disinvestment process. Attributes were considered to be descriptors of a disinvestment process that included the type of assessment used (non-comparator or comparator), the model type (linear or cyclical), and the type of decision making framework applied (centralized or decentralized).  Components were identified and organized into  common phases that emerged through the analysis and consisted of linked steps a decision maker might take to determine and implement options for disinvestment.  25  \u000C4. RESULTS  The flow diagram for the 44 included peer-reviewed papers is shown in Fig. 1. Twenty-two grey literature reports from health care organizations were also identified. Table 4 lists those peer-reviewed articles found within the health care literature and Table 5 outlines those identified from the business and public sector literature. The 24 health care papers comprised commentary, opinion or perspective pieces (n=10), case studies (n=8), analysis (n=2), and review articles (n=1). The 20 papers identified from the scoping review of the business and public sector literature comprised commentary, opinion or perspective pieces (n=10), case studies (n=8) and qualitative studies (n=2).  Figure 1: Flow charts to show assessment of identified peer-reviewed papers  26  \u000CTable 4: Articles identified from a systematic review of the health care literature No.  Author(s)  Title  Source, Year  1  Caldwell, C., Butler, G., Poston, N.  Cost reduction in health systems: Lessons from an analysis of $200 million saved by top-performing organizations  Frontiers of Health Services Management, 2010  2  Donaldson, C., Bate, A., Mitton, C., Dionne, F., Ruta, D.  Rational disinvestment  QJM, 2010  3  Elshaug, A., Hiller, J.E., Moss, J.R.  Exploring policy-makers' perspectives on disinvestment from ineffective health care practices  International Journal of Technology Assessment in Health Care, 2008  4  Elshaug, A., Hiller, J.E.,Tunis, SR., Moss, JR.  Challenges in the Australian policy processes for disinvestment from existing, ineffective health care practices  Australia and New Zealand Health Policy, 2007  5  Elshaug, AG, Moss, JR, Littlejohns, P, Kamon, J., Merlin, TL, Hiller, JE.  Identifying existing health care services that do not provide value for money  Medical Journal of Australia, 2009  6  Garner, S, Littlejohns, P.  Disinvestment from low value clinical interventions: NICEly done?  British Medical Journal, 2011  7  Giacomini, M.  The which-hunt: Assembling health technologies for assessment and rationing  Journal of Health Politics, Policy and Law, 1999  8  Giacomini, M. Hurley, J., Stoddart, G.  The many meanings of deinsuring a health service: the case of in vitro fertilization in Ontario  Social Science &amp; Medicine, 2000  9  Goplerud, E.N., Walfish, S., Broskowski, A.  Weathering the cuts: a delphi survey on surviving cutbacks in community mental health  Community Mental Health Journal, 1985  10  Hughes, D. Ferner, R.  New drugs for old: disinvestment and NICE  BMJ, 2010  11  Ibargoyen-Roteta, N., Gutierrez-Iborluzea, I, Asua, J.  Guiding the process of health technology disinvestment  Health Policy, 2010  12  Karnon, J, Carlton, J, Czoski-Murray, C, Smith, K.  Informing disinvestment through costeffectiveness modeling  Applied Health Economics and Health Policy, 2009  27  \u000CNo.  Author(s)  Title  Source, year  13  Leatt, P., Baker, G.R., Halverson, P.K., Aird, C.  Downsizing, reengineering, and restructuring: Long-term implications for healthcare organizations  Frontiers of Health Services Management, 1997  14  Mitton, C., Dionne, F., Damji, R., Campbell, D., Bryan, S.  Difficult decisions in times of constraint: Criteria based resource allocation in the Vancouver Coastal Health Authority  BMC Health Services Research, 2011  15  Mortimer, D.  Reorienting programme budgeting and marginal analysis towards disinvestment  BMC Health Services Research, 2010  16  Nuti, S., Vainieri, M., Bonini, A.  Disinvestment for re-allocation: A process to identify priorities in health care  Health Policy, 2010  17  O\u2019Cathain, A., Musson, G., Munro, J.  Shifting services from secondary to primary care: stakeholders\u2019 views of the barriers  Journal of Health Services Research &amp; Policy, 1999  18  Panzer, R.J., Tuttle, D.N. Kolker, M.  1995 Fast track: cost reduction and improvement  Quality Management in Health Care, 1997  19  Pearson, S., Littlejohns, P.  Reallocating resources: how should the National Institute for Health and Clinical Excellence guide the disinvestment efforts in the National Health Service?  Journal of Health Services Research, 2007  20  Robinson, S., Dickinson, H., Freman, T., Williams, I.  Disinvestment in health \u2013 the challenges facing general practitioner (GP) commissioners  Public Money &amp; Management, 2011  21  Spallina, J.M.  Analysis weighs issues in divestiture decisions  Healthcare Financial Management, 1990  22  Stuart, G.W., Erkel, E.A., Shull, L.H.  Allocating resources in a data-driven college of nursing  Nursing Outlook, 2010  23  Wasserfallen, J.B.  Cost reduction project in a Swiss university hospital: Methods and results of a bottom-up intervention  Journal of d'Economie Medicale, 2002  24  Zuckerman, A.M.  To divest or not to divest? That is (sometimes) the question  Healthcare Financial Management, 2004  28  \u000CTable 5: Articles identified from a scoping review of the public sector/business literature No.  Author(s)  Title  Source, year  1  Algie, J., Mallen, G., Foster, W.  Financial cutback decisions by priority scaling  Journal of Management Studies, 1983  2  Bartle, John R.  Coping with Cutbacks: City response to aid cuts in New York State  State and Local Government Review, 1996  3  Behn, R.D.  Cutback Budgeting  Journal of Policy Analysis and Management, 1985  4  Crompton, J.L., Lamb, C.W.  Eliminating community services - The leisure services example  Community Development Journal, 1980  5  Diminnie, C.B., Kwak, N.K.  Journal of Operational Research Society, 1986  6  Dranikoff, L., Koller, T., Schneider, A.  A hierarchical goal-programming approach to reverse resource allocation in institutions of higher learning Divestiture: Strategy's missing link  7  Geva-May, I  When the motto is \"till death do us part\": The conceptualization and the craft of termination in the public policy cycle  International Journal of Public Administration, 2001  8  Jones, L.R.  Phases of recognition and management of financial crisis in public organizations  Canadian Public Administration, 1984  9  Kissler, G.R.  Who decides which budgets to cut?  Journal of Higher Education, 1997  11  Levine, C.H.  Organizational decline and cutback management  Public Administration Review, 1978  12  Levine, C.H.  More on cutback management: Hard questions for hard times  Public Management Forum, 1979  13  Michael, S. O.  Restructuring US higher education: Analyzing models for academic program review and discontinuation  The Review of Higher Education, 1998  14  Packard, T., Patti, R., Daly, D., Tucker-Tatlow, Farrell, C.  Administration in Social Work, 2008  15  Pearson, J.V., Michael, H., and Michael, R.J.  Cutback management strategies: Experiences in Nine County Human Services Agencies Zero-base Budgeting - a Technique for Planned Organizational Decline  16  Petry, J.  Bringing rigor to cutback management: Eugene\u2019s constrained prioritization process  Government Finance Review, 1994.  Harvard Business Review, 2002  Long Range Planning, 1981  29  \u000CNo.  Author(s)  Title  Source, year  17  Potter, D. Chickering, A. Scherrens, M.  Maintaining momentum and quality in a time of decline: a case study  Journal for Higher Education Management, 1992  18  Raiguel, F.  Pulling museum education purse strings  Journal of Museum Education, 2010  19  Simpson, W.A.  Retrenchment in British universities: lessons learned  The Canadian Journal of Higher Education, 1985  20  Uehling, B.  So money is a problem  Journal for Higher Education Management, 1992  21  Wallender III, HW  A planned approach to divestment  Columbia Journal of World Business, 1973  Of the articles identified from the health care databases, seven (33.3%) were from authors based in the United States, 23.8% from the United Kingdom and 14.3% from Canada. Other countries represented included Australia (9.5%), Italy, Switzerland and Spain. Just over seventy-two percent of business and public sector articles were identified as United States\u2019 based, with the remaining derived from the United Kingdom and Canada (13.6 and 9.1 respectively). Two articles were not given a specific country label as they did not refer to any particular region.  4.1  Disinvestment definitions  It is important to note that very few authors explicitly defined disinvestment with the exception of articles found in the health care literature. Both business and public sector researchers assumed a common knowledge and understanding of concepts when using terms such as divestiture, budget cutbacks, termination, retrenchment and planned reductions. Definitions were applied implicitly and included the identification and removal of areas or 30  \u000Cunits that produced little or no value or were considered to be of lower value compared to others.  The result of a sector comparison of disinvestment characteristics consisting of measurements, actions and goals is presented in Table 6.  Table 6: Characteristics of a disinvestment activity by sector Sector  Synonyms  \uF0B7 Health care \uF0B7 Decommission  Qualitative measure \uF0B7 \uF0B7 \uF0B7 \uF0B7 \uF0B7 \uF0B7  \uF0B7 Public sector \uF0B7 Cutback \uF0B7 and business management \uF0B7 \uF0B7 Policy or program\uF0B7 termination \uF0B7 \uF0B7 Deinvestment \uF0B7 \uF0B7 Divestiture \uF0B7 \uF0B7 Pruning \uF0B7  Inefficient Ineffective Inappropriate Unsafe Lower value Obsolete  Action \uF0B7 Scale back or reduce services \uF0B7 Stop or remove \uF0B7 services \uF0B7 Partial withdrawal of\uF0B7 services \uF0B7 Limit services  Equal weight \uF0B7 Non-essential \uF0B7 In decline or weak \uF0B7 Marginal \uF0B7 Low value \uF0B7 Out-dated Inefficient \uF0B7 \uF0B7 Client dissatisfaction \uF0B7 \uF0B7 Unprofitable \uF0B7 Strategically-misaligned  Cutback programs Eliminate programs Wait for programs to \u201Cdie\u201D Ad hoc deletions Sale of lower profit units  Goal Free up resources to use for purposes of greater value Meet unfulfilled needs Meet budgetary constraints Redistribution of resources Allocation of the decrement Deficit elimination Increase profits Improve shareholder value  Within the health care literature, the most frequently used definition was category two, or what the project team refers to as absolute disinvestment at 38.1% (n=8). In the public and business literature, fifteen papers or 68.2 percent referred to disinvestment activity as category three, that is, a combination of absolute and relative disinvestments.  31  \u000C4.1.1 Comparison of definition use and its application  When assessing definition use across sectors, a clear pattern emerged. In general, the business and the public sector often addressed budgetary shortfalls from a variety of approaches with a clear goal of terminating services or units that are producing little or no value while also reducing or removing services or units that are deemed to be less effective than other services or units, that is, areas of lower value (Dranikoff, Koller &amp; Schneider, 2002; Cragg, 2003). There was a distinction between these sectors and health care, with both business and the public sector research highlighting the need to assess all programs or units within the organization in order to determine where disinvestments should occur: what is most profitable for the company and/or how best to serve the public or client (Goplerud, Walfish, &amp; Broskowski, 1985).  While similar goals were found within health care organizations, disinvestments as described in the health care literature tended to be absolute in that they were addressed through single intervention assessments as observed in health technology assessment and the National Institutes of Clinical Effectiveness\u2019s (NICE) \u2018do not do\u2019 list (Garner &amp; Littlejohns, 2011).  Such assessments are conducted in order to determine which interventions and  services are no longer effective or are inappropriate without reference to a comparator. While removal of the ineffective intervention or service offers some monetary relief, these efforts will not necessarily address budgetary shortfalls or provide the needed resources to move a company or organization into the black (Dranikoff et al, 2002).  32  \u000CThe theme of regular \u201Cflagging\u201D of disinvestment opportunities was also evident in the business sector literature (Wallender III, 1973). Authors recognized that the majority of organizations and companies conduct disinvestment activities as a reaction to budget cuts or shortfalls rather than incorporating such practices into day-to-day operations.  Business  researchers suggest that programs, services or units be flagged for termination or reduction on a regular basis; a process similar to the identification of investment opportunities that occurs throughout the fiscal year.  However, inclusion of such flagging of disinvestment  opportunities could not be found in the health care literature, with authors noting that organizations do not regularly engage in disinvestment activities and instead pursue options only during periods of fiscal crises (Donaldson et al, 2010).  In the health care literature little attention was given to the difference between reactive disinvestment activity and activity conducted when organizations are fiscally sound. While there is support for the on-going assessment of old technologies to determine if they should continue to be used and funded, such as in the case of NICE in the UK, there was little evidence to support regular flagging of disinvestment opportunities. Similarly, there was limited discussion of the inclusion of practices that compare effective services with the goal of reducing those considered to be of lower value; exceptions were found in case studies that applied Programme Budgeting and Marginal Analysis (PBMA) to get at disinvestment (Mitton, Dionne, Damji, Campbell &amp; Bryan, 2011). Instead, the health care sector primarily focused on the identification and removal of technologies, services and interventions that are ineffective or inappropriate, that is, areas that have little or no value.  33  \u000C4.1.2 Definition by time period  Due to the small number of peer reviewed articles in the two reviews, comparisons of definitions were made based on four separate time periods with categories collapsed into decades: &lt;1970, 1980-1989, 1990-1999, 2000-2011. The period that saw the greatest number of disinvestment-focused articles within the health care literature was from 2000 to 2011 (71.4%), followed by the 1990s at 23.8 percent. When comparing the results from the two reviews, periods of high activity varied with health care only recently describing the concept of disinvestment with the majority of articles published from 1990 onwards. Conversely, business and public sector produced the greatest number of relevant articles in the 1980s at the height of the economic recession.  When comparing definition type by decade, only one category produced enough numbers per cell per search to allow for a more complete analysis. In the health care literature, absolute disinvestment revealed an increasing trend from no articles produced in the 1980s to 37.5% in the 1990s and 62.5% in the 2000s. On the other hand, from the business and public sector search, the combination approach to defining disinvestment was used the most in the 1980s at 33.3% before declining in the 1990s to 26.6% and to 20% in the 2000s; definition category one and two increased slightly over time.  4.1.3 Definition by region  Since country comparisons by definition type resulted in a small number of cases per cell, for this particular crosstab analysis the country categories were collapsed into three areas:  34  \u000CNorth America, Europe and Australia. Within the health care literature, articles that identified as being based in North America defined disinvestment primarily as category two (40.0%), followed by category one (30.0%). The majority of Europe-based health care articles defined disinvestment as the identification and reduction of services based on effectiveness or value, that is, category one (37.5%). Two articles were identified as being based in Australia with both recognizing disinvestment as the removal of ineffective or inappropriate services that provide little or no value (category two).  From the results of the scoping review, only two regions were represented: Europe (13.6%) and North America (81.8%) and one article that could not be labeled. Within this sample, both regions referred to absolute disinvestment, with 66.7% of European papers and 80.0% of North American papers applying a combination of approaches.  4.2  Disinvestment processes  Results from the two searches identified 19 peer-reviewed papers, with nine from the health care literature, and ten from business and public sector. In total, seventeen distinct processes were identified, seven from the health care literature and ten from the business and public sector (Table 7).  Programme Budgeting and Marginal Analysis (PBMA) was  described in three papers, with one article presenting the results of a successful case study. Process names and descriptions can be found in Table 7.  35  \u000CTable 7: Processes identified from the health care and public sector/business literatures Sector Health care  Process # 1  Process label Programme Budgeting and Marginal Analysis (PBMA)  Source(s) Donaldson, C., Bate, A., Mitton, C., Dionne, F., Ruta, D. Rational disinvestment  Description Eight step process that incorporates criteria development and proposal generation.  Mitton et al. Difficult decisions in times of constraint: Criteria based Resource Allocation in the Vancouver Coastal Health Authority Mortimer, D. Reorienting programme budgeting and marginal analysis (PBMA) towards disinvestment 2  Not provided  Elshaug, A.G., Moss, J.R., Littlejohns, P., Karnon, J., Merlin, T.L., Hiller, J. Identifying existing health care services that do not provide value for money  Provides two sets of criteria to identify candidates for disinvestment and to rank candidates for implementation.  3  Not provided  Giacomini, M. Hurley, J., Stoddart, G. The many meanings of deinsuring a health service: the case of in vitro fertilization in Ontario  A panel-based decision making process for the deinsuring of services from the Ontario Health Insurance Program (OHIP).  4  Delphi decision making approach  Goplerud, E.N., Walfish, S., Broskowski, A. Weathering the cuts: a delphi survey on surviving cutbacks in community mental health  The application of a Delphi approach to decision making using a series of structured questionnaires.  5  History model and costeffectiveness  Karnon, J., Carlton, J., Czoski-Murray, C., Smith, K. Informing disinvestment through cost-effectiveness modeling: is lack of data a surmountable barrier?  Applied a history model coupled with a cost-effectiveness analysis in order to decide between four service provision options.  6  Not provided.  Stuart, G.W., Erkel, E.A., Shull, L.H. Allocating resources in a data-driven college of nursing  Decentralized approach to determine most cost-effective means of funding a new graduate program.  7  Participative approach  Wasserfallen, J.B. Cost reduction project in a Swiss university hospital: Methods and results of a bottomup intervention  Incorporates a proposal generation process to determine disinvestment options which are then assessed and ranked.  36  \u000CSector  Process #  Process label  Source(s)  Description  Public/ business  8  Multivariate scaling  Algie, J., Mallen, G., Foster, W. Financial cutback decisions by priority scaling  Determines concordance among decision makers applying implicit criteria to assess and weigh options for disinvestment  9  Life cycle audit; index of efficiency; direct responder input  Crompton, J.L., Lamb, C.W. Eliminating community services - The leisure services example  Outlines three methods to determine programs to be terminated: life cycle audit; an index of efficiency measure; direct responder input via surveys.  10  Hierarchical goalprogramming approach  Diminnie, C.B., Kwak, N.K. A hierarchical goalprogramming approach to reverse resource allocation in institutions of higher learning  Goal-programming model in which the 0-1 decision variables correspond to the budget cut alternatives; options are proposed by each level of the institution.  11  PROACTIVE approach  Dranikoff, L., Koller, T., Schneider, A. Divestiture: Strategy's missing link  Incorporates criteria in order to Identify candidates for disinvestment.  12  Quality model, cost model, market model, employment model, political model and academic model  Michael, S. O. Restructuring US higher education: Analyzing models for academic program review and discontinuation  Models that can be applied within educational institutions with the majority based on market analysis.  13  Not provided.  Packard, T., Patti, R., Daly, D., Tucker-Tatlow, Farrell, C. Cutback management strategies: Experiences in nine county human services agencies  Several strategies are described that includes key components of a successful approach.  14  Zero-based budgeting  Pearson, J.V., Michael, H., and Michael, R.J. Zero-base Budgeting - a Technique for Planned Organizational Decline  Four step process that includes formation of decision packages, assessment and ranking of options.  15  Not provided  Petry, J. Bringing rigor to cutback management: Eugene's constrained prioritization process  Three distinct scenarios are described that move from a public model to a centralized approach to determine funding priorities and disinvestment options.  37  \u000CSector  Process #  Process label  Source(s)  Description  Public/ business  16  Not provided.  Uehling, B. So money is a problem  Four step process that incorporates criteria development, determining the scope of the project and includes the chance for rebuttal once decisions have been made.  17  FAST process  Wallender III, HW A planned approach to divestment  A four step process with few details provided.  38  \u000CThematic analysis of the health care literature identified four phases of a disinvestment framework and common components that comprise each (Figure 2).  Attributes of a  disinvestment activity were identified in all 17 processes and vote counting was used to indicate their frequency (Table 8). Table 8 reports common components contained within each phase and frequency counts for each by sector.  Figure 2: Phases and components of a disinvestment process  39  \u000CFrom the systematic review of the health care literature, identified articles that described a disinvestment process represented regions that included Canada (n=2), the United States (n=2), the United Kingdom (n=2), Australia (n=2) and Switzerland (n=1). Processes identified from the scoping review were from the United States (n=8) and the United Kingdom (n=2).  Table 8: Attributes of a disinvestment process Health care (process #)  Public/ business (process #)  Frequency of reporting (total # of processes (%))  Assessment of single or multiple programs, services or units (noncomparator)  2,3  9,11,13,15  6 (33.33)  Assessment of multiple programs, services or unit (comparator, i.e., relative value)  1,5  8,10,14  5 (27.78)  Objectives, goals and scope defined  1,7  10,14,15  5 (27.78)  Linear process  2,3,4,5,6,7  8,9,10,11,12,13,14,15,16 ,17  17 (94.44)  Cyclical process  1  Attributes  Decentralized decision making  1 (5.0)  1,3,7  13  4 (22.22)  Centralized decision making  2,4  10,11,14,15,17  7 (38.89)  Trialed in real-world setting  1,3,6,7  8,10,13,15  9 (50.00)  1,3  10,15  5 (27.28)  1,2,3,6  10,13,16  7 (38.89)  1,2,3,4,7  12,  7 (38.89)  Defined roles and responsibilities Key stakeholder involvement Evidence-based decision making Participant training Incentives provided  1,7  3 (16.67) 11  1 (5.00)  40  \u000C4.2.1 Process attributes  Key attributes of a disinvestment process included type of assessment, model type and the decision making framework applied. From the analysis two main types of disinvestment assessments were found: the first is what is referred to in this thesis as a \u2018non-comparator assessment\u2019, and the second, a \u2018comparator assessment\u2019. Across all three sectors, the most common assessment type was a non-comparator assessment wherein options were measured based on the innate value of a program, service or unit. Innate value seeks to determine the essential worth of a program, service or unit without a comparator in place against which its value is assessed. Relative value, on the other hand, attempts to evaluate the worth of a service, program or unit compared to an alternative, which has been labeled comparator assessment.  Within the seven distinct processes identified from the health care literature, only four processes described the type of assessment applied: two outlined a non-comparator and two a comparator. From the health care literature, Karnon, Carlton, Czoski-Murray and Smith (2009), who reported the assessment of a disinvestment case study \u2013 screening for amblyopia and strabismus \u2013 outlined four distinct scenarios from which a choice is made between two possible candidates, each serving a different population. Contained within each of the two candidates is an alternative provision option: both requiring fewer funds to implement and from which decision makers must choose. This example exemplifies the complexity of a comparative assessment yet the importance of its inclusion. Donaldson et al (2010) illustrated this type of assessment through the description of two distinct procedures \u2013 one for hips and one for hearts \u2013 that are measured and compared using marginal analysis. In both examples, 41  \u000Cthe decision maker must take into consideration the opportunity cost involved in each decision made, that is, to devise what is lost in terms of outcomes when funds are taken from one service to fund another.  From the business and public sector literature, seven processes described the type of assessment used with four applying a non-comparator assessment (Table 7). The process that Crompton and Lamb (1980) outlined to address funding cutbacks within leisure services provided an example of non-comparator assessments conducted within a large service portfolio. The authors suggested that an individual program\u2019s value be measured using evaluative tools such as the product life cycle wherein programs are mapped to five distinct phases of development that move from introduction, to take-off, maturation, saturation and finally to decline.  It is recommended that programs in the stage of decline should be  considered for removal without measuring their value relative to another. Such an approach does not consider whether a program\u2019s current stage is related to the amount of dollars assigned or whether additional funds would move it out of a state of decline.  The key attribute of model type identified could be described as either a cyclical model or one that is linear. A linear model has a defined start and end point that moves the decision maker through the components in a stepwise progression. The linear process tends to support more \u2018one-off\u2019 objectives, that is, a budgetary shortfall or external funding cuts become the catalyst for applying a disinvestment process (Giacomini, Hurley &amp; Stoddart, 2000). This is not to say that a linear process can only be applied during crisis or to a single decision event; rather, the defined end point tends to deter the occurrence of any further iterations or applications of the process. The cyclical model, on the other hand, while it also employs a 42  \u000Cstepwise progression and has a clear starting point, the final component leads the decision maker back to the beginning of the process in an on-going cycle of candidate identification, selection and decision making. As described and applied in Mitton et al\u2019s (2011) case study, Programme Budgeting and Marginal Analysis (PBMA) implemented within a health care context is the only example of a cyclical model identified from the two reviews wherein process evaluation is conducted in order for improvements to be made and applied to the next fiscal cycle.  Another key attribute identified from the thematic analysis was the type of decision making framework applied; it can also be described as the level at which the disinvestment decision making occurs. This attribute sets the scope of any decision making model and either reflects the organizational structure that is in place, oftentimes one that is hierarchical in nature with decision making power resting at the top (centralized), or attempts to break the structure by shifting the power to employees and staff who otherwise have little input into the decision making process (decentralized). Within the health care literature, the type of decision making applied could be identified in five of the seven process models with a decentralized approach being the most common (n=4).  Decentralization took different forms and occurred at varying degrees with Giacomini et al (2000) describing the inclusion of two public members on a decision making panel, while Stuart, Erkel and Shull (2010) outlined a process wherein all faculty members of a nursing college were involved in deciding how to release resources in order to initiate a new graduate program. In the latter scenario, vote counting was used to reach decisions, whereas in the previous example decisions were made by consensus. 43  \u000CIn contrast, from the processes that identified a decision making framework, the majority of the public sector and business models applied a centralized decision making approach (n=4). Petry (2004) reported the movement from a decentralized to centralized model in order to address budgetary shortfalls for the city of Eugene, Oregon.  In his  description, Petry (2004) outlined three distinct attempts at managing funding cutbacks with the first permitting the public to decide how best to allocate city resources. When further funding deficits were announced, a combined approach was adopted that involved both the public and key policy makers. The last attempt at balancing the budget applied a \u2018constrained prioritization\u2019 wherein decision making power was placed solely in the hands of policy makers. Petry (2004) noted that the constrained prioritization approach was successful in that it created a defined structure that enabled the streamlining of services and programs in order to meet the budgetary shortfall.  Diminnie and Kwak\u2019s (1986) example of a hierarchical goal programming approach applied within a private university attempted to break with the centralized approach by enabling disinvestment options to be put forward at each structural level. While the vice president\u2019s office made the final decision as to which disinvestment options were implemented, participating chairmen reported that the model enabled each department to have a defined role in the budgetary process and minimized the tendency for politically driven decisions.  44  \u000C4.2.2 Phases and components  Thematic analysis of the seven processes described in the health care literature and ten from the public sector and business literature also identified common components of a disinvestment process (Table 9). From descriptions of these common components and placement within the individual processes identified from the two reviews, components were able to be categorized into four distinct phases of a disinvestment activity by sector type: identification, selection, implementation and evaluation. In most cases the disinvestment process began after the problem or opportunity had been identified, its scope determined, and agreement to proceed arrived at.  Phase 1: Identification  The identification phase of a disinvestment process contains the necessary components to assess and determine candidates from a defined set of programs, services, interventions or business units; in some cases, all budget items are considered to be viable options (Algie, Mallen, &amp; Foster, 1983). In cases where options need to be drawn from a larger pool, disinvestment candidates are arrived at through various methods of analysis and assessment. From the health care literature, six of the seven studies described such a selection process, with the seventh noting that one intervention would be used to assess the decision making model presented (Karnon et al, 2009).  45  \u000CTable 9: Phases and key components of a disinvestment process  Phase Identification  Selection  Implementation  Evaluation  Components Alignment to principles, goals or mandate Evaluative devices (economic measures, surveys, evidencebased analysis) Adverse effect analysis Feasibility assessment System impact assessment Opportunity cost considered Proposal generation Stakeholder input List-order programs, services or units No formal assessment Explicit criteria Implicit criteria Formal process to prioritize options (weighting, scoring/rating, ranking) Type of decision making Vote count Consensus Total score Plan and timeline established Roles and responsibilities defined Communicate decisions to stakeholders Process evaluation Outcome of decisions  Health care (model #)  Public/ business (model #)  1,3,6  10,11,13,14,16  Frequency of reporting (total # of models (%)) 8 (44.44)  1,2,3,5,6,7  9,11,14,15  10 (55.56)  2,6,7 7 2  10,14 11,13  5 (27.78) 1 (5.00) 3 (15.67)  1,2,6  8  4 (22.22)  1,3,4,7 1 7  10 9 8,15  5 (27.78) 2 (11.11) 3 (16.67)  1,3  8,10,11,13,16  8 (44.44)  1,2,4,7  8,10,13,15  8 (44.44)  16  1 (5.00) 4 (22.22) 2 (11.11)  6 1,4,6 1,4 7  2 (11.11) 15  2 (11.11)  1  9,11  3 (16.67)  1,7 1,6,7  10  3 (16.67) 3 (16.67)  46  \u000CEight processes from the business and public sector literature and all of the health care processes included an identification procedure that applied varying types of evaluative devices (Table 8). These devices varied between the two reviews and included alignment to an organization\u2019s principles, goals or mandate (Pearson, Michael &amp; Michael, 1981; Dimminie &amp; Kwak, 1986; Uehling, 1992; Packard, Patti, Daly, Tucker-Tatlow &amp; Farrell, 2008; Dranikoff et al, 2009), an adverse effect analysis (Pearson et al, 1981; Diminnie &amp; Kwak, 1986;) or the application of economic measures such as marginal analysis (Donaldson et al, 2010; Mortimer, 2010; Mitton et al, 2011) cost-effectiveness (Karnon et al, 2009) and market analysis (Michael, 1998; Dranikoff et al, 2002; Stuart et al, 2010). Determining each candidate\u2019s overall system impact (Dranikoff et al, 2002; Packard, 2008; Elshaug et al, 2009) and conducting a feasibility assessment (Wasserfallen, 2002) were some additional tools used to identify potential candidates.  As a starting point to determine disinvestment options within social service organizations, Algie et al (1983) described the rank ordering of all programs based on priority: programs deemed to have a lower rank were then assessed to identify potential disinvestment options within that service or program. From the health care literature, three models reported the inclusion of a proposal generation process wherein disinvestment options were put forward via a formal mechanism, such as a questionnaire or business case proposal (Goplerud, Walfish &amp; Broskowski, 1985; Wasserfallen, 2002; Mitton et al, 2011). Goplerud et al (1985) described several rounds of questionnaire submissions with the first generating a list of potential actions that local community mental health agencies could take to meet funding cutbacks. Similarly, Mitton et al (2011) described a proposal generation process whereby  47  \u000Cmanagers submitted business cases and created a working list of options from which selection could occur.  Phase 2: Selection  The selection phase involves a screening process wherein identified candidates are assessed and selected for implementation from the larger pool of options. Assessment can take several forms with the majority of processes describing the inclusion of criteria against which the candidates are measured. From the health care literature, PBMA has a clearly defined and distinct criteria development component wherein criteria are mapped to strategic priorities, weighted and determined through several iterations and in consultation with process participants (Mitton et al, 2011). This PBMA case study is an example of a more participatory model of decision making that ensures consensus is reached at each stage of criteria development. Elshaug et al (2009) also suggested the inclusion of criteria at every step of the process with a set defined then applied to assist in prioritizing disinvestment candidates.  Processes identified from the public sector literature also noted the inclusion of criteria for assessment purposes. While Diminnie et al\u2019s (1986) process did not involve criteria development per se; it did suggest that options be selected to reflect and achieve each department\u2019s goals and strategic objectives. Algie et al\u2019s (1983) description of priority scaling also suggested that decisions be made based on a set of criteria; however, unlike the previous examples, the criteria applied to assess options are implicit and defined by each individual involved in the decision making process. However, because the purpose of priority scaling is to measure if decision makers are setting priorities in a similar fashion, their results revealed 48  \u000Cthat while criteria are implicit, candidates ranked by individuals involved in the process tended to have a high degree of concordance. While this high degree of concordance implies that criteria need not be explicitly developed and defined, it is important to note that the type of decision making employed was a centralized approach wherein decision makers included in the analysis were executive directors of social service organizations.  Perhaps in a  decentralized model that includes a mix of decision making levels such homogeneity across criteria is less probable.  Candidate selection also involves a variety of formal processes to prioritize options once they have been identified.  Examples from the health care literature include the  application of a rating tool that enabled decision makers to apply a score based on a form of scaling as described in Goplerud et al\u2019s (1985) case study. Here, a four-point scale was introduced based on four discrete traits: importance, desirability, feasibility, and validity. The average score was determined and options ranked across all four traits. PBMA, as described by Mitton et al (2011), also incorporated a proposal rating tool that enabled decision makers to prioritize options based on how well each proposal mapped to a set of predetermined weighted criteria.  Another feature of candidate selection is the way in which decisions are made. While many processes identified from the two reviews incorporated tools and clearly defined methods for determining which candidates should move into the implementation phase, often the final list was arrived at through deliberative discussions with decision makers, and in some cases, key stakeholders. Vote counting (Stuart et al, 2010) and arrival at consensus (Goplerud et al, 1985) were the two forms identified outside of decisions made based on total score 49  \u000C(Mitton et al, 2011). Among these methods several processes contained multiple decision making points that incorporated more than one type. PBMA, for example, while it uses a total score to determine the candidates for disinvestment, consensus around the criteria \u2013 their inclusion, definitions and weighting \u2013 must be reached (Mitton et al, 2011).  Phase 3: Implementation  The implementation phase, wherein decisions are acted upon was the least described phase of the disinvestment processes identified from the two reviews.  Across the two  reviews, components included ensuring implementation roles and responsibilities were clearly defined (Petry, 2004), that an action plan and timeline were established (Wasserfallen, 2002), and a communication plan developed to ensure key stakeholders were informed of the decisions made (Crompton &amp; Lamb, 1980; Dranikoff et al, 2002; Mitton et al, 2011).  Wasserfallen et al (2002) noted that the three-year timeline established at the beginning of the process allowed for proposal negotiation even after candidates had been selected for implementation; however, details of how negotiations were initiated and conducted were missing from their report. Such an inclusion of an appeals process wherein stakeholders are given the opportunity to request the retraction of decisions was briefly noted in Uehling\u2019s (1992) process description wherein she championed the opportunity for rebuttal.  50  \u000CPhase 4: Evaluation  While the evaluation phase was not described in the majority of the processes identified, three papers from the health care literature and one from the public sector and business literature described a phase wherein two key components \u2013 the disinvestment process and decision outcomes \u2013 were evaluated and measured.  From the health care literature, Wasserfallen et al (2002) reported on a postimplementation evaluation that was conducted among the 175 participants to assess satisfaction with the process, to measure compliance with the decisions made and determine the confidence rate that implementation would occur. This study also described the end result of the disinvestment process noting that over 60 percent of the proposals were implemented and a cost estimate for the process was conducted and reported. Similarly, Mitton et al\u2019s (2011) case study of PBMA also included a post-implementation evaluation that assessed the decision making process in order to identify points of improvement and to determine whether the process could be applied to other areas within the health authority. Overall, they reported that respondents viewed the process as being more robust and an improvement over previous resource allocation processes that were often derived from historical practices or were politically driven.  The movement away from political drivers was also noted in Diminnie et al\u2019s (1986) paper wherein an evaluation of the model revealed that the process linked funding allocation to the university\u2019s goals rather than to a department chair regardless of their personal influence. Such movement away from ad-hoc processes and towards a more robust and 51  \u000Ctransparent process increases stakeholder buy-in and ensures a more fair and equitable approach to decision making.  4.3  Challenges to disinvestment  Challenges identified from the health care literature can be categorized into three central themes: system complexity, lack of resources, and resistance. The complexity of health care systems often deters disinvestment activities since the removal or reduction of a service can have a detrimental impact on the rest of the organization: clients may be shifted to other areas that will then require the expansion of remaining programs or services (Mortimer, 2010; Robinson et al, 2011). Such a shift could counteract any financial gains achieved from the disinvestment activity and increase the overall system costs. Shifting costs was also recognized as a barrier in Giacomini et al\u2019s (2000) description of a delisting process wherein the removal of a previously insured item relocates the service cost from the system to the patient, forcing the patient to seek care from private providers. Such a fiscal relocation can be viewed as a threat to a public structure by creating the need for a two-tiered service delivery system. The issue of system complexity is also apparent when attempting to remove established services whose infrastructure required a substantial amount of time, funds and expertise to develop (Elshaug, Hiller &amp; Moss, 2008).  Other barriers described include the dearth of resources, such as expertise (Pearson &amp; Littlejohns, 2007; Mortimer, 2010), lack of data (Elshaug et al, 2007), and the time required to conduct a structured disinvestment process (Giacomini et al, 2000; Donaldson et al, 2010). While expertise on resource allocation procedures may be lacking among health care 52  \u000Cproviders, Mitton et al (2011) found that by offering substantive training on the process itself, including the assessment of disinvestment options, committee members felt more equipped to participate in the disinvestment activity negating the need for so-called expertise. Such a focus on training also promotes capacity building within the organization and may help facilitate stakeholder buy-in once decisions are made.  Elshaug et al (2007) noted that one of the challenges faced by the Australian health care system was the lack of dedicated resources to develop disinvestment policy mechanisms for obsolete technologies or practices. They suggested that a political paradigm shift may be required to support and drive a more disinvestment-focused agenda. Some authors also suggested that the lack of evidence around ineffective practices or services were a major challenge to deciding where disinvestments can be made (Elshaug et al, 2007; Pearson &amp; Littlejohns, 2007; Garner &amp; Littlejohns, 2011). Pearson and Littlejohns (2007) stated that even when evidence is available it is difficult to assess and apply it, and to know which studies should be included in the decision making process. While a lack of evidence may hinder many processes, Giacomini et al 2000) found that evidence-based decisions, although considered to be objective in nature, were often difficult to make due to the differing stakeholder opinions regarding which type of evidence was considered more valuable]. Similarly, Caldwell, Butler &amp; Poston (2010) noted when assessing top-performing organizations and what they refer to as \u201Cnon-starters\u201D, often organizations became caught-up in the need to have \u2018perfect data\u2019 before moving forward.  Several authors described disinvestment as being controversial with strong resistance to its inclusion in resource allocation decisions (Elshaug et al, 2007; Elshaug et al, 2008; 53  \u000CMortimer, 2010; Robinson et al, 2011). Such resistance is due to its very nature wherein funding reductions signal an environment of less rather than more, an environment that is not readily embraced by health providers or the public alike. Making do with less, while necessary, does not win votes nor does it assure a government\u2019s popularity. Political and public resistance and backlash are often at the forefront of resource allocation decisions with the media playing a large role in which decisions are reported. Garner and Littlejohns (2011) suggested that support from practitioners and political leaders is necessary to reorient the current investment agenda to one that is more disinvestment focused.  In the assessment of the public sector and business literature, similar challenges were described, in particular around system complexity (Levine, 1978), lack of funding to support disinvestment, and resistance (Pearson, Michael &amp; Michael, 1981; Geva-May, 2001). Levine (1978) argued that in the public sector there appears to be a disincentive to underspend resources; instead, the public appears to demand and support government overspending. Public sector research also described psychological barriers that often are in play for the decision makers and even more so for those individuals directly affected by its decisions (Behn, 1985; Bartle, 1996). Behn (1985) suggested that people think differently about losses than they do about gains, and further, that individuals tend to focus on both the losses and the gains rather than the overall outcome. When applying this thought-process to health care, the public and providers tend to focus on the individual decisions made \u2013 on who gained and who lost \u2013 rather than the overall sustainability of the system.  Behn (1985) also described disinvestment as being \u201Cprecedent-breaking\u201D in that it is often only considered during fiscal crisis and is not part of conducting day-to-day business. 54  \u000CFurther, he suggested that incremental budgeting, which promotes investments, tends to occur more regularly and is rewarding for those involved in decision making; whereas decremental budgeting or the cutting back of funding requiring disinvestment is unpredictable, painful and generates mistrust among staff.  4.4  Facilitators to disinvestment  Some of the facilitators identified from the health care literature included the need for a defined and transparent process (Elshaug, 2008; Stuart et al, 2010), strong leadership (Mitton et al, 2011) and the involvement of stakeholders in the decision making process (Elshaug et al, 2008; Mortimer, 2010; Stuart et al, 2010; Mitton et al, 2011). Mortimer (2010) suggested that stakeholder involvement can be accomplished through the use of citizen juries, as one example. In Giacommi et al\u2019s (2000) disinvestment process, such involvement was conducted through the inclusion of two public representatives on the decision making committee.  Inclusion of the public can help to build stronger relationships between  stakeholder groups and promote disinvestment as a more common, and potentially more accepted, practice (Robinson et al, 2011).  The need for consensus and trust was also described as a facilitator to Mitton et al\u2019s (2011) application of PBMA. In their example, buy-in was achieved at each step in the process which enabled consensus to be reached with trust developed and fostered through the collaboration of managers and clinicians. Based on Cameron, Freeman and Mishra\u2019s work, Leatt et al (Leatt, Baker, Halverson &amp; Aird, 1997) described the need to decentralize the decision making process while at the same time ensuring that essential features are mandated 55  \u000Cfrom high-level decision makers. They suggested a combined approach with input also generated from lower staffing levels.  Similar approaches were supported from the public and business literature wherein Behn (1985) spoke to reorienting disinvestment towards an agenda that overrides the need to cut spending; that is create a goal that the majority of stakeholders can buy into and thus reframe the process\u2019s purpose. Echoing facilitators identified from the health care literature, several authors also called for strong leadership (Behn, 1985; Packard et al, 2008; Caldwell et al, 2010) and the inclusion of stakeholders in the decision making process (Packard, 2008). Behn (1985) also noted the importance of considering all programs as potential disinvestment candidates to ensure everyone \u2018shares the pain\u2019 and that no single program is targeted. This idea of pain-sharing promotes collaboration and allows programs to understand how their disinvestment option will work towards reducing the deficit.  4.5  Framework design  In developing the proposed framework described herein, its attributes and components as identified from the literature reviews and reported above are considered.  The key  attributes, components and facilitators that were taken forward in framework development are listed in Table 10.  56  \u000CTable 10: Key features identified from the literature and used to develop a framework Attributes  Components  \uF0B7 Comparator assessment \uF0B7 Cyclical model \uF0B7 Centralized decision making with some involvement of staff and the public  \uF0B7 Alignment to organizational mandate \uF0B7 Evaluative devices \uF0B7 Explicit criteria \uF0B7 Formal process to prioritize options \uF0B7 Effective communication \uF0B7 Process evaluation using established framework \uF0B7 Outcome assessment  Facilitators included \uF0B7 Distinct process \uF0B7 Stakeholder involvement \uF0B7 Collaboration across disciplines \uF0B7 Strong leadership  4.5.1 Defining the key attributes  Key attributes of a decision making process include type of assessment, model type and the decision making framework applied. From the health care literature, a comparatortype of assessment was adopted wherein the relative value of a service, program or intervention is measured against an alternative. This type of assessment is in contrast to a noncomparator assessment that considers the innate value of a service, program or intervention without a comparator in place. Approaching assessment through a relative value measure ensures that when decisions are being made, the decision maker considers what is lost from one area in order to gain in another, that is, the opportunity cost of such a decision.  While the majority of published processes adopted a linear model in that components were linked together with a clear starting point and end point (Giacomini et al, 2000; Wasserfallen, 2002; Elshaug et al, 2009; Stuart et al, 2010), a cyclical model was deemed to be more appropriate and may help to facilitate the inclusion of disinvestments into the annual resource allocation process. A cyclical model applies linked components that take a decision 57  \u000Cmaker from a defined starting point through established tasks with the final step linking back to the beginning of the process in an ongoing cycle of disinvestment identification and implementation. The framework developed incorporates the necessary evaluative actions required to identify process improvements and to assess process outcomes in order to make adjustments to the next fiscal cycle.  The type of decision making framework also varied between the health care literature and the public sector and business literatures. In health care, a decentralized approach was applied more often with options for disinvestment generated at the program or management level (Behn, 1985; Mitton et al, 2011). While these processes are considered decentralized, in that they include lower staffing levels, only Stuart et al\u2019s (2010) process facilitated a forum for all faculty to determine which programs should be disinvested with final decisions voted on and consensus reached. In this example, the evaluative measures used were program costs and revenue generated; no qualitative factors were considered. In a health service delivery organization, where cost-effectiveness is one of many measures used to assess a program\u2019s value, more subjective principles such as equity and client experience require greater consensus-building and collaboration between decision makers.  In contrast, the public sector and business research supported the inclusion of a topdown decision making process or what is often termed a centralized approach. Petry (2004) noted that a top-down model produced better results than a decentralized model, which was used previously to address budget restrictions for the city of Eugene, Oregon (Petry, 2004). The centralized model provided a defined structure which enabled the streamlining of services and programs that may not have been achieved otherwise. Diminnie et al\u2019s (1986) example of 58  \u000Ca hierarchical goal programming approach attempted to break with the centralized approach by enabling disinvestment options to be put forward at each structural level. However, the vice president\u2019s office made the final decision as to which disinvestment options were implemented, similar to the identification and decision making structure of the PBMA framework (Mitton et al, 2011).  Drawing from the public sector and business approach to decision making, a top-down process was adopted for the purposed framework and supported by members of the decision makers advisory committee. Here, potential candidates for disinvestment are brought forward by the central decision making body comprised of members of the executive team, management and clinical leaders. Once options are identified, a second committee, comprised of management, clinicians and members of the public, develop criteria and selects the options to be implemented based on the agreed-upon criteria. Criteria development and selection procedures are described in greater detail below.  4.6  Development of the disinvestment framework  Based on the findings from the systematic review of the health care literature and supplemented by the results of a scoping review of the public sector and business literature, a framework comprising four phases, seven key components and model attributes was constructed (Figure 3). These seven components represent distinct steps in a decision making process. Linked together in chronological order, assigning each a box, these steps provide a detailed guide to a disinvestment activity. Each component is described in more detail, including roles and responsibilities and a proposed timeframe in Table 11. Members of the 59  \u000Cexternal advisory committees critiqued the framework and provided feedback. Suggestions to improve clarity and simplification of the process were received and applied. It is important to note that several of the process components as described below were drawn from PBMA, which was the most well-defined and sucessfully trialed tool identified from the literature. However, the core structure of the proposed disinvestment framework differs in how decisions are made and by whom. The inclusion of two distinct committees, each with defined roles and responsibilities, ensures that the identification of disinvestment options and their subsequent assessment for selection are undertaken in isolation to minimize any potential biases and to promote greater staekholder buy-in.  How deicions are made is determined by the type of decision making model applied. The proposed framework is based on a centralized model rather than a decentralized one as described in PBMA. Rationale for promoting a centralized approach is adopted from the scoping and business literatures which describe the challenges often faced when attempting to disinvest from existing services and programs. Tough decisions are more readily made by those individuals in the organization who routinely make them, and subsequently, are able to enforce them. This is not to say that a more decentralized approach cannot be undertaken once buy-in is obtained and capacity-built; rather, that an organization at first may find greater success with the inclusion of a centralized approach.  60  \u000C4.6.1 Process participants  The process requires the establishment of two central committees both of which have well-defined roles and responsibilities. The Oversight Committee (OC) is comprised of key decision makers in the organization, including members of the Executive Team, representatives from management and clinical leaders.  The Oversight Committee is  responsible for completing the majority of the process components including making final decisions regarding which disinvestment options to implement and how implementation will proceed.  Figure 3: Disinvestment framework  61  \u000CTo minimize bias when selecting options for disinvestment, a second committee is formed. The Assessment Committee is responsible for determining and defining the criteria, weights and scale used to assess disinvestment options. Criteria development is an iterative process wherein feedback is solicited from staff and key stakeholders with consensus reached among committee members. To support the incorporation of both the values of the organization as well as the public it serves, it is recommended that the Assessment Committee be comprised of managers, clinicians and staff, along with public representatives. While engagement of public members is difficult and not always feasible or appropriate, the framework supports their inclusion with a defined role in the decision making process.  A third committee, the Support Committee, provides essential tools to assist in the assessment of disinvestment options in the form of evidence, financial analysis and evaluative measures. Members may include researchers and financial personnel.  4.6.2 Identification  The identification phase is comprised of three steps. In the first two steps, the Oversight Committee defines the objectives and scope of the disinvestment process, including the overall goals of the decision making exercise and the service areas under review. It is also critical that expenditures are mapped, incorporating both capital and operating costs, and strategic priorities identified in order to link project objectives to the organization\u2019s mandate.  The third component in this first phase is to assess and determine candidates from a defined set of programs, services, interventions or business units; in some cases, all budget 62  \u000Citems may be considered viable options. In cases where options need to be drawn from a larger pool of programs and services, disinvestment candidates are arrived at through various methods of analysis and assessment. With input from the Support Committee and by applying evaluative tools such as adverse effect analysis, cost-effectiveness, and determining each candidate\u2019s overall system impact , candidates are identified (Giacomini et al, 2000; Wasserfallen, 2002; Elshaug et al, 2009). Other tools may include benchmarking, clinical variation, clinical pathway assessment and public service value. Once candidates have been identified, the Oversight Committee presents the list to the Assessment Committee for review and selection.  While the Oversight Committee is working through steps one to three, the Assessment Committee determines and defines the criteria, weights and rating scale against which the options will be measured (Elshaug et al, 2009; Mitton et al, 2011). Criteria development is an iterative process with input received from key stakeholders as determined by the organization.  4.6.3 Selection  The selection phase involves a screening process wherein identified candidates are assessed and selected for implementation from the larger list of options. The Assessment Committee appraises the disinvestment candidates against the predetermined criteria and applies a rating scale with scores calculated to obtain a total for each option (Mitton et al, 2011). Once scores have been calculated and agreed upon, the options are then rank-ordered. Selection begins at the top of the list with candidates chosen for implementation until process objectives are met. The selected options are then sent to the Oversight Committee to review. 63  \u000CBefore the final list is put forward for implementation the OC members must approve the candidate or make the necessary changes.  Once the list of disinvestment candidates is  finalized, an appeals process is enacted with appeals collected and responded to (Uehling, 1992). Any changes to the list are amended and adopted before moving to implementation.  4.6.4 Implementation  While implementation is a critical phase in any decision making process, it is the area where the least amount of research has been conducted. From the published processes, actions were identified that are required to ensure implementation is successful.  Working with  program leaders, the Oversight Committee develops an implementation plan including timelines for completion and clearly defined roles and responsibilities to ensure accountability is established (Wasserfallen, 2002). Clear and effective communication is a critical feature of the implementation phase wherein the results of the decision making process are communicated to staff and key stakeholders (Crompton &amp; Lamb, 1980; Dranikoff et al, 2002; Mitton et al, 2011). Implementation is closely monitored through regular meetings with program directors or managers and predetermined check points to assess progress. Once implementation is completed, funds are allocated according to the objectives of the disinvestment exercise: to meet the budgetary shortfall and/or to be used to reinvest into other services or programs.  64  \u000C4.6.5 Evaluation  Three types of evaluations comprise the fourth and final phase of the framework: an evaluation of the process itself, an outcome assessment, and an impact analysis of disinvestments implemented. While the evaluation phase was not well described or included in the majority of published processes, three processes from the health care literature (Wasserfallen, 2002; Stuart et al, 2010; Mitton et al, 2011) and one from the public sector and business literature (Diminnie &amp; Kwak, 1986) described this phase, which included an evaluation of the process and outcomes assessment. The framework developed has adopted both types of evaluations and has added a third: an impact analysis of the disinvestments implemented.  To measure the success of the process and to determine if the process is a worthwhile activity, it is recommended that an established evaluation framework be used such as Accountability for Reasonableness (A4R) that determines if the process was fair, transparent and equitable (Gibson, Mitton, Martin, Donaldson &amp; Singer, 2006). Other theoretical models that can be adopted from the evaluation research include methods-focused, use-focused and valuing-focused approaches (Smith, Mitton, Cornelissen, Gibson &amp; Peacock, 2012).  In  methods-focused approaches cause and effect drive the assessment with the outcome of the disinvestment as the primary measure and the evaluator perspective being central. Usefocused approaches, on the other hand, adopt the perspective of the decision maker, shifting the evaluation\u2019s focus from outcomes to \u2018what works\u2019. Finally, the valuing-focused approach takes the perspective of the public to include the assessment of more subjective measures such as power dynamics, equity and social justice. According to Smith et al (2012), all three 65  \u000Capproaches can guide evaluation efforts to ensure that the disinvestment framework offers real value to any resource allocation activity.  Evaluation devices that can be employed include holding public forums, facilitating focus groups, or conducting interviews with participants to determine what worked and what did not work in order for improvements to be identified and changes applied to the next iteration. An outcomes assessment can be captured from the organization\u2019s financial system to assess whether objectives were met in terms of released resources and how these resources were allocated.  An impact analysis may also be employed to determine the effect of  disinvestments on the individual program and the organization as a whole.  66  \u000CTable 11: Detailed description of the disinvestment framework Phase Identification  Step 1. Determine objectives and scope; map expenditures  Actions 1. Determine aim and scope of the disinvestment process and clarify decision-making roles and responsibilities \u2022 Ensure goals outline how released resources will be allocated: to overall budget and/or re-investment  Accountability Oversight Committee  Timeline Month 1  2. Perform a strategic assessment \u2022 Revisit mission, vision and values \u2022 SWOT analysis \u2022 Environmental scan to identify barriers/facilitators, stakeholders, decision inputs 3. Compile a map of existing activities and expenditures \u2022 Determine Net Economic Value: NEV = Total revenue - operating costs - capital charges [adjusted for inflation] 4. Communicate aim, scope, goals and process to key stakeholders; provide opportunity for feedback  Identification  2. Identify strategic priorities and develop criteria  5. Provide training to OC and AC members on disinvestment process, ethical principles and decision-making tools  Support Committee  1. Develop decision-making criteria and weighting system; obtain feedback and revise  Assessment Committee  2. Identify strategic priorities; revise based on feedback from stakeholders  Oversight Committee  Month 12  3. Work with internal staff to develop communications plans for key stakeholder groups 4. Develop business case template 5. Develop appeals process 6. Determine evaluation framework  67  \u000CPhase Identification  Step 3. Identify disinvestment options and risks  Actions  Accountability  Timeline  Oversight Committee \u2022 Input from managers, clinicians Oversight Committee \u2022 Data analysts \u2022 Researchers \u2022 Finance staff \u2022 Managers \u2022 Clinicians  Month 2  3. Review business case proposals and evaluate against criteria; determine feasible options and provide feedback on unsuccessful proposals  Assessment Committee  Month 34  4. Identify risks associated with each disinvestment option  Oversight Committee  1. Call out for business case proposals  2. Conduct analyses to determine disinvestment options \u2022 Benchmarking \u2022 Reduce clinical variation \u2022 Assess clinical pathways \u2022 Performance frameworks  Month 23  5. Finalize list of options based on feasibility, criteria and risks; send list to Assessment Committee Selection  4. Rank options  1. Assess disinvestment options and rank  Assessment Committee  Month 4  Oversight Committee  Month 45  2. Present final ranked options to OC members 3. Assess AC decisions and determine final ranked disinvestments 4. Communicate decision to OC members and all stakeholder groups 5. Enact appeals process; assess appeals and communicate decisions Implementation  5. Develop implementation plan  1. Determine roles and allocate responsibilities 2. Prepare risk mediation plans  Month 5 Oversight Committee  Month 56  3. Prepare communication materials 6. Conduct disinvestment  1. Execute risk mediation and communication plans  Month 6  2. Oversee disinvestment activities; hold regular meetings with directors/managers to ensure accountability  68  \u000CPhase  Step  Actions  Implementation  6. Conduct disinvestment  3. Allocate resources  Evaluation  7. Assess disinvestment process  1. Determine if objectives were met and evaluate process based on predetermined framework  Accountability Oversight Committee  Timeline Month 68 Month 8  2. Engage stakeholders in identifying good practices and opportunities to improve process 3. Develop plan to implement suggestions for improvement into next fiscal cycle  69  \u000C5. DISCUSSION  The findings of this study revealed variability between how the different sectors approach disinvestment and its related activities. From the perspective of time, when combining sectors, data indicated that the 2000s produced the greatest number of articles related to disinvestment research and practice, with both the public and business sectors consistently engaged in disinvestment activities from 1970 onwards; there was a clear pattern of growth in the health care literature with the majority of research conducted more recently. These results are perhaps a reflection of a shift in resource allocation processes, moving from across-the-board cuts to the need for more fair and defined approaches; this also may represent a change to government mandates to be more transparent in decision-making activities (Elshaug et al, 2008).  5.1  Definition of disinvestment  Of greatest interest was the difference in how sectors defined disinvestment. Health care often applied a more limited definition that, when operationalized, identifies and terminates those services and interventions that have little or no value. Perhaps this focus on absolute disinvestment is due to the challenges faced when conducting relative disinvestment. Even though all sectors acknowledged that barriers to disinvestment exist, disinvestment in business appears to be less complex with profitability at the forefront of most resource allocation decisions. While the goals and objectives of public sector institutions vary, similar to business, in many public sector institutions, the market has a strong influence on the value of a service or program: consumer demand can prompt a program\u2019s initiation, and conversely, 70  \u000Cwhen demand wanes, its termination (Crompton &amp; Lamb, 1980). For example, if the hightech sector is flooded with qualified computer technicians, student enrollment in relevant postsecondary education programs may decline to such an extent that their removal from an institution\u2019s curriculum is warranted; the consumer chooses what to purchase, and in doing so decides which product, service or program is of greater value. Such a scenario does not hold true across all public sector institutions. Municipal governments, for example, reside outside of market forces; services such as fire and emergency services, police services, and transportation are often deemed to be essential although not always exempt from budgetary cuts (Petry, 2004).  In a publicly-funded health care system, the consumer, and therefore the market, has little influence in determining the relative value of services or interventions. As such, relative value is difficult to measure and resource allocation decisions that require its incorporation become more politically-driven: which services or interventions can be removed or scaledback without incurring public backlash?  While the research supports the removal of ineffective or inappropriate programs or technologies, this thesis also recognizes the need to identify lower value services in order to scale back operations, despite the difficulty in doing so. Both business and public sector support this approach with examples also drawn from the health care literature. In business, researchers describe this type of disinvestment activity as the pruning of fruit-bearing branches that allows the rest of the tree to bloom to its full potential thus producing the best results (Dranikoff et al, 2002).  In other words, disinvestment should be considered an  71  \u000Cessential tool to achieve the greatest possible value from a pool of limited resources (Garner &amp; Littlejohns, 2011).  5.1.1 Proposal of a new definition  Applying the acquired knowledge from all three sectors, a change to the definition of disinvestment as referenced in the health care literature is proposed. This thesis puts forward the adoption of the third definition category that combines two distinct but complimentary parts:  1. The identification and removal of services, treatments and interventions that are ineffective and/or inappropriate; and  2. The identification of services, treatments and interventions that are effective, but of lower value than other services and treatments.  Within this proposed definition, the term \u2018effective\u2019 is a relative measure permitting a choice of options in the provision of a treatment, intervention or service.  The relative  effectiveness measure as contained within the revised definition would produce three potential options for disinvestment: 1) stop a treatment, intervention or service; 2) scale back a treatment, intervention, or service; or 3) replace a treatment, intervention or service with an alternative. Such a decision to replace a treatment, intervention or service would involve a comparison of existing practices with the lower cost alternative.  72  \u000CThe inclusion of the term \u2018value\u2019 also raises important considerations when attempting to apply its meaning within any particular context. To determine a program, service or intervention\u2019s value is to assess its worth in comparison to other programs, services, or interventions as measured against an organization\u2019s principles, mandate and goals. While costeffectiveness may be one of the measures used to determine value, additional principles such as access, appropriateness, and equity may also be relevant and appropriate measures to apply. As a subjective and inclusive term, value is highly adaptable; to impose a more fixed measure within the definition would limit its applicability to such diverse and highly complex systems as health service delivery organizations.  In operationalizing the definition, the exclusion of cost savings approaches as previously described is necessary since the two concepts are distinct. When an organization engages in cost savings activities the goal is to lower costs within a program, service or intervention compared to historical costs while maintaining or improving health outcomes; it is also referred to as cost containment (Dionne et al, 2009). Common cost savings tools identified in the literature include service outsourcing, re-engineering, and clinical or nonclinical integration, consolidation or standardization of service areas. Such engagement may assist in reducing the overall budget of an organization, yet it may not be sufficient to meet budgetary needs.  Disinvestment, on the other hand, does not seek to decrease the cost of treatments or services in terms of price to produce said treatment or service; rather, its goal is to reduce the overall budget through the removal or reduction of ineffective or less effective services or programs. To meet budgetary constraints, an organization may first decide to engage in cost 73  \u000Csavings measures and if such measures are not enough to meet budgetary needs then disinvestment activities as per the proposed definition could be introduced.  However,  disinvestment may also be applied as a first response to budget cuts independent of cost saving efforts or be used to free-up resources that can then be re-invested into existing programs or new initiatives.  5.2  Disinvestment processes  Another key finding from this study is the need to include disinvestment activities on an on-going basis (Dranikoff et al, 2002). Both the public sector and business literatures emphasized the importance of adopting a proactive approach to resource planning by identifying options for disinvestment alongside the generation of investment opportunities. In health care, disinvestment is often considered only when a fiscal crisis arises, one that often is externally motivated. This thesis proposes that health care organizations introduce a flagging system to determine ineffective and lower value services throughout the fiscal cycle. Such a flagging system will enable organizations to be well prepared for potential budget reductions, ensure the most effective use of finite health care resources, and assist in normalizing the concept of disinvestment so that it becomes part of day-to-day operations. As such, disinvestment may be more widely accepted by stakeholder groups as a common and necessary step in the allocation of resources.  From the systematic and scoping review, several trends were found in the identification, selection, implementation and evaluation phases of a disinvestment process. Although the vast majority of disinvestment processes reported on the components that 74  \u000Ccomprise the first two phases, identification and selection, two processes from the health care literature (Wasserfallen, 2002; Mitton et al, 2011) and three from the public sector and business literature (Crompton &amp; Lamb, 1980; Dranikoff, 2002; Petry, 2004) provided information about the implementation of selected options which included planning, communicating decisions, and defining roles and responsibilities. Only three health care processes described components in the evaluation phase, with two describing an evaluation of the process itself (Wasserfallen, 2002; Mitton et al, 2011) and three that measured the process\u2019s success (Wasserfallen, 2002; Stuart et al, 2010; Mitton et al, 2011). One process from the public sector and business search noted the inclusion of a process evaluation (Diminnie &amp; Kwak, 1986) that described the experience of the process participants and their acceptance of the decision making framework.  Neither of the two reviews identified a process wherein decision outcomes were measured and evaluated even though the inclusion of such a component is necessary to determine feasibility of the options and to identify potential barriers to implementation. Results from both reviews also revealed shortcomings in the use of formal processes to prioritize options with four processes from health care (Goplerud et al, 1985; Wasserfallen, 2002; Elshaug, 2009; Mitton et al, 2011) and four from the public sector and business (Algie et al, 1984; Diminnie &amp; Kwak, 1986; Petry, 2004; Packard et al, 2008) reporting on this step in the process.  Several models, including PBMA (Donaldson et al, 2010; Mitton et al, 2011; Mortimer, 2010), the Delphi-decision making approach (Goplerud et al, 1985) and the participative approach (Wasserfallen, 2002) from the health care literature, and the 75  \u000Chierarchical goal programming approach (Diminnie &amp; Kwak, 1986) from the public sector and business literature, demonstrated effectiveness in determining options for disinvestment and evaluating the processes once completed. However, gaps still remain, in particular around implementation of disinvestment decisions and evaluation of those decisions once they are completed, both of which need to be included in any process that seeks to address budgetary constraints.  Overall, while processes could be identified from the literature few provided a robust framework for decision making specifically related to disinvestment. It may be that process discrepancies arise due to how case studies were reported: not all components were described even though they may have been included in the decision making process. Regardless of the potential for selective reporting, from the processes identified five of the articles from the health care literature (Goplerud et al, 1985; Giacomini et al, 2000; Wasserfallen, 2002; Stuart et al, 2010; Mitton et al, 2011) and three from the public sector (Diminnie et al, 1986; Petry, 2004; Packard et al, 2008) present case studies wherein processes were tested in a real world resource allocation exercise.  This lack of trialed processes speaks to the need for more appropriate and tested models to be developed. It also suggests that disinvestment decisions are either a rare event or one that is rarely reported, or when they are made they are most likely conducted through a less methodical and systematic approach. This is perhaps surprising noting that the economic crisis was well under way by the end of 2008. Even when allowing for publication delay, one might reasonably have expected to see greater focus on disinvestment processes, not to mention the reporting of actions related to recessions in previous decades. 76  \u000C5.3  Development of a disinvestment framework  One of the aims of this research was to develop a comprehensive framework to aid in disinvestment activity for use within health service delivery organizations.  Given the  limitations identified from existing published processes, it was considered necessary to design a new framework that incorporates the broad range of factors involved in a decision making approach of this nature. At the same time, the framework seeks to address some of the challenges that deter decision makers from attempting to disinvest from existing programs, services or interventions.  The three central challenges identified from the literature included system complexity, lack of resources, and resistance. Through the adoption of a comparator-type of assessment wherein disinvestment candidates are measured against predetermined criteria, the potential of service demand being shifted to other areas in the organization is diminished. The inclusion of program and department representatives in the decision making process provides a forum for discussions between members wherein potential issues can be addressed before decisions are made. While the framework presented does not negate the difficulty that arises when bringing together diverse individuals for the purposes of collaborative decision making, it may be one practical tool that can help mitigate the complexity around disinvestment in resource allocation activities.  As Mitton et al (2011) described, the lack of expertise among process participants was remedied through extensive training and support. Such training not only promotes greater equity among decision makers, enabling a more equal voice among participants, but also helps 77  \u000Cto build capacity within the organization, allowing for a more inclusive process to be developed. Although the draft framework is modeled on a more centralized approach, decentralization could occur should such capacity be developed. Other resource issues include a lack of data and the time and costs that such a process requires. Since cost-effectiveness may be only one of many criteria against which options are assessed, a lack of data is more surmountable within this type of process.  Time and costs required to conduct such an activity have not been evaluated as of yet. However, one needs to consider whether the framework provides real-value-added, that is, if the process satisfies the expectations of those individuals directly affected. While some providers and public members may not agree with the individual decisions made, the overall result \u2013 a more effective service delivery system \u2013 and the process applied to reach these decisions can be seen as fair and equitable.  While the draft framework assists in addressing several potential barriers, many of the challenges reported in the literature are the result of how providers and the public perceive disinvestments. Resistance to disinvestment may only be modified through the application of a transparent process that enables such decisions to be incorporated into annual resource allocation activities rather than being adopted only during a fiscal crisis. If disinvestment is associated only with monetary constraints or deprivation, then resistance will persist. However, if it is realigned to promote service delivery effectiveness and assist in improving population level health outcomes, then perhaps acceptance of disinvestment can be fostered and its inclusion more readily endorsed.  78  \u000CHowever, within the Canadian context, the buy-in from key stakeholders remains problematic. With physicians paid on a fee-for-service basis, their inclusion in disinvestment decision making processes, regardless of the framework applied, is difficult since the removal or reduction of services can lead to direct wage loss. Until a payment model is adopted that shifts physicians\u2019 focus away from the number of services provided to a more inclusive approach, physician resistance will remain a challenge to any disinvestment activity.  79  \u000C6. LIMITATIONS  While this study involved 44 peer-reviewed papers that self-identified as being disinvestment focused, with 24 found through a systematic review of the health care literature and 20 from business and public administration literatures, limitations to the searches have been identified. First, the term disinvestment was not recognized as a subject heading in any of the six databases searched.  This exclusion required the project team to identify an  exhaustive list of synonyms and broader concepts in order to ensure a high capture rate of relevant articles.  The search protocol was revised for each database search with terms  changed and adjusted accordingly. While replication of these searches is possible, it is more difficult than a more traditional systematic review.  Second, the number of relevant, peer-reviewed papers was small which limited the scope of the analysis; due to the lack of available data, the ability to identify and describe fully all four phases of a disinvestment process with implementation and evaluation phases was difficult. However, this small number of processes speaks to the dearth of disinvestment research that has been conducted to date and highlights the need for further research and development.  Finally, the proposed framework, while incorporating components and attributes identified from published processes, has not been tested in the \u2018real world\u2019, and as such, it cannot be fully endorsed until such an application and subsequent evaluation has been conducted.  80  \u000C7. CONCLUSIONS  It is evident from the results of this research that there is a need to apply lessons from business and the public sector to health care service research where disinvestment is less common and less advanced. The reviews have indicated a need to expand the disinvestment definition to include the identification of lower value services if budget constraints are to be met; distinguish between cost savings and disinvestment thereby allowing decision makers greater flexibility in how they address funding shortfalls; and encourage the ongoing and routine flagging of disinvestment options that will assist in bringing the concept of disinvestment from the shadows into the light.  Clearly, further study is needed in terms of reporting and evaluating of empirical disinvestment exercises. This work also encourages other researchers to challenge, critique and test the proposed definition in order to build a more robust body of work that will better inform health care resource allocation policy and practice.  Assessing the papers identified from both the systematic review of the health care literature and scoping review of the public sector and business literatures revealed that while budgetary cutbacks are experienced across government, non-profit and the private sector, very few processes have been developed to identify and implement disinvestment options. Regardless, from the processes captured it was possible to determine key attributes and common components that can be used to identify four phases that are presumed to be part of a robust and transparent disinvestment process: the identification of disinvestment candidates; the selection of candidates for implementation; the implementation phase wherein decisions 81  \u000Care acted upon; and the evaluation of decision outcomes and the disinvestment process in order to identify areas for improvement.  No single process was found that could address the unique context that an environment of fiscal constraint produces. Further, less than half of the processes included all four phases with the majority focusing their efforts on identification and selection even though implementation is often considered to be the most difficult and the area where resistance occurs. As such, future health care services research should be designed to develop, test and refine disinvestment practices in order to build more applicable models of a decision making process that address resource allocation during fiscal constraint.  Despite the critical need for disinvestment activities to be incorporated into decision making processes, there is no single approach that can be applied as a standalone tool or be integrated into already existing resource allocation processes. The framework proposed in this thesis attempts to address potential challenges to disinvestment while providing a transparent process that involves the participation of a wide range of stakeholders. Since no one solution could be found, the framework is a novel addition to the literature and as such, it has the potential to improve decision making practices while supporting the adoption of disinvestment activities into the annual budgetary cycle.  Presentations of the framework to the external decision maker advisory committee comprised of Chief Financial Officers from Health Authorities across Western Canada resulted in support for its application. As a result, a commitment has been made to test the framework in a six-month pilot exercise within one of these organizations. An evaluation will 82  \u000Cbe conducted and the results published to better inform disinvestment practices within publically funded health service delivery organizations.  83  \u000CREFERENCES  Algie, J., Mallen, G. &amp; Foster, W. (1983). Financial cutback decisions by priority scaling. Journal of Management Studies, 20(2), 233-257.  Banta, H.D. &amp; de Wit, G.A. (2008). Public health services and cost-effectiveness analysis. Annual Review of Public Health, 29, 383-397.  Braun, V. &amp; Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77\u2013101.  Caldwell, C., Butler, G., Poston, N. (2010). Cost reduction in health systems: Lessons from an analysis of $200 million saved by top-performing organizations. Frontiers of Health Services Management,27(2).  Cragg, L.C. (2003, Summer). Resource maximization: using risk management to rethink budgetary choices in hard times. The Public Manager, 32 (2), 58.  Crompton, J.L. &amp; Lamb, C.W. (1980). Eliminating community services - The leisure services example. Community Development Journal, 15(2), 138-145.  84  \u000CDepartment of Health.(2006). The Chief Medical Officer on the state of public health Annual Report 2005. s.l.: Department of Health. Retrieved from http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/AnnualReports/DH_4 137366  Diminnie, C.B. &amp; Kwak, N.K. (1986). A hierarchical goal-programming approach to reverse resource allocation in institutions of higher learning. The Journal of the Operational Research Society, 1, 59-66.  Dionne F., Mitton C., Shoveller J., Peacock S. &amp; Barer M. (2009). How to control the costs of health care services - an inventory of strategic options. Healthcare Management Forum. 22(4), 23-30.  Donaldson, C., Currie, G. &amp; Mitton, C. (2002). Cost effectiveness analysis in health care: contraindications. BMJ, 325, 891-894.  Donaldson, C., Bate, A., Mitton, C., Dionne, F., &amp; Ruta, D. (2010). Rational disinvestment. QJM: An International Journal of Medicine, 103, 801-807.  Dranikoff, L., Koller, T. &amp; Schneider, A. (2002, May). Divestiture: Strategy's missing link. Harvard Business Review, 74-83.  85  \u000CElshaug, A.G., Hiller, J.E., Tunis, S.R. &amp; Moss, J.R. (2007). Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Australia and New Zealand Health Policy, 4(23), doi: 10.1186/1743-8462-4-23.  Elshaug, A.G., Hiller, J.E. &amp; Moss, J.R.(2008). Exploring policy-makers' perspectives on disinvestment from ineffective health care practices. International Journal of Technology Assessment in Health Care, 24 (1), 1-9.  Elshaug, A.G., Moss, J.R., Littlejohns, P., Karnon, J., Merlin, T.L. &amp; Hiller, J.E. (2009a). Identifying existing health care services that do not provide value for money. Medical Journal of Australia, 190, 269-273.  Elshaug, A.G., Watt, A.M., Moss, J.R., &amp; Hiller, J.M. (2009b, October). Policy Perspectives on the Obsolescence of Health Technologies in Canada. Canadian Agency for Drugs and Technologies in Health (CADTH). Retrieved from http://cadth.ca/policyforum/discussion-papers/policy-perspectives-on-the-obsolescence-of-health  Etizoni, A. (1986). Mixed scanning revisited. Public Administration Review, 46(1), 8-14.  EUnetHTA. (2008, June 16). EUnetHTA collaboration proposal - way forward for HTA in Europe. EUnetHTA. Retrieved from http://www.eunethta.eu/upload/EUnetHTA%20Collaboration%20proposal/EUnetHTA %20Collaboration%20Proposal_Final_June2008.pdf  86  \u000CEvans, R. (2007). Extravagant Americans, healthier Canadians: The bottom line in North American health care. In: D. M. Thomas (ed.), Canada and the United States: Differences that Count. Peterborough, ON: Broadview Press.  Garner, S. &amp; Littlejohns, P. (2011). Disinvestment from low value clinical interventions: NICEly done? BMJ, 343, doi: 10.1136/bmj.d4519.  Gerard, K. (1992). Cost-utility in practice: A policy maker\u2019s guide to the state of the art. Health Policy, 21, 249-279.  Geva-May, I. (2001). When the motto is \"till death do us part\": The conceptualization and the craft of termination in the public policy cycle. International Journal of Public Administration, 24(3), 263-288.  Giacomini, M. Hurley, J. &amp; Stoddart, G. (2000). The many meanings of deinsuring a health service: The case of in vitro fertilization in Ontario. Social Science &amp; Medicine, 50,1485-2000.  Goplerud, E., Walfish, S. &amp; Broskowski, A. (1985). Weathering the cuts: a delphi survey on surviving cutbacks in community mental health. Community Mental Health Journal, 21, 14-26.  87  \u000CHarrison, E.F. (1993). Interdisciplinary models of decision making. Management Decision, 31(8), 27-33.  Health Of Commons Health Committee. (2002, July 3). Second Report of Session 2001\u201302, Volume I: Report and Proceedings of the Committee. House of Commons London : The Stationery Office Limited. Retrieved from http://www.publications.parliament.uk/pa/cm200102/cmselect/cmhealth/515/515.pdf  Hughes, D. &amp; Ferner, R. (2010). New drugs for old: disinvestment and NICE. British Medical Journal, 340(572), 690-692.  Ibargoyen-Roteta, N., Gutierrez-Iborluzea, I. &amp; Asua, J.(2010). Guiding the process of health technology disinvestment. Health Policy, 98m 218-226.  Karnon, J., Carlton, J., Czoski-Murray, C. &amp; Smith, K. (2009). Informing disinvestment through cost-effectiveness modeling: is lack of data a surmountable barrier? Applied Health Economics and Health Policy, 7(1), 1-9.  Leatt, P., Baker, G.R., Halverson, P.K., &amp; Aird, C. (1997). Downsizing, reengineering, and restructuring: Long-term implications for healthcare organizations. Frontiers of Health Services Management, 13(4), 3-37.  88  \u000CLevine, C.H. (1978, July/August). Organizational decline and cutback management. Public Administration Review, 316-325.  Mariner, W.K. (1995). Rationing health care and the need for credible scarcity: why Americans can\u2019t say no. American Journal of Public Health, 85(10), 1439-1444.  Michael, S. O. (1998). Restructuring US higher education: Analyzing models for academic program review and discontinuation. The Review of Higher Education, 21(4), 377-404.  Mitton, C., Dionne, F., Damji, R., Campbell, D. &amp; Bryan, S. (2011). Difficult decisions in times of constraint: Criteria based resource allocation in the Vancouver Coastal Health authority. BMC Health Services Research, 11(169), doi: 10.1186/1472-6963-11-169.  Mortimer, D. (2010). Reorienting programme budgeting and marginal analysis (PBMA) towards disinvestment. BMC Health Services Research, 10(288), doi:10.1186/14726963-10-288.  National Institute for Health and Clinical Excellence. (2006, September 5). NICE welcomes new initiative to help NHS reduce. Retrieved from http://www.nice.org.uk/niceMedia/pdf/2006_042_reducing_spending_on_innefective_ treatments.pdf  89  \u000CNuti, S., Vainieri, M. &amp; Bonini, A. (2010). Disinvestment for re-allocation: A process to identify priorities in health care. Health Policy, 95, 137-143.  Nutt, P.C. &amp; Backoff, R.W. (1995). Strategy for public and third-sector organizations. Journal of Public Administration Research and Theory, 5, 189-211.  Packard, T., Patti, R., Daly, D., Tucker-Tatlow &amp; Farrell, C. (2008). Cutback management strategies: Experiences in nine county human services agencies. Administration in Social Work, 33(1), 55-75.  Peacock, S., Ruta, D., Mitton, C., Donaldson, C., Bate, A. &amp; Murtagh, M. (2006). Using economics to set pragmatic and ethical priorities. British Medical Journal, 332, 482485.  Pearson, J.V., Michael, H., &amp; Michael, R.J. (1981). Zero-base budgeting - a technique for planned organizational decline. Long Range Planning, 14(3), 68-75.  Pearson, S. &amp; Littlejohns, P. (2007). Reallocating resources: how should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service? Journal of Health Services Research &amp; Policy, 12, 160-165.  Petry, J. (2004, February). Bringing rigor to cutback management: Eugene's constrained prioritization process. Government Finance Officers Association, 20 (1), 18-23.  90  \u000CPoetz, A. &amp; Hoch, J.S. (2005) Understanding cost-utility analysis in health care. University of Toronto Medical Journal, 83 (2), 126-130.  Pope, C., Mays, N. &amp; Popay, J. (2007). Synthesizing Qualitative and Quantitative Health Evidence. Maidenhead, Berkshire, England: Open Universtiy Press.  Rainey, H.G., Ronquillo, J.C. &amp; Avellaneda, C.N. (2010). Decision making in public organizations. In: P.C. Nutt and D.C. Wilson (Eds.), Handbook of Decision Making (pp. 349-377). Chichester, West Sussex, UK: John Wiley &amp; Sons, Ltd.  Rich, G., Leonard, P., Zalmanovitch, Y. &amp; Vashdi, D.R. (2010, March). Can we make cuts that will not harm health care? British Medical Journal, 340, 628-631.  Robinson, S., Dickinson, H., Freeman, T, Williams, I. (2011). Disinvestment in health \u2013 the challenges facing general practitioner (GP) commissioners. Public Money &amp; Management, 31(2), 145-148.  Schwappach, D.L.B. (2002). Resource allocation, social values and the QALY: a review of the debate and empirical evidence. Health Expectations, 5, 210-222.  Smith, G. &amp; May, D. (1980). The artificial debate between rationalist and incrementalist models of decision making. Policy and Politics, 8(2), 147-161.  91  \u000CSmith, N., Mitton, C., Cornelissen, E., Gibson, J., &amp; Peacock, S. (2012). Using evaluation theory in priority setting and resource allocation. Journal of Health Organization and Management, 26(5), 655-671.  Stuart, G.W., Erkel, E.A. &amp; Shull, L.H. (2010). Allocating resources in a data-driven college of nursing. Nursing Outlook, 58, 200-296.  Tarter, C.J. &amp; Hoy, W.K. (1998). Towared a contingency theory of decision making. Journal of Educational Administration, 35(3), 212-228.  Uehling, B. (1992). So money is a problem. Journal for Higher Education Management, 7(2), 7-13.  Wallender III, H. (1973, July). A planned approach to divestment. Management Review, 62(7), 54-8.  Wasserfallen, J.B. (2002). Cost reduction project in a Swiss university hospital: Methods and results of a bottom-up intervention. Journal d \u2019Economie Medicale, 20(1), 33-43.  92  \u000CAPPENDIX A Database: Ovid MEDLINE(R) 1946 to Present with Daily Update Search Strategy: ----------------------------------------------------------1 national health programs/ (23377) 2 national health insurance, united states/ (1955) 3 single-payer system/ (390) 4 state medicine/ (44943) 5 regional health planning/ (4914) 6 community health planning/ (4145) 7 regional medical programs/ (3390) 8 state health plans/ (4197) 9 government agencies/ (12337) 10 technology assessment, biomedical/ or technology, high-cost/ (8714) 11 Primary Health Care/ (48176) 12 Public Health/ (55162) 13 Community Health Services/ (25365) 14 hospitals/ or exp hospitals, public/ (75033) 15 Hospitals, Community/ (9771) 16 Family Practice/ (59528) 17 Health Services Accessibility/ (44527) 18 or/1-17 (381671) 19 Health Policy/ (46055) 20 leadership/ (24508) 21 policy making/ (11000) 22 Decision Making, Organizational/ (10130) 23 decision making/ (60697) 24 or/19-23 (144877) 25 resource allocation/ (6764) 26 health care rationing/ (9919) 27 health priorities/ (8203) 28 Efficiency, Organizational/ (16088) 29 Organizational Innovation/ (19333) 30 Cost Allocation/ (1918) 31 \"cost control\"/ (19240) 32 \"cost savings\"/ (7667) 33 Cost-Benefit Analysis/ (54495) 34 or/25-33 (130438) 35 18 and 34 (25371) 36 Economics, Pharmaceutical/ (2347) 37 Investments/ (6818) 38 Health Expenditures/ (12478) 39 ec.fs. [economics] (305065) 40 budgets/ (9145) 41 Total Quality Management/ (11559) 42 Financial Management/ (15035) 43 Financing, Government/ (17740) 44 or/36-43 (339287)  93  \u000C45 46 47 48 49 50 51 52 53 54 55 55 56  18 and 34 and 44 (14329) og.fs. [organization &amp; administration] (345654) 18 and 34 and 44 and 46 (4458) 24 and 34 and 44 (6944) 24 and 34 and 44 and 46 (2327) 47 or 49 (5927) comment/ or editorial/ or letter/ (1139659) 50 not 51 (5532) limit 52 to English language (5297) limit 53 to yr=\"1970 - 1979\" (117) limit 53 to yr=\"1980 - 1989\" (186) limit 53 to yr=\"1990 - 1999\" (2207) limit 53 to yr=\"2000 - 2011\" (2787)  Database: Ovid MEDLINE(R) 1946 to Present with Daily Update Search Strategy: ----------------------------------------------------------1 disinvestment.tw. (57) 2 discontinuation.tw. (27034) 3 decommission.tw. (11) 4 deinvestment.tw. (1) 5 (termination and priority setting).tw. (0)  94  \u000CAPPENDIX B Assessment questions 1.  Is disinvestment explicitly defined?  2.  How is disinvestment defined?  3.  Other terms used to describe the concept of disinvestment.  4.  Is the article peer-reviewed?  5.  Date of publication.  6. 7.  Type of publication: case-study, commentary/opinion, descriptive or other What is the article\u2019s primary theme or question?  8.  Country where disinvestment is applied.  9.  Type of organization: government, private or non-profit (with subcategories) Purpose of the article: academic/theory-oriented or practiceoriented. Sector: health or other (with subcategories)  10. 11. 12. 13.  If a disinvestment activity was undertaken, describe the framework or method used. What were the barriers to disinvestment?  14.  What were the facilitators to disinvestment?  15.  What was the reason for disinvestment?  16.  What was the scope of the disinvestment activity?  17.  Was the disinvestment activity a success?  18.  How was success measured?  19.  What new knowledge was produced?  20.  What were the author\u2019s conclusions?  21.  What was the author\u2019s suggestion for future research, if any?  95  "@en
<https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0073252>