extfieldsofstudy:
Computer Science
Medicine

text:
A comparison of stochastic programming methods for portfolio level decision-making

ABSTRACT Several methods have been presented in the literature for the management of a pharmaceutical portfolio, i.e. selecting which clinical studies should be conducted. We compare two existing approaches that use stochastic programming techniques and formulate the problem as a mixed integer linear programme (MILP). The first approach will be referred to as the ROV (real option valuation) approach since values are assigned to drug development programmes using methods for real option valuation. The second approach will be referred to as the PS (project scheduling) approach as this approach focusses on the scheduling of clinical studies and is formulated similarly to the resource constrained project scheduling problem. The ROV approach treats the value of a drug development programme as stochastic whereas the PS approach treats the trial outcomes as the stochastic component of the programme. As a consequence, the two approaches may select different portfolios. An advantage of the PS approach is that a schedule for when trials are to be conducted is provided as part of the optimal solution. This advantage comes at a much increased computational burden, however.

Introduction
The drug development process is both long and expensive with typical drug development programmes taking 10-15 years from drug discovery to launch and costing hundreds of millions of pounds (DiMasi et al. 2016). Not only is the process long and expensive, it also contains a high level of uncertainty. Each of the developmental tasks within a drug development programme may take longer or cost more than was originally expected, which can cause problems with scheduling and budgeting. The biggest source of uncertainty comes from the fact that the outcomes of the clinical studies are unknown. Supposing a novel treatment does perform well in all of its associated studies and is approved then launched, it will also encounter uncertainty in the revenue that will be generated and the impact that it will have due to the many external factors that affect these outcomes, such as competitors in the marketplace and demand (Colvin and Maravelias 2011).
Traditionally a drug development programme is the sequence of tasks a single drug must undertake in order to make it to being launched for use by the target population. A drug development programme may be thought of as a sequence of five main parts, which are highlighted in Figure 1.
Typically, a pharmaceutical company will have several products undergoing clinical studies within their portfolio including new products and existing products that are being tested in a different indication. One challenge that arises when we consider portfolios rather than individual drug development programmes is the fact that the decisions made within one drug development programme are likely to have an effect on the other programmes in the portfolio. This is due to the fact that a pharmaceutical company will have a finite level of resources and budget for which the drugs in the portfolio will be competing. Consequently, within a pharmaceutical portfolio the decisions will include selecting the studies that should be prioritised, considering when to either abandon or defer development for certain drugs given the current state of the portfolio and other logistic decisions such as scheduling and budget/resource allocation.
One of the biggest challenges within drug development and portfolio management is the previously mentioned stochasticity of the underlying process. Therefore, while a pharmaceutical company will make sure that care is taken in ensuring that the drug they are developing will be beneficial to both the company and the target patient population, the true performance of a drug is an unknown parameter and can only ever be estimated given relevant data.
The success rates in the different phases of drug development are presented in Table 1, where success is defined to be transition to the next phase of development (Thomas et al. 2016). The most common reasons for a drug not continuing to the next stage of development are also presented in Table 1, with the most common reason for failure being due to a lack of efficacy (Arrowsmith and Miller 2013). In Phase II, 59% of failures are due to a lack of efficacy and in Phase III this lowers slightly to 52%.
Reducing these failure rates, especially those corresponding to a lack of efficacy, will be beneficial to a pharmaceutical company so that resources can be used elsewhere. Therefore, ensuring that decisions concerning a pharmaceutical portfolio are well-informed and consider the associated uncertainties in each stage of development, alongside the expected costs and the revenue that would be generated if the drug were commercialised in comparison with the other drugs in the portfolio, is of high importance.
Several methods have been presented in the literature for the management of a pharmaceutical portfolio in terms of selecting which studies should be conducted. These methods typically employ optimisation techniques but draw upon a range of different areas and often have different focuses. Schmidt and Grossmann (1996) presented a model for scheduling non-sequential testing tasks with an aim to maximise the expected net present value and provided several reformulations that assist in solving the model and focus on different aspects of the problem. Rotstein et al. (1999) presented a twostage stochastic programme that also aims to maximise the expected net present value of the portfolio. This model considers decisions such as capacity investments, product selection and resource allocation Table 1. Success rates in the different phases of drug development (Thomas et al. 2016) and the reasons for drugs failing to continue to the next stage of development between -2012(Arrowsmith and Miller 2013 in the first stage and capacity adjustments in the second stage. Blau et al. (2004) model drug development programmes using probabilistic network models. Bubble charts are used to find a prioritisation scheme for the drugs that is then used as a starting point for the genetic algorithm-based search. The search takes into account interdependencies between drugs and resource constraints. The aim of the method is to find the solution that maximises the expected net present value for a given level of risk. Varma et al. (2008) presented a method that combines stochastic simulation and mixed-integer linear programming in order to maximise the expected net present value of the portfolio while also evaluating different strategies on the pipeline. Sundaramoorthy et al. (2012) presented a multi-scenario multiperiod mixed-integer linear programme that aims to maximise the expected net present value of the portfolio. The decisions considered in the formulation include things such as building/expanding facilities, capacity decisions, production levels and storage decisions but it considers no resource constraints during product development.
In this manuscript, we will focus on two approaches that are of particular interest because they provide models which capture the true aspects of pharmaceutical portfolio management. These approaches focus on the process of decision-making when we are considering multiple drug development programmes and consider how we might compare these different programmes that are in different stages of development rather than focussing on different aspects of production. They also provide clear results and information on the decisions that should be made to help the decisionmaker achieve their goal. These are the reasons we will limit our attention to these approaches.
Both of the methods that we will discuss draw upon stochastic programming techniques and formulate the problem using mixed integer linear programmes. Stochastic programming is beneficial in the setting of pharmaceutical portfolio management due to the inherent stochastic nature of the process. Stochastic programming allows us to model the uncertainty of the process and let this contribute to the decisions that are made. Furthermore, it allows us to consider what the optimal decisions might be based on different outcomes of the uncertain process. This will then allow decision-makers to consider the impact of certain decisions and to compare different sets of decisions in terms of the costs incurred and potential benefits.
In Section 2, we will describe the two approaches that are the main interest of this manuscript. We will highlight that, while both use a similar methodology, the focus of the approaches is actually quite different. The first approach by Rogers et al. (2002) draws upon real option valuation from the financial setting and the focus of this approach is the stochasticity in the value of the drug development programmes. The second approach by Colvin and Maravelias (2008) and Colvin (2010) is similar to the formulation of the resource constrained project scheduling problem and the stochasticity considered here is in the uncertainty of the trial outcomes, which are the main focus of the problem.

Stochastic programming methods
In this section, we will review two portfolio management approaches that are based on stochastic programming and formulate the decision-making process as a mixed integer linear programme (MILP). We will then go on to provide a comparison and critical discussion of the implementation of these approaches in the next section.
The methods that we will review in this section were presented by Rogers et al. (2002) and Colvin and Maravelias (2008). Both methods model the decision-making process as scenario-based multi-stage stochastic programmes. In a scenario-based multi-stage stochastic programme, the scenarios correspond to realisations of the vector of random variables and the stages correspond to the times at which some uncertainty is resolved. The two approaches that we will discuss consider different types of uncertainty, hence the scenarios and the stages will be different in each approach. Both methods consider the pharmaceutical planning horizon and include information about the drugs within the portfolio. They consider the potential decisions that can be made in the planning horizon (e.g. continue or abandon development at each stage) and when the stochastic programme is solved, the optimal set of decisions is returned along with the value of the optimal solution.
Stochastic programming is beneficial when we are modelling a process that involves randomness as it allows us to take into account the uncertainty in the underlying process and considers the recourse action which should be taken given different observations of the random variable. The recourse action is the decision that should be made in order to compensate for the effect of what has just been observed. The recourse action relating to each uncertain observation will be contained in the solution to the stochastic programme. For example, if the observation was that the value of the drug dropped significantly over the most recent stage then the recourse action might be to abandon development. Alternatively, if the observation was that a study failed in the sense that the drug was shown to lack efficacy or be harmful then the recourse action might be to allocate resources to the development of a different drug.
Stochastic programming is able to consider all of the different potential outcomes of the uncertain component when finding the best set of decisions overall. For example, let us consider the problem of scheduling studies with uncertain outcomes. A stochastic programme would consider all of the potential combinations of study outcomes and return the set of initial studies to run along with those that should be run in the event of the different potential study outcomes. We could represent this using a set of Gantt charts, one for each trial outcome scenario, or using a decision tree.
If we do not want to use a stochastic programme, we could instead consider the problem in a deterministic setting using expectations. For example, in the scheduling problem, we could calculate the expected resource requirements and expected revenues using the probability of trial success. We could use these to build a deterministic mixed integer linear programme that does not consider the study outcomes. This deterministic model will assume that we are able to run all studies, regardless of the outcome of a preceding study. Solving the mixed integer linear programme would return a single schedule that maximises the expected revenue subject to constraints on the expected resource requirements under the assumption that we are able to run everything. If we do this, then we would need to build a new mixed integer linear programme after we observe a study outcome to reflect the new information gained and solve this programme for the next set of decisions. A deterministic model is not able to consider the different possible outcomes of the stochastic element, or the effect that these will have on the optimal decisions. Therefore, the recourse action that is considered in a stochastic programme is neglected when we model a stochastic problem deterministically. It is likely that the solution found using this approach would be suboptimal compared to the solution that is found when it is modelled using stochastic programming (Colvin and Maravelias 2008). When we consider the problem deterministically it cannot take into account the fact that it may be beneficial to wait and observe certain outcomes before making some decisions. A stochastic programme, however, accounts for all future outcomes and decisions when finding the optimal decisions. Colvin and Maravelias (2008) provide a comparison between using stochastic and deterministic models for a small portfolio management problem. The same information is used in both of the models, but the deterministic model uses the study success probabilities, resource requirements and revenues to find the expected resource requirements and revenues. When solved, the deterministic approach selects to run two studies simultaneously for which the future studies cannot be run together due to resource constraints. The stochastic method only chooses to run one of these studies and then waits until the study has been completed in order to make the decision for whether to run the second study or continue development of the first. This leads to a higher expected net present value of the solution where the net present value is the overall profit of the portfolio taking into account time discounting on the revenues and costs. Rogers et al. (2002) noted that the sequential nature of the investments made for each study in a drug development programme are comparable to a series of call options. A call option is the right but not the obligation to buy an asset by a given future date for a specified price (Kodukula and Papudesu 2006).

Real option valuation (ROV) approach
Real options are similar to financial options but, instead of the asset of interest being a financial asset, it is a real, non-financial asset. This means that, rather than having the right to buy the underlying asset by a future date, we instead have the right, but not the obligation, to take an action on the asset by a future date where the action could be to continue development, for example (Kodukula and Papudesu 2006).
When a pharmaceutical company invests in the current stage of a drug development programme this, in turn, gives them the option to invest in later stages, should the current stage be successful. The asset in this setting is the present value of the future cash flows of the product should it be commercialised. The cost of buying the real option is the cost of the current study and the predetermined price of the asset is the cost of future studies (Rogers et al. 2002). For example, after investing in a Phase I study, we have the chance to invest in a Phase II study and potentially a Phase III study if Phase I was successful enough to be carried forwards.
Drawing these parallels between real options and drug development programmes allows us to use methods for real option valuation to assign values to each of the drug development programmes (Rogers et al. 2002). The real option value of each of the drug development programmes, as presented by Rogers et al. (2002), takes into account many different aspects that affect the value of a development programme including the uncertainty in the trial outcomes, the potential market movements throughout the development process and the potential to abandon development.
After each of the drugs within the portfolio have been assigned a real option value (ROV), we will have an ordering of the most attractive programmes to run where the programme with the highest ROV is the most attractive and the programme with the lowest ROV is the least attractive. However, the pharmaceutical portfolio management problem is a real-world problem with finite resources. Therefore, resource constraints must also be included in the decision-making process. Hence, the selected portfolio will not always contain the most attractive programmes if they do not satisfy the constraints of the model.
In order to find the real option value of the drug development programmes, Rogers et al. (2002) uses a quadranomial pricing approach. This approach considers the market movements at discrete time intervals and assigns probabilities to the movement being either upward or downward. These movements correspond to the value of the drug increasing or decreasing, respectively. The market movements are represented by a multiplier which is calculated using the standard deviation of the value of drug i, σ i , which represents the beliefs that the team has about the volatility of the value of the drug in the marketplace. This value can be predicted by looking at historical data for similar products. The multipliers for upward and downward market movements are then given by Rogers et al. (2002) to be respectively, where ΔT is the discrete-time interval that the market movements are considered over. These potential movements over each time step, u i and d i , will be treated as constant through time, i.e., the multiplier used to calculate the value after the market movement will not depend on where we are in the planning horizon.
At the end of each study, ði; jÞ, there will be a set of possible values for the drug based on the different combinations of upward and downward market movements over the length of the study. For example, if we consider a study that lasts for two time intervals then there will be three possible values of the drug at the end of the study corresponding to: two downward market movements, one downward and one upward market movement or two upward market movements.
We will refer to these possible final values as the value scenarios and these are the scenarios that make up the scenario-based multi-stage stochastic programme presented by Rogers et al. (2002). We will denote the value scenario at the end of study j by k jþ1 where the set of all value scenarios at this point is given by 1; 2; :::; N ijþ1 È É . Using this notation, k j ¼ 1 will correspond to the worst value scenario (the scenario with the lowest value) and k j ¼ N ij will correspond to the best value scenario (the scenario with the highest value) at the beginning of study j. At the beginning of the first study for drug i in the planning horizon, we will only have one value scenario, N i1 ¼ 1, as we know the present value of the product at the initial time point.
An example of potential market movements may be seen in Figure 2. In the diagram in Figure 2, we have N i2 ¼ 3 where k 2 ¼ 1 represents the scenario with two downward market movements over the course of the study and final value d 2 V 0 and k 2 ¼ 3 represents the scenario with two upward movements over the study and final value u 2 V 0 . A possible path is highlighted, which leads to the scenario k 2 ¼ 2 and consists of an upward market movement followed by a downward market movement resulting in a final value of V 0 .
However, the ROV approach does not only consider the market movements but also the probability of study success, ϕ ij , and the potential to either continue or abandon development dependent on what is observed. Note that, in this setting, when we refer to study success we are referring to the situation where the drug may continue to further stages of development after the study in question is concluded. There are many different approaches for calculating the probability of study success, with many of these methods being based on the assurance (O'Hagan et al. 2005). The assurance considers aspects such as the planned sample size of the study along with prior beliefs on the treatment effect. The ROV approach does not require a particular definition or method of calculation for this input parameter, but we do recommend that the same method of calculation should be used across the different drug development programmes for comparability. This success probability is included in the calculation of the real option value of the drug, which is given below.
Under the discrete pricing approach for real options, the value of drug i at the beginning of study j in value scenario k j is given by (2) where for drug i and stage j: p ik j k jþ1 is the probability of moving from scenario k j to k jþ1 during stage j; ΔT is the length of the time interval that we consider the market movements over (Rogers et al. 2002). Then, as given in the nomenclature in Table 2, τ ij is the study duration, c ij is the study cost and ϕ ij is the study success probability. This is a recursive formula for which we begin at the expected reward received during launch. The reward received during launch in each scenario, k jJ i jþ1 ¼ 1; :::; N ijJ i jþ1 , is given by We may then iteratively work backwards from this reward to find the values at each time point in each value scenario for the drug i. Let us consider a simple example of a single drug with a single study with V 01 ¼ 50, c 11 ¼ 10, ϕ 11 ¼ 0:8, σ 1 ¼ 0:6, τ 11 ¼ 1 and r ¼ 0:05. We will consider two market movements per time step, ΔT ¼ 1=2. The number of value scenarios at the end of the study will be given by N 2 ¼ 3, and at the beginning we have N 1 ¼ 1. This is the same setting as was considered in Figure 2. We use the values associated with the final scenarios to find the values M 12k 2 , as in equation 3. This gives where u ¼ 1:53 and d ¼ 0:65, to two decimal places, using equation 1. The transition probabilities in this example are given by p 111 ¼ 0:33, p 112 ¼ 0:49 and p 113 ¼ 0:18. We will not go through the details of the transition probabilities here, however, and refer the reader to Rogers et al. (2002) for the full details. We can find the value of M 111 using equation 2.
The above form of M ijk j does not take into account the fact that we may choose to abandon development due to limited resources, for example, and then the value of the drug would go to 0 as it cannot add any value to the portfolio if it is not selected to be part of the portfolio. Therefore, the calculation of M ijk j must be reformulated to include the continue/abandon decision variable, y ijk j , which is equal to one when the study ði; jÞ is continued in scenario k j and zero otherwise. the set of all studies remaining for drug i j the index used to denote a specific study ði; jÞ the notation denoting the jth ordered study for drug i N ij the number of value scenarios at the beginning of ði; jÞ k j the index used to denote a particular value scenario at the beginning of stage j S the set of all possible trial outcome scenarios s the index used to denote a particular trial outcome scenario in the set S T the set of all time points considered in the planning horizon t the index to denote a specific time point in the planning horizon V 0i the present value of drug i at t ¼ 0 c ij the cost of study ði; jÞ ϕ ij the probability of drug i continuing to the next stage after study ði; jÞ τ ij the duration of study ði; jÞ σ i the market volatility of drug i γ D i the loss per time period due to shorter active patent life of drug i γ L i the loss per time period due to smaller market share of drug i B t the available budget at time t r the risk free interest rate The objective function of this optimisation programme, which will be defined later in the section, is to maximise the overall value of the portfolio. Hence, if M ijk j dropped below 0 then y ijk j would be set equal to 0 as it would not be profitable to continue with the study. Therefore, we can write the reformulation as This reformulation satisfies equation 2 whilst also allowing for abandon decisions, y ijk j ¼ 0, which can be related to limited resources. In our simple example, the value of the drug at the beginning of the study was given by M 111 ¼ 30:02. Since this value is positive and we only considered one drug in the example, we might expect to select to run this study, which would correspond to y 111 ¼ 1. However, if the available budget is less than the cost of this study, c 11 ¼ 10, we would not be able to run the study. This would lead to y 111 ¼ 0 and, from the reformulation given in equation 4, we would have M 111 ¼ 0 as we have not been able to run the study and therefore it has not added any value to the portfolio.
The value of M i11 is equal to the real option value (ROV) of drug i as there is only one value scenario, k 1 ¼ 1, at the starting point in the planning horizon, j ¼ 1. When M i11 ¼ 0 this means that the drug i has not been selected as part of the optimal portfolio.
The calculation of the values of M ijk j is core to both the decision-making process and the model formulation of the ROV approach. The uncertainty modelled in this approach is in the value of the drug and the values calculated above will be the values that we want to maximise in the decisionmaking process. For example, if M 111 > M 211 > 0 this means that drug i ¼ 1 is preferable to drug i ¼ 2 as drug i ¼ 1 has a higher ROV, which is given by M i11 . Furthermore, the objective value of the optimisation will be to maximise the sum of M i11 . This is because the drugs with the highest values of M i11 are deemed the most attractive under this approach and hence our aim will be to select the drugs which lead to the maximal value of P i M i11 subject to practical constraints. The formulation of the decision-making process is presented by Rogers et al. (2002) as a mixed integer linear programme, which means that the objective function and the constraints of the model are linear in terms of the decision variables. The decision variables for this formulation are the values, M ijk j , and the continue/abandon binary decision variables, y ijk j ; the values of these variables in the optimal solution are found by solving the mixed integer linear programme.
This means that the above formula for M ijk j should be added as a constraint to the mixed integer linear programme so that the values can be found by solving the programme. However, the form of this constraint as given above is not linear in terms of the decision variables, M ijk j and y ijk j . Therefore, in order to include this in the mixed integer linear programme, a reformulation is required to linearise this constraint. This reformulation requires upper bounds on the values of M ijk j to be found by solving a separate linear programme and adding these upper bounds as inputs to the final mixed integer linear programme in order to find the optimal solution. For full details of the linearisation, see Appendix A1.
The objective function for this formulation, as mentioned previously, is given by maximising the real option value (ROV) of the portfolio and may be written as This is subject to constraints including: the calculation of the values M ijk j ; drug precedence constraints, which ensure that future studies are not selected when a previous study was abandoned; value monotonicity constraints to represent the fact that if a no-go decision is made in a particular value scenario then the same decision must be made for all worse value scenarios; investment constraints, which ensure that the expected budget required at a particular time point, t, does not exceed the available budget, B t . For a full model formulation, see Appendix A1.
The investment constraints consider the expected budget required at each time point, t, and assume that each study begins as soon as possible and the cost of a study is incurred at the commencement of the study. These constraints are given by where w ijt is an indicator for if study ði; jÞ starts at time t. Here, the expectation is taken over the different value scenarios since the market value of the drug is the uncertainty that is modelled in this approach. The probability of trial success, ϕ ij , is not used directly in calculating the expected budget required, but it is included in the calculation of the values M ijk j . These values affect the go/no-go decisions, y ijk j , which are included in the budget constraint. If the value of M ijk j dropped below 0, for example, due to low success probabilities, then y ijk j would be set to 0 and this would reduce the expected budget required at the time that study ði; jÞ is initiated.
Solving the resulting mixed integer linear programme returns the set of values, M ijk j , and the continue/abandon decisions, y ijk j , in the optimal solution. In the ROV approach, each drug development programme may be thought of as a series of continue/abandon decisions that are dependent on the value scenario of the programme at a particular time point. This can be represented by a diagram such as the one seen in Figure 3.
The diagram in Figure 3 shows the decisions that should be made after each phase in a single drug development programme, assuming that the phase was successful, based on the value of the programme at that point. For example, at the end of Phase I if the trial was successful the decision should be to continue no matter what the value of the programme is at this point. At the end of Phase II, if it is observed that the value of the programme is less than or equal to d 8 V 0 then the team should abandon development, whereas if the value of the programme at this point is greater than d 8 V 0 then the team should continue development if Phase II is successful. These continue/abandon decisions are the recourse action in this approach.
Note that no scheduling is considered in this approach; the decision points are at the completion of each of the studies hence the next study is either started as soon as possible or not at all. This is because the method does not model the study outcomes, it models the uncertainty in the market value. In addition, if scheduling was considered and studies were allowed to be delayed then this would affect the ROV of the drugs in the portfolio. The focus of this approach is on portfolio selection rather than portfolio scheduling. The fact that a study may not be successful is considered by including the study success probabilities in the calculation of the values of M ijk j , which also affects the expected budget required. The full details of this formulation can be found in Appendix A1.
An alternative formulation for this approach was presented by Lo Nigro et al. (2014) which uses a continuous pricing approach to find the ROV of each of the drug development programmes rather than the binomial pricing approach. This reduces the size of the formulation in terms of the number of variables and number of constraints and decreases the computational time required to find an optimal solution.
Lo Nigro et al. (2014) also presented two extensions to the model: reinvestment of attained profits in the future and joint development with another partner company. However, this approach does not use stochastic programming, which means that it does not consider the recourse action that should be taken under different value scenarios, it simply selects the programmes to include at the initial time point.

Project scheduling (PS) approach
A second stochastic programming method was presented by Colvin and Maravelias (2008) which was compared in Colvin (2010) to the stochastic version of the resource constrained project scheduling problem. Hence, this approach considers the scheduling of the tasks, unlike the previous approach. In this approach, we consider a set of projects that correspond to drug development programmes. Each of these projects is made up of a series of tasks, which correspond to the clinical studies within the programme. In order to complete a project, all associated tasks must be completed successfully. Hence, in the setting of a pharmaceutical portfolio, this corresponds to a set of drug development programmes containing studies that must be successfully completed in order to complete the programme. Unlike the real option valuation approach which considers the value of the programmes to be stochastic, the stochasticity that is considered here is in the trial outcomes. The value of the programme, if successful, is not considered to be stochastic and is a linearly decreasing function of time in this approach. Colvin (2010) note that previous stochastic programming methods for resource allocation problems treated the timing of the tasks as fixed, as was seen in Section 2.1 when each trial was considered to start as early as possible and no later. The case that is considered by Colvin and Maravelias (2008) treats both the outcomes of the developmental tasks and the timing at which these outcomes are observed as uncertain. This is because the scheduling of the tasks is dependent on what is observed and the timings of the observations are dependent on the schedule. It is further noted by Colvin (2010) that this type of stochastic programme is difficult to solve and hence most approaches use the repeated solution of deterministic models rather than solving the full stochastic programme. Since this does not allow for the consideration of the "wait and see" approach this may lead to optimal deterministic solutions that are in fact suboptimal compared to the stochastic solution.
The problem is formulated by Colvin and Maravelias (2008) as a scenario-based multi-stage stochastic programme and requires similar information to that in the ROV approach. In this setting, however, the scenarios correspond to the different combinations of trial outcomes rather than the value of the programmes.
Let us consider the simple case where there is a series of studies that must be completed sequentially and failure in a study means that no further studies may be run. Then, we can represent the outcome of each drug development programme by the number of successful studies and hence each scenario will correspond to the number of successful studies in each drug development programme. Hence, the number of scenarios will be given by where J i is the set of studies of the drug i. For example, if we consider a portfolio containing three drugs, jIj ¼ 3, with the number of remaining studies being given by jJ 1 j ¼ 3 and jJ 2 j ¼ jJ 3 j ¼ 2 then the number of scenarios will be given by jSj ¼ 4 Â 3 Â 3 ¼ 36 and the set of scenarios can be denoted by where the ith element of each scenario, s 2 S, gives the number of successful studies for the drug i in scenario s. We can find the probability of each scenario, p s ð Þ, using the study success probabilities, ϕ ij . If we consider the scenario s ¼ 0; 0; 2 ð Þ from the above example, then the probability of this scenario would be given by The study success probabilities were also a required input parameter for the ROV approach and, as in the ROV approach, there are no assumptions on the modelling technique used to calculate these parameters but we would again recommend using the same approach across different programmes to avoid misleading results. The decision variable in this approach is given by X ijts , which is equal to one when the study ði; jÞ is chosen to be run at time t in scenario s. Note that, in this approach, the decision variable is also dependent on time due to the fact that this formulation considers scheduling.
The constraints of this formulation are largely similar to those in the ROV approach: each trial can be performed at most once; resource requirements at each time point must not exceed limits (note that budget may be considered as a resource in this approach if required); studies must be completed in the correct order; a study must not be performed if a previous study has failed. The resource/budget constraints in this approach are quite different, however, to their alternatives in the ROV approach. Firstly, multiple resources can be considered in this approach, rather than just the budget. The exact resources required are used, as opposed to the expected resources required, since the constraints are considered in each individual trial outcome scenario. In addition, this approach typically considers resources to be required throughout the study, rather than being incurred at the beginning of the study. This, however, can easily be modified to incur these costs at the beginning of the study, as in the ROV approach.
The main difference in the constraint set from the ROV approach is the addition of nonanticipativity constraints (NACs). In this approach, the scenarios correspond to the programme outcomes. Since the decision variables are dependent on the scenarios, we must ensure that the programme does not exploit the scenario information before the corresponding uncertainty has been resolvedit must not anticipate future outcomes. Including non-anticipativity constraints vastly increases the scale of the problem as can be seen in Table 3. Therefore, some possible model reductions were presented by Colvin and Maravelias (2008) and Colvin and Maravelias (2009) that consider, for example, the structure of the problem and the fact that expressing certain subsets of NACs ensures that all NACs are satisfied.
The objective function for the PS approach maximises the expected net present value (ENPV) and is given by maximise ENPV ¼ X s pðsÞ Rv s þ FRv s À Cst s f g where: Rv s is the total revenue generated in scenario s; FRv s is the future revenue in scenario s supposing ongoing programmes are completed; Cst s is the total development cost in scenario s. These values are functions that are dependent on the decisions, X ijts , made in each of the scenarios during the planning horizon. Both the revenue, Rv s , and future revenue, FRv s , are linearly decreasing in time, which encourages trials to be run earlier rather than being postponed until the end of the time frame. The components included in these functions that make them linearly decreasing include a reduction for reduced active patent life, γ D i , and a reduction for late completion hence reduced market share, γ L i . The revenue, Rv s , is used to capture the value of the programmes that have had all studies initiated during the planning horizon in scenario s. The future revenue, FRv s , however, aims to capture the value that has been gained by running additional studies for programmes that have not yet been completed in scenario s. This encourages the selection of studies, even if the programme cannot be completed during the planning horizon, which allows the model to consider further into the future and reflects the way that decisions would be made in the real world. The calculation of FRv s assumes that the first study that has not been initiated in each programme during the planning horizon is either initiated at the end of the planning horizon or upon the completion of the preceding study in the programme, whichever is later. It then assumes that all subsequent studies are initiated upon completion of their preceding study. For a full formulation, see Appendix A2.
Solving the stochastic programme returns not only the optimal portfolio but also the optimal schedule under each scenario, which can be represented using Gantt charts as seen in Figure 4. This way, after each trial outcome is observed the decision-makers can discard the set of schedules that have scenarios that do not match the observed outcome and use the schedules that do correspond to what has been observed so far.
In Figure 4, the effect of the NACs may be seen as (D1, PII) is selected to be run in both scenarios despite the fact that it will be unsuccessful in Scenario 26. Until the point where we observe the difference in these scenarios at the end of (D1, PII), the schedules are the same. We also see that the development of Drug 3 is delayed such that (D1, PII) and (D3, PII) complete at the same time. This is due to there not being enough available resources to run (D1, PII) and (D3, PIII) at the same time hence the method has chosen a schedule which means that (D3, PIII) would not need to be delayed should (D3, PII) be successful. As mentioned previously, delaying a study in the middle of the development programme results in a smaller revenue due to a reduced active patent life in this method. Furthermore, in this example the revenue reduction, or "penalty", incurred due to a shorter active patent life was larger than that which was incurred due to a reduced market share. This means that it is preferable in some circumstances to run a full programme later in the planning horizon without delays between studies than it is to start it early and delay certain studies within the programme. This is because the method assumes that the patent is filed at the beginning of the first study in the development programme.

Comparison
A summary of the key differences that will be discussed in this section can be found in Table 4. The examples discussed in this section were generated and solved using the JuMP package (Lubin and Dunning 2015) in Julia (Bezanson et al. 2017) with the Cbc solver.

Comparison of approaches
The main difference between the two approaches is the choice of which element is modelled as stochastic i.e. which uncertain element we observe and track over the decision-making process. The ROV approach (Rogers et al. 2002) considers the value of the drug development programmes to be stochastic and tracks the potential market movements over time. The standard deviation of the value of the drug is used in calculating the possible value scenarios. The potential failure of a study is only included via the probability of success which is used in the calculation of both the expected cost/ resource requirement at each stage, although indirectly, and the values associated with the drug development programmes. The PS approach (Colvin and Maravelias 2008) considers the trial outcomes to be stochastic and does not consider potential market movements or the variance of the value of the drug. The trial outcomes are tracked and future decisions depend on the trial outcomes. However, there is potential to include the variance in the value of the drug due to external competition into the PS approach via the specification of the objective value and different penalties for drugs that have higher competition in the market place.
The difference in the stochastic component modelled leads to three main differences in the approaches. These differences are: the type of recourse action considered, the values assigned to the programmes and the scheduling decisions made.
The recourse action is the action that should be taken to compensate for the effect of an observed outcome and is one of the decisions that is returned upon solving the programme. In the ROV approach, the recourse action relates to the value of the programme. If the value drops below a threshold then the recourse action would be to abandon development whereas for all values above the threshold development should continue, unless a study results in a failure. If a study does end in failure, the ROV approach does not offer any alternative solutions in the sense that it does not recommend investing in alternative programmes to replace the one that has just been abandoned. Therefore, if we wanted to find which other studies should be run given the study failure, the model would need to be updated given the new information and solved again. This can lead to suboptimal decisions compared to the stochastic version of the problem as the ROV approach cannot plan ahead I  I  II II II II   I  I  II II II III   I  I  II II II   Drug 1   Drug 2   Drug 3   I  I  II II II II  III III   I  I  II II II III III III   and consider what might happen if a study is unsuccessful. The PS approach, however, is able to do this. A limitation of the ROV approach is that it assumes that the company's utility is directly related to the ROV of the product, which may not necessarily be true. For example, if a company only had a single product undergoing development then even if the ROV was zero due to low success probabilities, they may still wish to continue development should the clinical studies be successful. Hence, the recourse action in the ROV approach may not always be realistic.
In the PS approach, the recourse action is more complex as it can involve other drug development programmes. If a study is successful, the recourse action could be to continue development immediately or to delay development if, for example, the available resources are limited at that time point. If a study is unsuccessful, the recourse action will consist of selecting which study should be run next for another drug development programme. This allows the commencement of studies for other drug development programmes to compensate for a given study failure.
The values assigned to the drug development programmes are different in the two approaches. In the ROV approach, the values are calculated using real option theory and they incorporate the market volatility of the drug, which was denoted by σ i for drug i. Values are assigned to each drug at the beginning of each stage and in each value scenario and were denoted by M ijk j . M i11 corresponds to the ROV of drug i and these values are found recursively by starting at the possible values at the conclusion of the programme, should all associated studies be successful, and working backwards. This results in a clear ordering in terms of the attractiveness of the drugs with the most attractive having the highest value of M i11 and the least attractive having the lowest value of M i11 . The value of σ i is used in the ROV approach to capture the key beliefs about the stochasticity in the value of a drug development programme and is described by Rogers et al. (2002) as the estimated annual standard deviation in the value of the drug after commercialisation. This parameter has a significant impact on the final value of the drug in the ROV approach; increasing the market volatility increases the value of the drug making it more likely to be included in the optimal portfolio. In the current model, it is assumed that the decision points are at study completion but we could also consider the decisions at an interim analysis in this framework. Considering the market information at additional points in the drug development process could add value and improve the decision-making process.
However, in the PS approach the market volatility is not considered at all. The only parameters that affect the value of the drug over time in the PS approach are the reductions in revenue for shorter active patent life and smaller market share and a time-discounting factor. This can lead to discrepancies in the attractiveness of the drugs under the two methods. Therefore, a direct comparison for which approach selects the best portfolio is not simple as the portfolio that is deemed the best is dependent on how the values are assigned. In the PS approach, the values that are calculated for the revenue, future revenue and costs are dependent upon the scenario, which, in the PS approach, refers to the trial outcomes.
Solving the stochastic programme in the PS approach will provide the optimal order in which the studies should be run, but this order will not necessarily correspond to the value order of the individual programmes. For example, the first study to be run will not necessarily belong to the most valuable individual programme. This is because the PS approach considers multiple programmes that are competing for resources and the optimal ordering of the studies will depend on more than the individual programme values. Therefore, it may be preferable, for example, to run two programmes simultaneously which have a higher combined value than a third highly valuable programme with resource requirements equal to the total requirements of the other two studies. If we want to learn about the value of the individual programmes, we may consider each drug separately and assume that all studies begin as soon as possible in order to find the revenue and associated costs. Then, the drug with the highest profit could be considered as the most valuable and the drug with the lowest profit could be considered as the least valuable, if we assume that utility is directly related to profit, providing an ordering in terms of the value of the individual programmes. This value order may be different to the optimal running order due to the reasons discussed above.
The final main difference due to the uncertainty considered is scheduling. Since the ROV approach focusses on the value uncertainty rather than the trial uncertainty, scheduling is not considered. It is assumed that each study begins as soon as possible or not at all; the choice to postpone a study is not available in the model formulation. One of the downfalls of this is that the ROV approach cannot take into account the potential benefit of waiting to observe certain outcomes in advance of making decisions. Instead, when a trial is concluded the model may be adjusted to reflect the current state of the portfolio in order to select which studies should be run. This is because the ROV approach focusses on the uncertainty in the market value rather than the trial outcomes. Colvin and Maravelias (2008) provide a simple example that illustrates the differences between a model that considers the different potential study outcomes and one that uses expectations as an attempt to capture the uncertainty in trial outcomes. That is, they compare a stochastic programme that models trial outcomes with its deterministic alternative. They show how this may lead to suboptimal decisions compared to when we allow studies to be postponed until certain outcomes are observed. The reason that the stochastic version finds a solution with a higher expected revenue in this example is that the deterministic model selects two drugs to run at the initial time point for which the second studies may not be run at the same time due to limited resources. The stochastic model, however, takes this into consideration and selects a different pair of studies to be run at the same time which can be completed simultaneously. Since a penalty is incurred when studies are delayed this leads to the stochastic model achieving a higher expected revenue.
Furthermore, the resource constraints in the ROV approach are all calculated in the expected sense due to the fact that the scenarios in the ROV approach do not contain information on trial success. This means that if the expected resource usage for a portfolio at a time point exceeds the resource limit then the development of some of the drugs included in this portfolio will need to be abandoned since they cannot be postponed. Using the expected resource utilisation in the constraints may even allow the ROV approach to select studies that have costs exceeding the available budget; we will see an example of this in the next section.
Conversely, modelling the trial outcomes and scheduling is one of the key features of the PS approach. The PS approach takes into account the fact that it may be beneficial to wait and observe certain outcomes before making some decisions. It also allows studies to be postponed if there are not enough resources/budget available to run the study immediately. This means that the PS approach is often able to find schedules that facilitate more development programmes being run than in the ROV solution. Also, the PS approach is able to calculate resource requirements exactly, rather than the expected requirements, since the information about trial outcomes is included in the scenarios of the PS approach. This means that, under the decisions suggested by the PS approach, there will never be a case where the resources required exceed the resources available, which can happen when we only calculate the expected resources required.
It should also be noted that, in both of the approaches, the resources/budget are treated as fixed at each time point and any resources that are not used in one time point do not carry over to be used in the next time point. This, however, could be added as an extension to the methods if it was required by a company to make the budget allocation more realistic. Another potential modification could involve neglecting to include budget constraints entirely and instead including constraints on different resource types, e.g. staffing resources, and considering the trial costs within the objective value alone. Note that this is actually how the formulation of the PS approach is set up. The PS formulation includes resource constraints, which we have taken to be the budget in order to compare it to the ROV approach, and considers the trial costs within the objective function. The ROV approach assumes that the cost of a study is incurred at the initiation of the study, whereas the PS approach is able to incur these costs, or resource requirements, either at the beginning of the study or throughout the study. In the comparison that follows, we have modified the resource constraint in the PS approach so that the study costs are incurred at the beginning of the study in order to provide a fair comparison to the ROV approach.
In the PS approach, some programmes may not be completed in the time frame that is considered in the model. This is due to the fact that studies are allowed to begin at any time, subject to the previous study having been completed and resulting in a success. In order to compensate for the fact that some programmes may not complete in the planning horizon, the future revenue is considered, which is calculated by assuming that the ongoing studies are completed as quickly as possible. Considering the future revenue is beneficial as it encourages studies to be run where possible, even if the revenue will not be realised in the planning horizon, which is the type of forward planning that we would expect to see in real-life decisions. However, it should be noted that the future revenue is not able to capture the information in the same way as extending the planning horizon would. In fact, if we change the length of the planning horizon in the PS approach, even by a single time step, the set of optimal schedules may change. Hence, when using this method a sensitivity analysis may be required to study the effect of different planning horizon lengths on the set of optimal schedules for a particular portfolio.
In terms of the model formulation, there are three main things to discuss: flexibility, complexity and size.
Flexibility will often be desirable so that the model can be adjusted to accurately represent the portfolio in question. A company may also wish to add constraints that reflect their decision-making process, e.g. there may be two drugs that they would only want to develop at most one of. Due to the fact that the ROV approach is based on the way that the values of the drugs are calculated, there is little flexibility in the choice of the objective function. In the PS approach, however, there is a lot more flexibility in terms of the objective function. If we chose, we could modify the PS approach to include an objective function that maximises the ROV of the portfolio. Adding further constraints is relatively straightforward in both approaches, provided that some consideration is given to what the scenarios refer to in each of the approaches and what this means in terms of the constraints.
In terms of the complexity of the models, the difficulty arises in different areas. For the ROV approach, the complexity arises in the calculation of the value of drug i at the beginning of study j in value scenario k j , which was denoted by M ijk j . In this approach, a separate linear programme must be solved in order to find upper bounds on the value of M ijk j , which are then used as some of the input parameters of the second mixed integer linear programme that returns the optimal portfolio. Essentially, the upper bounds that we find by solving the first programme correspond to the values of the drugs if we did not include any resource constraints. As was mentioned in Section 2.1, including the decision to continue/abandon development in the calculation of M ijk j results in a nonlinear constraint. The linearisation of this constraint also adds to the complexity of the model. In the PS approach, the complexity arises from the fact that we require non-anticipativity constraints for this formulation. While the interpretation of these constraints is relatively straightforward, the formulation of them is less so.
This leads us to the final consideration to make, which is the size of the model formulation. The non-anticipativity constraints in the PS approach vastly increase the number of constraints in the formulation of the PS approach compared to the ROV approach. Also, the number of variables required for the PS approach typically exceeds the number in the ROV approach. This is due to the number of scenarios typically being larger for the PS approach. This will be illustrated in the next section.

Results
We implemented both of the approaches in Julia for an illustrative example portfolio with parameters given in Table 5. This example is used to highlight our main findings regarding the use of the two methods and the differences between them. The optimal portfolios were considered for three different budgets: 200, 250 and 300 per time point. That is, B t ¼ 200, B t ¼ 250 and B t ¼ 300 for all time points, t, in the planning horizon. A summary of the comparison is provided in Table 6. Gantt charts showing some of the schedules selected under the PS approach with a budget of 200 per time point, B t ¼ 200, are shown in Figure 5. The aspects for which comparative results are provided include: speed to obtain the solution, size of the problem in terms of the variables and constraints, selected portfolio and ordering of the most attractive drug development programmes.
Note that we are not assuming an underlying truth for the illustrative example. In this section, our aim is to illustrate the way the methods work and to highlight any differences between the methods and the results that they may lead to through the use of our illustrative example. Therefore, we will not draw conclusions on which method has performed better. It should be further noted that, even if an underlying truth was assumed, we would still not necessarily be able to conclude which method performs better. This is because the methods assign values to programmes differently and so the most valuable programme under the ROV approach may be different to the most valuable programme under the PS approach.
One of the first things that is apparent in Table 6 is the difference in the size of the formulations. Despite the fact that the ROV approach requires two programmes to be solved (the first to find upper bounds on the values in the second and the second to find the optimal portfolio), it still has far less variables and constraints than the PS approach. This is because the scheduling in the PS approach comes at a high computational burden due to the inclusion of the non-anticipativity constraints, which ensure that the optimisation does not use information regarding trial outcomes before they have been revealed. The pattern observed here will be the same for most sets of input Table 5. Parameter values used for the comparison in (a) both approaches, (b) the ROV approach and (c) the PS approach. A planning horizon of jTj ¼ 6 was used in both approaches. parameters. The main thing that could increase the size of the ROV formulation past the size of the PS formulation would be if we applied the market movements over much smaller periods as this would increase the number of value scenarios to consider. However, it is unlikely that a user would require a level of granularity that would be small enough to cause the ROV formulation size to exceed the PS formulation size. We see a similar ordering in the time taken as we saw in the numbers of variables and constraints and this is typical of what we would observe for most example portfolios. For this example containing five drugs, all of the programmes were solved within a reasonable time. This will not necessarily scale as we increase the portfolio size, unfortunately, as it is noted by Colvin and Maravelias (2010) that without model reductions this approach cannot solve the problem for portfolios containing more than six drugs. There have been heuristics presented for the PS approach that aim to tackle this problem (Christian and Cremaschi 2015). The most promising heuristic was presented by Christian and Cremaschi (2015) and it decomposes the problem into a series of smaller knapsack problems. While the solution of the knapsack decomposition algorithm will not always match the optimal solution of the PS approach, the reported results are within 3% of the optimal solution and the solution is found much more quickly than in the full formulation of the PS approach and is also able to be found for much larger portfolios.
In Table 6, we see that, although the two methods assign values to drugs differently, the ordering of the drugs in terms of the value of the associated programme is similar in both. The biggest difference is that Drug 2 is deemed the most attractive in the PS approach but only the third most attractive in the ROV approach. This highlights the fact that the different methods of assigning values to the drug development programmes can lead to different decisions being made. Although Drug 2 has the second highest present value, it has the second lowest volatility causing it to be attractive under the PS approach but not as attractive in the ROV approach. This is because increasing the volatility of a drug also increases the ROV of the drug.
It should be noted that even if the portfolios selected in each approach were similar, or even the same, the objective value under each method would still be quite different due to the different methods of valuation. Also, we are not able to calculate the objective function of one model in a meaningful way for the decisions made under the other model. The objective function in the ROV approach is calculated under the assumption that trials start as soon as possible, which will not necessarily be the case with the decisions made in the PS approach, as we can see in Figure 5. The ROV objective function does not consider any reductions in revenue given the late completion of a programme, and the market movements are considered over the minimum time taken to complete the programme. Therefore, the ROV objective function cannot be calculated appropriately for the decisions made under the PS approach. Similarly, looking at the decisions made in the ROV approach in the PS framework will not lead to a meaningful objective value. This is because scheduling was not an option in the ROV framework and there will be "better" scheduling decisions available. Hence, the nonscheduled decisions will be sub-optimal in terms of the PS objective function.  I  II II III   I  II II   II II III III   II II   III III   Drug 1   Drug 2   Drug 3 Drug 4 Drug 5 Scenario 64 = (2,1,2,0,1) Time   I   I  II II III   II II III III   II II III III   III III   Drug 1   Drug 2   Drug 3 Drug 4 Drug 5 Scenario 176 = (0,3,1,1,1) Rather than comparing the objective values across the approaches, it is more interesting to look at how the objective values change in each approach given additional budget/resources. For example, when we increase the budget from 200 to 250, the objective value of the ROV approach remains the same, but the objective value of the PS approach increases due to the fact that it is able to find a more profitable schedule. This leads us on to the next part of the comparison.
One of the most significant differences in terms of the portfolio that we see in Table 6 is that the PS approach is able to select more drugs than the ROV approach. This is due to the fact that it is able to schedule when the studies should be run and hence allows delays in the commencement of a study, which often facilitates the selection of more drugs as it has done here for our example portfolio. In Table 6, we see that when we have budgets of 200 and 250 for each time period, the ROV approach is unable to select Drug 5 despite the fact that it has the second-highest ROV. The PS approach, however, is able to include this drug in its portfolio under all three budgets. The PS approach is, in fact, able to include all five drugs in its optimal portfolio under all three budgets. We see that for a budget of 300 both methods are able to select their most attractive drugs to include in their optimal portfolio.
If we consider a much smaller budget than those considered in the table, B t ¼ 100 "t, the ROV approach selects Drugs 1 and 4. However, Drug 4 has a cost of 180 for its second study. This highlights the way that taking the expected value for resource constraints may not always be realistic. Here, the cost of a study is almost double the available budget yet it is still selected. While this might not always be an issue, it could certainly be a problem in the real world if a company were to run into the situation where the extra budget could not be found and hence the investment in previous studies would be wasted.
In Figure 5, we see that, under the assumption that the costs are incurred at the commencement of a trial, (D5, PIII) is only selected to be run in the second time period due to budget constraints not allowing it to start alongside any of the other studies. This also applies to (D4, PIII). This leads to a sparse amount of studies being run in the first two time steps and a much denser schedule later on. This may not be realistic in terms of what would be preferred by a company. If this situation were to arise, a company may add constraints to the model which reflect their preferences and explore the effect of these preferences on the overall scheduling and optimal value. For example, these constraints might be to run certain studies straight away or to ensure that the pipeline is not left idle when there are available studies to be run.
We also see that the two most valuable programmes under the PS approach, Drugs 2 and 4, only have their first study initiated at the third time point. This shows how, as we discussed in the previous section, the optimal order to initiate the studies may not reflect the most valuable programmes due to the complexities added by the consideration of multiple programmes, resource constraints and the effect of early decisions on later decisions. Since (D4, PIII) can only be initiated at the same time as (D1, PI) due to budget constraints, its preceding study, (D4, PII) is only initiated later in the planning horizon to allow for the initiation of other studies. If we assume that the optimal time to initiate (D4, PIII) is at t ¼ 5 then there is no reason to start (D4, PII) until t ¼ 3 as the PS approach discounts study costs throughout the planning horizon to encourage studies to be selected. This means that leaving (D4, PII) to start as late as possible without affecting the initiation of (D4, PIII) will increase the ENPV.
Another component that will affect the optimal decisions are the penalties incurred due to shorter active patent life and smaller market share. It is assumed that the patent life of a drug starts at the initiation of Phase I, therefore the penalties associated to shorter active patent life are only incurred after Phase I is initiated. It is therefore beneficial to reduce the delays between phases. We see the effect of this in Figure 5 for Drugs 1 and 2 as both programmes are completed without any delays, despite the fact that the earlier phases could be initiated sooner but with delays between the later phases as this would lead to a reduction in the ENPV.

Conclusion
We have provided a comparison and discussion of two stochastic programming approaches for pharmaceutical portfolio management. We used an illustrative example to highlight our findings and the differences between the two approaches.
The first approach (Rogers et al. 2002) uses real option theory from the financial setting and focusses on the uncertainty in the value of a drug. While this approach has a reasonably sized formulation and is quick to solve, it lacks flexibility and there is some discussion to be had in terms of the relevance of the market volatility, which drives this approach, in the pharmaceutical setting. Also, this approach assumed that every study would start as soon as possible and it is not able to schedule tasks which in turn leads to drugs being omitted from the optimal portfolio that could, in fact, be included if scheduling were considered.
The second approach (Colvin and Maravelias 2008) provides a modification of the stochastic version of the resource constrained project scheduling problem in order to find the optimal portfolio and the optimal schedule under each scenario. While this approach was preferable in terms of scheduling and flexibility, this came at a high computational burden.
Extensions have been presented for both of these methods. For the ROV approach, Rogers et al. (2005) presented an extension that considers partnership opportunities. This extension considers both the optimal timing of the partnership and the best investment policy. Enea and Lo Nigro (2011) noted the perceived complexity of the method presented by Rogers et al. (2002) and presented a more user-friendly simplification, which uses a different option pricing approach and finds the optimal portfolio under this pricing approach. However, this approach does not track the continue/ abandon decisions over time. This was then extended by Lo Nigro et al. (2014) to consider partnership opportunities. Colvin and Maravelias (2009) presented methods to tackle the problem of the size of the model formulation for the original PS approach presented by Colvin and Maravelias (2008). This was then extended further to consider solution methods and a branch and cut algorithm was presented by Colvin and Maravelias (2010). Then, Colvin and Maravelias (2011) presented several extensions that include: resource planning decisions such as expansion and outsourcing; task interdependencies in terms of uncertainty, resources and revenues; risk management approaches.
Although the two approaches use different methods for valuation, neither of them capture the effect that treatment efficacy and safety can have on the revenue of a new drug. Treatment safety and efficacy estimates will often be driving factors in our estimate of the revenue and these safety and efficacy estimates will evolve throughout the drug development programme. The consideration of these estimates in the valuation method could make it more appropriate and realistic in the setting of drug development. Also, both methods are reliant upon point estimates for the revenue generated upon successful programme completion and they are not able to consider our prior uncertainty in this estimate. We think that the ability to do so would add further benefit to the methods as the prior estimates of the revenue are not always representative of what is actually seen.
Both of these methods have different benefits and focuses and therefore the best approach will depend on what the decision-maker deems more importantmodelling trial uncertainty or modelling value uncertainty. We believe that the flexibility offered in the project scheduling approach offers an advantage over the real option approach as, if one chose to, the valuation of the drugs in this approach could be modified to consider the market volatility. We also believe that, in the setting of clinical trials, the impact of the trial outcomes is more significant than the market volatility. The pharmaceutical industry has high development failure rates compared to many other industries. Therefore, the stochastic modelling of the trial outcomes may be more important than market volatility in this setting. This may not be true in other industries where development is much less high risk, and so other market forces may dominate. Furthermore, the ROV approach is centred on the estimates of the net present value of the future cash flows of the drug and the estimates of the market volatilities. If these estimates are not accurate or representative of the drug then the focus of this approach is wasted.
In conclusion, we believe that the advantages of the PS approach for tackling the true portfolio management problem are many and, with the flexibility in this approach, further adjustments to make the approach match the individual requirements of a company can be added easily making this approach very useful and applicable for real-world problems.