<title>Background</title> <p>With growing interest in systems biology, mathematical models have been widely used to study metabolic networks, gene regulatory networks and cell signaling pathways [##REF##12424381##1##, ####REF##15364960##2##, ##REF##16025103##3##, ##REF##17024084##4##, ##REF##16980977##5##, ##REF##17625511##6####17625511##6##]. These mathematical models are used to reproduce experimental data and predict unobserved behaviors of the system. However, many sources of uncertainty including errors, inconsistency and noise of experimental data, absence of parameter information, incomplete representation of underlying process details, and poor understanding of the biological mechanisms impose a limit on model confidence. Furthermore, intrinsic variability or noise of the system such as the occurrence of stochastic events also affects the output of the model. Therefore, it is important not only to understand the dynamical properties of the model with particular parameter values, but also to further investigate the effect of their perturbations on the system [##REF##15710397##7##]. Sensitivity analysis is a powerful approach for investigating which parameters in a model have the strongest effect on overall behavior. In addition to identifying key parameters in a model, sensitivity analysis is valuable in pinpointing parameters, which should be in the focus of experimental perturbation [##REF##17060902##8##].</p> <p>Sensitivity analysis has been widely utilized for the systems biology research [##REF##15364960##2##,##REF##15710397##7##,##UREF##0##9##, ####REF##9078251##10##, ##REF##12242336##11##, ##UREF##1##12##, ##REF##15454422##13##, ##REF##16623463##14##, ##REF##16986622##15##, ##REF##17616983##16####17616983##16##]. However, it is time consuming for researchers to apply different algorithms to their specific models. In order to automate sensitivity analysis for different types of systems biology models, we developed a free software tool named SBML-SAT: a systems biology markup language (SBML) based sensitivity analysis tool. SBML is a language developed by the systems biology community to represent and exchange models of biochemical reaction networks [##REF##14641091##17##]. SBML is being used by a large group of software developers and researchers. More than 120 software systems have so far been developed for supporting SBML <ext-link ext-link-type="uri" xlink:href="http://sbml.org"/>. Although a few existing software systems such as COPASI [##REF##17032683##18##] and SBToolbox [##REF##16317076##19##] incorporate local sensitivity analysis, a powerful, flexible and broadly applicable sensitivity analysis platform is still lacking. In particular, some important features missing from the existing software systems are described below.</p> <p>Firstly, some mathematical models of biological system include discontinuous events, such as the division of cells, removal of biological signal at a specific time or blocking protein synthesis during an experiment. Most existing SBML supported software systems (except for SBML-PET [##REF##16926221##20##], MathSBML [##REF##15087311##21##], SBTOOLBOX2 [##REF##16317076##19##], etc.) do not support models involving such discontinuous events. The broad applicability of these software systems is thus limited.</p> <p>Secondly, none of the existing SBML software packages allows for global sensitivity analysis. A few of the existing software systems can run local sensitivity analysis which introduces a small perturbation of one parameter for each simulation. Therefore, local sensitivity analysis investigates sensitivity of the model outputs with respect to a particular point in the parameter space. However, a single "true" point of parameter set may not occur in nature. It is likely that biological parameters such as rate constants and initial concentrations are variable in a large range depending on the specific cell types and cellular environments. For this reason, a global sensitivity analysis is valuable to explore sensitivities of model outputs to simultaneous variations of all the parameters over a large range and examine possible non-linear effects of the parameters as well as their interactions [##REF##15710397##7##].</p> <p>Thirdly, the results of sensitivity analysis correspond to specific model outputs. The specific model outputs of interest usually vary from case to case. In some cases, users may want to study the integrated or maximum response of certain species, while in other cases interest may be placed on particular time dependent or steady state responses of the system. Thus, a good sensitivity analysis software platform should provide various options for specifying model outputs.</p> <p>Here, we present the software system SBML-SAT that encompasses all of the above capabilities. It is worth pointing out that the purpose of this paper is not to explain the technical details of the software (described in the manual file) or the published algorithms, nor to present any particular biological findings. Rather, we provide an overview of the software, its validation with a variety of mathematical models for biological systems and demonstrate its broad applicability.</p>
<title>Results</title> <p>In this section, we will demonstrate the functions and broad applicability of SBML-SAT using a variety of mathematical models for the biological systems. All of the models presented here are pre-encoded in SBML format and most of them are taken from the BioModels Database [##REF##16381960##39##]. At the start of each subsection, a brief description of the instructions to operate SBML-SAT for each function are provided to enable the reader to further envision the interaction with the software tool and facilitate its use.</p> <title>Simulation of SBML models</title> <p>To simulate a pre-constructed SBML model, the user loads the SBML model, sets the time course for simulation, and selects "Run Simulation".</p> <p>SBML-SAT provides an easy way to run a simulation and visualize the simulation results for SBML models. The output screen for SBML-SAT model simulation is shown in Figure ##FIG##1##2##. In order to test the wide applicability of SBML-SAT, we ran simulations for a variety of models from the BioModels Database, which include biophysical models, signaling pathways, gene expression and metabolic networks. The results shown in Figure ##FIG##2##3## demonstrates that SBML-SAT appropriately simulates both continuous SBML models (signaling pathway, gene expression and metabolic models), as well as those with discontinuous events (cell cycle model) with different degrees of complexity and nonlinearity.</p> <title>Sample local sensitivity analysis</title> <p>To conduct the local sensitivity analysis, the user</p> <p>• loads the SBML model,</p> <p>• sets the time course,</p> <p>• chooses the parameter(s),</p> <p>• defines the perturbation coefficient, and</p> <p>• selects the objects (ODE variables or reaction rates) and the model output operation for the analysis,</p> <p>• select the appropriate analysis approach to run.</p> <p>The result of a SBML-SAT normalized local sensitivity analysis on the MAPK cascade model (BioModels ID: BIOMD0000000010) is shown in Figure ##FIG##3##4##. For this analysis, the objects of the sensitivity analysis were the state variables associated with the various phosphorylated forms of the MAPK cascade elements and the model output analyzed were the integrated responses. The parameters perturbed were the initial concentrations of each form with the default perturbation coefficient. These results indicate that the integrated response of the MAPK concentration was the most sensitive to the initial concentration of MAPK.</p> <title>Sample global sensitivity analysis</title> <p>The user interface and operation for performing global sensitivity analyses is similar to that for the local sensitivity analysis: the user specifies the time course, object(s) and parameter(s) as well as the model output(s) for global sensitivity analysis. In addition, the user chooses the global sensitivity analysis method, and sets the variation range of the parameter values. The user must also define the number of Monte-Carlo simulations to be performed to base the analysis upon: this is highly dependent upon the nature of the model, the number of parameters (factors) to be analyzed, and the size of the parameter space (factor levels). The user needs to try different "Number of Simulations". If the analysis results are not significantly changed by the increasing of "Number of Simulations", then the results are assumed to be reliable and accurate enough. Once all these settings are done, SBML-SAT is ready to perform the specified global sensitivity analysis. The time required to complete the analysis varies from several minutes to several hours. It depends on the complexity of the model and the number of Monte-Carlo simulations.</p> <p>We use a model of the receptor trafficking network to demonstrate how to implement global sensitivity, steady state and robustness analyses in SBML-SAT. The general model of receptor trafficking networks is composed of the de novo production of surface receptor, ligand-receptor interaction, internalization, recycling and degradation of both empty and occupied receptors (Figure ##FIG##4##5##). The symbols of the parameters in the model and their corresponding biological processes are listed in Table ##TAB##0##1##. Detailed information about this model is available in our previous work [##REF##17825822##24##,##REF##18546478##40##].</p> <p>The results of global sensitivity analysis of the integrated response of the state variables in the receptor trafficking model using all four different methods are shown in Figure ##FIG##5##6A–D##. The exact values of the sensitivity indices obtained by different methods are not comparable because of their distinct definitions. However, the ranks or relative importance of the parameters to the model output are similar among different global sensitivity analysis methods. The results suggest that the rates of ligand-receptor complexes formation (parameters <italic>k</italic><sub>2 </sub>and <italic>k</italic><sub>3</sub>) are very important to the integrated response of ligand concentration (<italic>L</italic>). In contrast, the integrated response of ligand-receptor complexes (<italic>LRs </italic>and <italic>LRi</italic>) are shown to be mainly affected by the rates of the internalization, recycling and dephosphorylation of the occupied receptors (parameters <italic>k</italic><sub>6</sub>, <italic>k</italic><sub>7 </sub>and <italic>k</italic><sub>8</sub>). The MPSA global sensitivity analysis result of the time dependent response (Figure ##FIG##5##6E##) indicates that <italic>k</italic><sub>2 </sub>is the key regulator for <italic>R</italic><sub><italic>s </italic></sub>behavior at the early stage (before 20 minutes), but its effect is reduced significantly at a later stage. Upon further analysis, the MPSA global sensitivity analysis of the steady state response (Figure ##FIG##5##6F##) shows that the steady state of <italic>R</italic><sub><italic>s </italic></sub>is not very sensitive to <italic>k</italic><sub>2</sub>.</p> <title>Sample steady state analysis</title> <p>A steady state analysis of a user loaded SBML model simply requires to select such analysis from the icons or pull down menu. SBML-SAT initially tries to algebraically solve the system of ODEs for equilibrium solutions. If that fails, the model is simulated over an extended time period to approach the stable steady state related to the initial conditions provided.</p> <p>The results of the steady state analysis of the model of receptor trafficking network are provided in Figure ##FIG##6##7##. At steady state, all the ligand molecules are taken up by the receptors and eventually degraded, while the internalized and surface receptors that remain unbounded by ligand reach non-zero equilibriums. This information helps to interpret the steady-state global sensitivity analysis results shown in Figure ##FIG##5##6F##.</p> <title>Sample robustness analysis</title> <p>To conduct a robustness analysis, the user</p> <p>• loads the SBML model,</p> <p>• sets the time course,</p> <p>• chooses the parameter(s),</p> <p>• defines the variation range of the parameter(s), and</p> <p>• selects the objects (ODE variables or reaction rates) and the model output operation for the analysis and eventually</p> <p>• runs the analysis</p> <p>Figure ##FIG##7##8## shows the result of robustness analysis of the receptor trafficking model. The steady state concentrations of different forms of receptors are plotted as a function of the total parameter variation (TPV) and the quantitative robustness metric is provided in the subplot title. The results indicate that the steady state concentrations of unbound receptors are less robust to parameter perturbations than the internalized unbound receptor concentration. Not surprisingly, the ligand-bound receptors' concentrations are very robust to the parameter perturbations since their steady state solutions are zero.</p>
<title>Discussion and conclusion</title> <p>Currently, a SBML model editor module is not available in SBML-SAT. Fortunately, many existing free software packages such as CellDesigner, SBMLeditor and COPASI, share a common functionality for constructing and editing SBML models. The users can easily generate their models with these free software packages and then run a variety of analyses in SBML-SAT by importing the model in SBML format. Although SBML-SAT doesn't provide a SBML editor for model construction, it provides a convenient track for modifying the initial conditions of the state variables and parameter values in the model. Moreover, delay differential equation models are not supported in SBML-SAT, as in most existing software systems. In practice, delay differential equations can be solved in approximation by converting to ordinary differential equations using the linear chain transformation [##UREF##11##41##]. Therefore, users can still apply SBML-SAT to their delay differential equation models.</p> <p>There are more than 120 SBML-supporting software packages for kinetic analysis of biological models and this number continues to grow. However, a powerful, flexible and broadly applicable software package for global sensitivity analysis and robustness analysis has been lacking. In reality, it is difficult and time consuming to implement different sensitivity analysis algorithms especially the global sensitivity analysis methods. Here we introduced, a free Matlab-based software tool, SBML-SAT, for both local and global sensitivity analysis of SBML models. With a user-friendly graphic interface, this tool allows the user to run sensitivity analysis, steady state analysis and robustness analysis for a variety of model outputs. Models involving events are also supported in SBML-SAT. Furthermore, created in Matlab, the most popular software used in the community of systems biology [##REF##17420739##42##], SBML-SAT has a good cross-compatibility with different platforms. Taken all together, we can expect that SBML-SAT will have a broad applicability among systems biologists.</p>
<title>Discussion and conclusion</title> <p>Currently, a SBML model editor module is not available in SBML-SAT. Fortunately, many existing free software packages such as CellDesigner, SBMLeditor and COPASI, share a common functionality for constructing and editing SBML models. The users can easily generate their models with these free software packages and then run a variety of analyses in SBML-SAT by importing the model in SBML format. Although SBML-SAT doesn't provide a SBML editor for model construction, it provides a convenient track for modifying the initial conditions of the state variables and parameter values in the model. Moreover, delay differential equation models are not supported in SBML-SAT, as in most existing software systems. In practice, delay differential equations can be solved in approximation by converting to ordinary differential equations using the linear chain transformation [##UREF##11##41##]. Therefore, users can still apply SBML-SAT to their delay differential equation models.</p> <p>There are more than 120 SBML-supporting software packages for kinetic analysis of biological models and this number continues to grow. However, a powerful, flexible and broadly applicable software package for global sensitivity analysis and robustness analysis has been lacking. In reality, it is difficult and time consuming to implement different sensitivity analysis algorithms especially the global sensitivity analysis methods. Here we introduced, a free Matlab-based software tool, SBML-SAT, for both local and global sensitivity analysis of SBML models. With a user-friendly graphic interface, this tool allows the user to run sensitivity analysis, steady state analysis and robustness analysis for a variety of model outputs. Models involving events are also supported in SBML-SAT. Furthermore, created in Matlab, the most popular software used in the community of systems biology [##REF##17420739##42##], SBML-SAT has a good cross-compatibility with different platforms. Taken all together, we can expect that SBML-SAT will have a broad applicability among systems biologists.</p>
<p>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0"/>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</p> <title>Background</title> <p>It has long been recognized that sensitivity analysis plays a key role in modeling and analyzing cellular and biochemical processes. Systems biology markup language (SBML) has become a well-known platform for coding and sharing mathematical models of such processes. However, current SBML compatible software tools are limited in their ability to perform global sensitivity analyses of these models.</p> <title>Results</title> <p>This work introduces a freely downloadable, software package, SBML-SAT, which implements algorithms for simulation, steady state analysis, robustness analysis and local and global sensitivity analysis for SBML models. This software tool extends current capabilities through its execution of global sensitivity analyses using multi-parametric sensitivity analysis, partial rank correlation coefficient, SOBOL's method, and weighted average of local sensitivity analyses in addition to its ability to handle systems with discontinuous events and intuitive graphical user interface.</p> <title>Conclusion</title> <p>SBML-SAT provides the community of systems biologists a new tool for the analysis of their SBML models of biochemical and cellular processes.</p>
<title>Implementation</title> <title>Overview of the software system</title> <p>SBML-SAT is designed to run simulation, steady state analysis, robustness analysis, as well as local and global sensitivity analysis for ordinary differential equations (ODE) based biological models. SBML-SAT meets the needs mentioned in the rational section with the following features:</p> <p>Inspired by our previous work in SBML-PET project [##REF##16926221##20##], we enabled SBML-SAT to support a variety of models including assignment rules and events, even for complicated event scenarios such as bisecting mass in case of cell division. Therefore, SBML-SAT will have a wide applicability for different types of models.</p> <p>In addition to the implementation of traditional local sensitivity analysis, SBML-SAT provides four different global sensitivity analysis methods, including multi-parametric sensitivity analysis [##REF##15710397##7##,##UREF##1##12##], partial rank correlation coefficient analysis [##UREF##0##9##,##UREF##2##22##], SOBOL's method [##REF##16986622##15##,##UREF##3##23##] and weighted average of local sensitivities [##REF##15364960##2##]. Furthermore, steady state analysis and robustness analysis are also available in this tool. The algorithms for these different types of analyses are briefly described in the following section.</p> <p>The sensitivity analysis can be performed with respect to any ODE model variable (species amount or concentration) and reaction rate; these quantities are referred to as the object of the sensitivity analysis in SBML-SAT. The model output, which the sensitivity analysis is performed on, is defined through a functional operation on the object. The predefined model outputs in SBML-SAT are: steady state response, maximum response, integrated response, and time dependent response. The steady state response is only applicable for model objects that are ODE variables as the sensitivity analysis is computed with respect to the equilibrium solution of the system (when all derivatives of the ODE variables are algebraically set to zero). The maximum response is the maximum value of the object <italic>X</italic><sub><italic>i </italic></sub>(state variable or reaction rate) over the time course simulated:</p> <p></p> <p>The integrated response corresponds to the area under the curve when plotting <italic>X</italic><sub><italic>i </italic></sub>versus time. SBML-SAT approximates the integrated response for object <italic>X</italic><sub><italic>i </italic></sub>by the discrete summation [##REF##17825822##24##]:</p> <p></p> <p>The time dependent response performs multiple sensitivity analyses based on the values of the object, <italic>X</italic><sub><italic>i</italic></sub>, at selected time points during the simulated time course.</p> <p>SBML-SAT for Windows, Mac, and Linux can be freely downloaded from its website <ext-link ext-link-type="uri" xlink:href="http://sysbio.molgen.mpg.de/SBML-SAT"/>. The manual documentation file including a detailed tutorial for the usage of SBML-SAT is also available in the website. The future updates of SBML-SAT will be released on the website as well. Like most other SBML supported software systems, SBML-SAT requires a link to libSBML and utilizes SBMLToolbox [##REF##16574696##25##], which allows us to import SBML into MATLAB [##UREF##4##26##]. Once the SBML model is imported into SBML-SAT, a MATLAB file will be automatically generated, which includes the ODEs of the model. This is very helpful for the user, who wants to code in MATLAB for other purposes. To speed up the process of solving the ODEs, we employed the CVODE module of SUNDIALS (Suite of Nonlinear and Differential/Algebraic Equation Solvers) as the ODE Solver [##UREF##5##27##]. An interface to setting the options of CVODE solver is also available in SBML-SAT. Both SBMLToolbox and SUNDIALS [##UREF##6##28##] can be freely downloaded.</p> <p>In order to run the analysis in SBML-SAT, the users need to represent their models in SBML format which can be easily done using the existing software systems such as CellDesigner [##UREF##7##29##], COPASI [##REF##17032683##18##] and SBMLeditor [##REF##17341299##30##]. Then, the users can load the SBML models to SBML-SAT and perform a variety of analyses.</p> <p>Simulation, robustness analysis and sensitivity analysis can be easily implemented using SBML-SAT's graphical user interface (Figure ##FIG##0##1##). SBML-SAT allows the user to browse the model information, to save the model as well as to simulate and analyze the model. Simulation and sensitivity analysis results can be exported as text files, making it convenient for post-processing. In addition to the export function, SBML-SAT provides automatic visualization of the results. Furthermore, SBML-SAT is smart in remembering the user's settings for the corresponding tasks. The user can save his/her project settings as a project file and load it again to SBML-SAT for further analysis later.</p> <title>Local sensitivity analysis</title> <p>Local sensitivity analysis is a study of the changes in the model outputs with respect to parameter (factor) variations around a local point in the parameter space, which are quantified by the sensitivity coefficients. Mathematically, the sensitivity coefficients are the first order derivatives of model outputs with respect to the model parameters:</p> <p></p> <p>where <italic>O</italic><sub><italic>i </italic></sub>is the <italic>i</italic>-th model output and <italic>p</italic><sub><italic>j </italic></sub>is the <italic>j</italic>-th parameter. This is called "Unnormalized Sensitivity" in SBML-SAT. SBML-SAT employed centered difference approximation to compute the sensitivity coefficients in the following way [##REF##15695639##31##]:</p> <p></p> <p>When the model output and parameters are non-zero, the normalized sensitivity coefficients are defined as:</p> <p></p> <p>The new model outputs are calculated by a small perturbation (Δ<italic>p</italic><sub><italic>j</italic></sub>) of parameter <italic>p</italic><sub><italic>j </italic></sub>while keeping all the other parameter values the same: SBML-SAT computes one-at-a-time (OAT) local sensitivity coefficients.</p> <p>The proper choice of perturbation size is a delicate issue as it depends on the nature of the model and the numerical solution method. The perturbation should be small enough to achieve a negligible error in the centered difference approximation, and large enough to be unaffected by the numerical inaccuracies of the ODE solver. Too large parameter perturbation violates the implied linearity of the approximations in (4) and (5) and will provide inaccurate results. The user can modify the perturbation coefficient in the "Sensitivity Analysis" panel of SBML-SAT. The default perturbation is 0.1% of the corresponding parameter value, ie. Δ<italic>p</italic><sub><italic>j </italic></sub>= 0.001 × <italic>p</italic><sub><italic>j</italic></sub>.</p> <title>Global sensitivity analysis</title> <p>As mentioned in the rationale section, there are many sources of uncertainty in the model parameter values. Global sensitivity analysis is a useful way to investigate the global effects of parameters on the model output by simultaneously perturbing all the parameters within a parameter space. In the SBML-SAT tool, four different global sensitivity analysis methods are available. Each method has a distinct mathematical rationale and can be used for different purposes.</p> <p>(1) Multi-Parametric Sensitivity Analysis (MPSA): This method was first proposed by Hornberger et al [##UREF##8##32##] in the field of hydrology and further applied to modeling of biological systems by Cho et al. [##UREF##1##12##] and Zi et al. [##REF##15710397##7##]. MPSA can be used to study the relative importance of the parameters with respect to the model output. The basic idea of MPSA is to map the uncertainty of the parameters into the model output by randomly generating parameter values from predefined distributions (without prior knowledge, uniform distributions are assumed). SBML-SAT uses Latin Hypercube Sampling (LHS) method to sample the parameter values under the given ranges of the parameters [##REF##15710397##7##]. The LHS method is an efficient method to sample random parameter vectors while guaranteeing that individual parameter ranges are evenly covered [##UREF##9##33##]. The ranges of the parameter distributions are usually determined from the available literature or guided by experience of the researchers.</p> <p>For each randomly generated parameter set, the objective function is computed by the sum of square errors between the model outputs from the random parameter set and the reference parameter set. The next step is to classify each parameter set as acceptable or unacceptable by comparing its objective function value to the average of all the objective function values. If the objective function value is smaller than the average, the parameter set is classified as "acceptable"; otherwise it is "unacceptable". Then, the cumulative frequency is calculated for both acceptable and unacceptable cases for each selected parameter with increasing parameter values. Finally, the sensitivity of the parameter is measured by the maximum vertical distance of the two cumulative frequency curves according to the Kolmogorov-Smirnov statistics [##REF##15710397##7##]. The calculated MPSA sensitivities are between 0 and 1, where a value closer to 1 indicates a relatively higher importance of the parameter variation to the overall corresponding model output.</p> <p>(2) Partial Rank Correlation Coefficient Analysis (PRCC): The PRCC method is a rank transformed linear regression analysis that is routinely used for analysis of systems with a nonlinear and monotonic relationship between the system inputs and outputs [##UREF##2##22##]. Linear regression analysis methods best fit a straight line to input and output values. When nonlinear, monotonic relationships exist between system input and outputs, poor linear regression fits can be alleviated by performing the linear regression analysis on a rank ordered list of the model output and input values. PRCC calculates the sensitivity indices from the Pearson correlation coefficients between the model output and input parameters as well as each pair of parameters after rank transformation [##UREF##0##9##]. Interactions among different parameters are eliminated by evaluating multiple regression models on a subset of parameters that excludes a single parameter. The calculated PRCC sensitivity indices are a standardized sensitivity measurement between -1 and 1 with 0 indicating an input to which the model output is completely insensitive. SBML-SAT computes PRCC as implemented in [##REF##16986622##15##] with LHS sampling of the parameter space.</p> <p>(3) SOBOL's Method: SOBOL's method is a variance based method that makes no assumptions on the relationship between the system inputs and outputs. It is computationally expensive since it utilizes a large number of model simulations with parameter values sampled from the parameter space by the winding stair algorithm. The variance of the numerous model outputs is estimated by Monte Carlo integrations. The model output variance is apportioned into summands of partial variances from combinations of input parameters with increasing dimensionality [##UREF##3##23##]. The total effects sensitivity indices quantify all of the effects that a parameter, in combination with any other parameter(s), has on the model output. They are defined as the ratio of the sum of the related partial variances to the overall variance of the model output. The larger the fraction, the higher is the corresponding sensitivity. SBML-SAT calculates the total effect sensitivity indices.</p> <p>(4) Weighted Average of Local Sensitivities: In this approach, local sensitivity indices are calculated at multiple random points within the parameter space; a weighted average of the local sensitivity indices serves to provide some approximation of the global parameter sensitivities. Bentele et al. [##REF##15364960##2##] proposed a Boltzmann-Distribution weighting function, exp(-E/k<sub>b</sub>T), where E is the error between the model simulation and experimental data and k<sub>b</sub>T is a customizable scaling factor. Herein we define E as the least squares error (LSE) between the perturbed model simulation and reference model simulation and k<sub>b</sub>T as the minimum LSE. Based on this weighting function, the random points in the parameter space with low LSE contribute the most to the calculated global sensitivity indices.</p> <title>Steady state analysis</title> <p>SBML-SAT uses two different methods to check the existence of a steady state for the SBML model. The first strategy is to set the ordinary differential equations to zero and solve the algebraic system by KINSOL, which is part of the software family called SUNDIALS and is an algebraic system solver based on Newton-Krylov method [##UREF##5##27##]. Another method is called quasi steady state method, which runs the simulation for a very long time and check the rate of change of the ODE variables (such as species and other state variables) at different time points. When the rates of change for all the variables are smaller than a certain threshold (1 × 10<sup>-10</sup>), a quasi steady state is reached. The latter method is useful for steady state analysis of models that include events and implicit mass conservation rules. These two methods will only find a single steady state to which the initial condition converges. Other existing steady states as well as the steady state of oscillatory and unstable system will not be detected. SBML-SAT automatically selects the method for steady state analysis. If the model doesn't have events, SBML-SAT will use the algebraic method to detect the steady state of the model. Otherwise the second quasi steady state method will be used.</p> <title>Robustness analysis</title> <p>Robustness is one of the fundamental properties of biological systems, which allows the system to maintain its behavior against random perturbations [##REF##15369668##34##,##REF##17882156##35##]. SBML-SAT employs a method proposed in previous studies to investigate the robustness of model output against the total parameter variation, <italic>TPV</italic>, which is defined as [##REF##9202124##36##, ####REF##14604583##37##, ##UREF##10##38####10##38##]:</p> <p></p> <p>where <italic>k</italic><sub><italic>n </italic></sub>are the perturbed model parameters randomly generated by the LHS method;  are the corresponding reference parameter values in the model; <italic>L </italic>is the total number of parameters that are randomly varied.</p> <p>To measure robustness, we use the robustness metric , which quantifies the change in a function of the system (model output) induced by TPV:</p> <p></p> <p>where <italic>f</italic><sub>0 </sub>and <italic>f</italic><sub><italic>p </italic></sub>are the model output which describes the biological function under non-perturbed condition (reference model) and perturbed condition (parameters varied model), respectively. <italic>N </italic>is the total number of perturbations or model simulations. <italic>M </italic>denotes the model for the corresponding system. When the reference model output is zero, the following alternative definition is used:</p> <p></p> <p>According to the definition of (7) and (8), the robustness score of a biological system (model) usually assumes a negative value. The closer it is to zero, the more robust the system (model) is against the perturbations (parameter variations). When the robustness score of a system is zero, it means this system is absolutely robust against the imposed perturbations.</p> <p>The difference between the robustness scores of two systems (models) with respect to a certain model output against the perturbations can be evaluated as:</p> <p></p> <p>The comparison of the robustness scores of two systems/models is meaningful only when the evaluated model output of the two systems/models are the same and perturbations are operated in the same way.</p> <title>Availability and requirements</title> <p><bold>Project name: </bold>SBML-SAT: A Systems Biology Markup Language (SBML) based Sensitivity Analysis Tool</p> <p><bold>Project homepage: </bold><ext-link ext-link-type="uri" xlink:href="http://sysbio.molgen.mpg.de/SBML-SAT/"/></p> <p><bold>Operating system(s): </bold>Windows, Linux, Mac</p> <p><bold>Programming language:</bold> Matlab</p> <p><bold>Other requirements: </bold>SBMLToolbox, SUNDIALS TB</p> <p><bold>License:</bold> none</p> <p><bold>Any restrictions to use by non-academics:</bold> none</p> <title>Abbreviations</title> <p>SBML: Systems Biology Markup Language; SBML-SAT: Systems Biology Markup Language based Sensitivity Analysis Tool; MPSA: Multi-Parametric Sensitivity Analysis; PRCC: Partial Rank Correlation Coefficient; WALS: Weighted Average of Local Sensitivities; GUI: Graphic User Interface; LSE: Least Squares Error; TPV: Total Parameter Variation</p> <title>Authors' contributions</title> <p>ZZ proposed the project, designed the GUI interface and wrote all the source code of the software. AR and YZ contributed some algorithms for global sensitivity analysis methods. ZZ, YZ, AR and EK wrote the manuscript and tested the software. All authors have read and approved the final manuscript.</p>
<title>Acknowledgements</title> <p>Z. Zi is supported by PhD program of the IMPRS for Computational Biology and Scientific Computing. EK thanks the Yeast System Biology Network (YSBN, EU project: grant LSHG-CT-2005-018942) for support.</p>
<fig position="float" id="F1"><label>Figure 1</label><caption><p><bold>GUI of SBML-SAT</bold>. The graphic user interface (GUI) of SBML-SAT provides an easy way for the user to run the simulation, steady state analysis, sensitivity analysis and robustness analysis for SBML models.</p></caption></fig> <fig position="float" id="F2"><label>Figure 2</label><caption><p><bold>Plot of simulation result in SBML-SAT</bold>. The plot function enables the user to visualize the time course profiles of species and reaction rates. This graph shows the simulation result of the MAPK cascade model [##REF##10712587##43##] (BioModels ID: BIOMD0000000010).</p></caption></fig> <fig position="float" id="F3"><label>Figure 3</label><caption><p><bold>Simulation of different types of models in SBML-SAT</bold>. (A) Simulation result of the fission yeast cell cycle model (events included, BioModels ID: BIOMD0000000111), identical to Fig. 4 of [##REF##12779461##44##]. (B) Simulation result of a NF-κB signalling pathway model (BioModels ID: BIOMD0000000140), identifical to Fig. 2F of [##REF##12424381##1##]. (C) Simulation result of a T cell gene expression model (BioModels ID: BIOMD0000000122), identical to Fig. 4a of [##REF##17031595##45##]. (D) Simulation result of a metabolic model (BioModels ID: BIOMD0000000106), identical to Fig. 2A of [##REF##17381237##46##].</p></caption></fig> <fig position="float" id="F4"><label>Figure 4</label><caption><p><bold>Local sensitivity analysis in SBML-SAT</bold>. Local sensitivity analysis of the integrated response of MAPK cascade model [##REF##10712587##43##] (BioModels ID: BIOMD0000000010) with respect to variation of initial conditions.</p></caption></fig> <fig position="float" id="F5"><label>Figure 5</label><caption><p><bold>Scheme of receptor trafficking network model</bold>. Schematic description of the receptor trafficking network. The symbols <italic>L, Rs, LRs, Ri, LRi </italic>represent the ligand, unbound cell surface receptor, cell surface ligand-receptor complex, internalized unbound receptor and internalized ligand-receptor complex, respectively. The parameter information is listed in Table 1.</p></caption></fig> <fig position="float" id="F6"><label>Figure 6</label><caption><p><bold>Global sensitivity analysis in SBML-SAT</bold>. Results of different types of global sensitivity analysis for the receptor trafficking model. (A) MPSA analysis, (B) PRCC analysis, (C) SOBOL's total effect sensitivity analysis, and (D) WALS analysis of integrated response. (E) MPSA analysis of the time dependent response. (F) MPSA analysis of the steady state response.</p></caption></fig> <fig position="float" id="F7"><label>Figure 7</label><caption><p><bold>Steady state analysis in SBML-SAT</bold>. Steady state analysis of the receptor trafficking network model in SBML-SAT.</p></caption></fig> <fig position="float" id="F8"><label>Figure 8</label><caption><p><bold>Robustness analysis in SBML-SAT</bold>. Robustness analysis of the steady state response of the receptor trafficking model against simultaneous variations of the parameter values. The red circles correspond to the reference model output. The blue points correspond to the model outputs under perturbed parameter values. (Specifications for SBML-SAT, "Model Output": "Steady State Response"). (A-B) Robustness of unbound receptor steady state concentration. (C-D) Robustness of ligand bound receptor steady state concentration.</p></caption></fig>
<table-wrap position="float" id="T1"><label>Table 1</label><caption><p>Parameters for the model of the receptor trafficking network</p></caption><table frame="hsides" rules="groups"><thead><tr><td align="center">Symbols of Parameters</td><td align="left">Corresponding Biological Processes</td></tr></thead><tbody><tr><td align="center">k<sub>1</sub></td><td align="left">de novo synthesis of surface receptor</td></tr><tr><td align="center">k<sub>2</sub></td><td align="left">formation of ligand-receptor complex</td></tr><tr><td align="center">k<sub>3</sub></td><td align="left">dissociation of ligand-receptor complex</td></tr><tr><td align="center">k<sub>4</sub></td><td align="left">recycling of internalized unbound receptor</td></tr><tr><td align="center">k<sub>5</sub></td><td align="left">internalization of unbound surface receptor</td></tr><tr><td align="center">k<sub>6</sub></td><td align="left">recycling of internalized ligand-receptor complex</td></tr><tr><td align="center">k<sub>7</sub></td><td align="left">internalization of surface ligand-receptor complex</td></tr><tr><td align="center">k<sub>8</sub></td><td align="left">dephosphorylation of ligand-receptor complex</td></tr><tr><td align="center">k<sub>9</sub></td><td align="left">degradation of unbound receptor</td></tr><tr><td align="center">k<sub>10</sub></td><td align="left">degradation of ligand-receptor complex</td></tr></tbody></table></table-wrap>
<disp-formula id="bmcM1"><label>(1)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1" name="1471-2105-9-342-i1" overflow="scroll"><mml:semantics><mml:mrow><mml:munder><mml:mrow><mml:mtext>max</mml:mtext></mml:mrow><mml:mrow><mml:mi>t</mml:mi><mml:mo>∈</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mtext> </mml:mtext><mml:msub><mml:mi>t</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mtext> </mml:mtext><mml:mn>...</mml:mn><mml:mo>,</mml:mo><mml:mtext> </mml:mtext><mml:msub><mml:mi>t</mml:mi><mml:mrow><mml:mi>e</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo>}</mml:mo></mml:mrow></mml:mrow></mml:munder><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mi>t</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM2"><label>(2)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M2" name="1471-2105-9-342-i2" overflow="scroll"><mml:semantics><mml:mrow><mml:msub><mml:mi>I</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mrow><mml:munderover><mml:mo>∫</mml:mo><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mn>0</mml:mn></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mrow><mml:mi>e</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:munderover><mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mrow></mml:mstyle><mml:mo>≌</mml:mo><mml:mfrac><mml:mrow><mml:mi>∆</mml:mi><mml:mi>t</mml:mi></mml:mrow><mml:mn>2</mml:mn></mml:mfrac><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mn>0</mml:mn></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mn>2</mml:mn><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mn>....</mml:mn><mml:mo>+</mml:mo><mml:mn>2</mml:mn><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mrow><mml:mi>n</mml:mi><mml:mo>−</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>t</mml:mi><mml:mi>n</mml:mi></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM3"><label>(3)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M3" name="1471-2105-9-342-i3" overflow="scroll"><mml:semantics><mml:mrow><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mo>∂</mml:mo><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:mo>∂</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mfrac></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM4"><label>(4)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M4" name="1471-2105-9-342-i4" overflow="scroll"><mml:semantics><mml:mrow><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mo>∂</mml:mo><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:mo>∂</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mo>≈</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mi>∆</mml:mi><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>−</mml:mo><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo>−</mml:mo><mml:mi>∆</mml:mi><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mi>∆</mml:mi><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM5"><label>(5)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M5" name="1471-2105-9-342-i5" overflow="scroll"><mml:semantics><mml:mrow><mml:msubsup><mml:mi>S</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mi>i</mml:mi><mml:mi>z</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mtext> </mml:mtext><mml:mfrac><mml:mrow><mml:mtext>    </mml:mtext><mml:mfrac><mml:mrow><mml:mo>∂</mml:mo><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mtext>    </mml:mtext></mml:mrow><mml:mrow><mml:mfrac><mml:mrow><mml:mo>∂</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mfrac></mml:mrow></mml:mfrac><mml:mo>≈</mml:mo><mml:mfrac><mml:mrow><mml:mtext>    </mml:mtext><mml:mfrac><mml:mrow><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mi>∆</mml:mi><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>−</mml:mo><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo>−</mml:mo><mml:mi>∆</mml:mi><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:msub><mml:mi>O</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mtext>    </mml:mtext></mml:mrow><mml:mrow><mml:mfrac><mml:mrow><mml:mn>2</mml:mn><mml:mi>∆</mml:mi><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mfrac></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM6"><label>(6)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M6" name="1471-2105-9-342-i6" overflow="scroll"><mml:semantics><mml:mrow><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>L</mml:mi></mml:munderover><mml:mrow><mml:mrow><mml:mo>|</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>log</mml:mi></mml:mrow><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:msub><mml:mi>k</mml:mi><mml:mi>n</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msubsup><mml:mi>k</mml:mi><mml:mi>n</mml:mi><mml:mn>0</mml:mn></mml:msubsup></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mo>|</mml:mo></mml:mrow></mml:mrow></mml:mstyle></mml:mrow></mml:semantics></mml:math></disp-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M7" name="1471-2105-9-342-i7" overflow="scroll"><mml:semantics><mml:mrow><mml:msubsup><mml:mi>k</mml:mi><mml:mi>n</mml:mi><mml:mn>0</mml:mn></mml:msubsup></mml:mrow></mml:semantics></mml:math></inline-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M8" name="1471-2105-9-342-i8" overflow="scroll"><mml:semantics><mml:mrow><mml:msubsup><mml:mi>R</mml:mi><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mi>p</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi></mml:mrow><mml:mi>M</mml:mi></mml:msubsup></mml:mrow></mml:semantics></mml:math></inline-formula> <disp-formula id="bmcM7"><label>(7)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M9" name="1471-2105-9-342-i9" overflow="scroll"><mml:semantics><mml:mrow><mml:msubsup><mml:mi>R</mml:mi><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mi>p</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi></mml:mrow><mml:mi>M</mml:mi></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mo>−</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mrow><mml:mrow><mml:mo>|</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>log</mml:mi></mml:mrow><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>p</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mn>0</mml:mn></mml:msub></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mo>|</mml:mo></mml:mrow></mml:mrow></mml:mstyle></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mo>.</mml:mo></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM8"><label>(8)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M10" name="1471-2105-9-342-i10" overflow="scroll"><mml:semantics><mml:mrow><mml:msubsup><mml:mi>R</mml:mi><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mi>p</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi></mml:mrow><mml:mi>M</mml:mi></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mo>−</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>p</mml:mi></mml:msub><mml:mo>−</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mstyle></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mo>.</mml:mo></mml:mrow></mml:semantics></mml:math></disp-formula> <disp-formula id="bmcM9"><label>(9)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M11" name="1471-2105-9-342-i11" overflow="scroll"><mml:semantics><mml:mrow><mml:mi>∆</mml:mi><mml:msubsup><mml:mi>R</mml:mi><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mi>p</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi></mml:mrow><mml:mrow><mml:mi>M</mml:mi><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>M</mml:mi><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msubsup><mml:mi>R</mml:mi><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mi>p</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi></mml:mrow><mml:mrow><mml:mi>M</mml:mi><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:mo>−</mml:mo><mml:msubsup><mml:mi>R</mml:mi><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mi>p</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi></mml:mrow><mml:mrow><mml:mi>M</mml:mi><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>.</mml:mo></mml:mrow></mml:semantics></mml:math></disp-formula>
<graphic xlink:href="1471-2105-9-342-1"/> <graphic xlink:href="1471-2105-9-342-2"/> <graphic xlink:href="1471-2105-9-342-3"/> <graphic xlink:href="1471-2105-9-342-4"/> <graphic xlink:href="1471-2105-9-342-5"/> <graphic xlink:href="1471-2105-9-342-6"/> <graphic xlink:href="1471-2105-9-342-7"/> <graphic xlink:href="1471-2105-9-342-8"/>
[{"surname": ["Blower", "Dowlatabadi"], "given-names": ["SM", "H"], "article-title": ["Sensitivity and Uncertainty Analysis of Complex-Models of Disease Transmission - an Hiv Model, as an Example"], "source": ["International Statistical Review"], "year": ["1994"], "volume": ["62"], "fpage": ["229"], "lpage": ["243"]}, {"surname": ["Cho", "Shin", "Kolch", "Wolkenhauer"], "given-names": ["KH", "SY", "W", "O"], "article-title": ["Experimental design in systems biology based on parameter sensitivity analysis using a Monte Carlo method: a case study for the TNF alpha-mediated NF-kappaB signal transduction pathway"], "source": ["SIMULATION"], "year": ["2003"], "volume": ["79"], "fpage": ["726"], "lpage": ["739"]}, {"surname": ["Draper", "Smith"], "given-names": ["N", "H"], "source": ["Applied Regression Analysis"], "year": ["1981"], "edition": ["2nd"], "publisher-name": ["New York , Wiley"]}, {"surname": ["Sobol"], "given-names": ["IM"], "article-title": ["Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates"], "source": ["Mathematics and Computers in Simulation"], "year": ["2001"], "volume": ["55"], "fpage": ["271"], "lpage": ["280"]}, {"collab": ["Matlab"]}, {"surname": ["Hindmarsh", "Brown", "Grant", "Lee", "Serban", "Shumaker", "Woodward"], "given-names": ["AC", "PN", "KE", "SL", "R", "DE", "CS"], "article-title": ["SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers"], "source": ["Acm Transactions on Mathematical Software"], "year": ["2005"], "volume": ["31"], "fpage": ["363"], "lpage": ["396"]}, {"collab": ["SUNDIALS"]}, {"surname": ["Funahashi"], "given-names": ["A"], "suffix": ["Tanimura, N., Morohashi, M., and Kitano, H"], "article-title": ["CellDesigner: a process diagram editor for gene-regulatory and biochemical networks"], "source": ["BIOSILICO"], "year": ["2003"], "volume": ["1"], "fpage": ["159"]}, {"surname": ["Hornberger", "Spear"], "given-names": ["GM", "RC"], "article-title": ["An Approach to the Preliminary-Analysis of Environmental Systems"], "source": ["Journal of Environmental Management"], "year": ["1981"], "volume": ["12"], "fpage": ["7"], "lpage": ["18"]}, {"surname": ["Mckay", "Beckman", "Conover"], "given-names": ["MD", "RJ", "WJ"], "article-title": ["Comparison of 3 Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code"], "source": ["Technometrics"], "year": ["1979"], "volume": ["21"], "fpage": ["239"], "lpage": ["245"]}, {"surname": ["Zi", "Sun"], "given-names": ["ZK", "ZR"], "article-title": ["Robustness analysis of the IFN-gamma induced JAK-STAT signaling pathway"], "source": ["Journal of Computer Science and Technology"], "year": ["2005"], "volume": ["20"], "fpage": ["491"], "lpage": ["495"]}, {"surname": ["Fall;", "Marland;", "Wagner;", "Tyson"], "given-names": ["CP", "ES", "JM", "JJ"], "source": ["Computational Cell Biology"], "year": ["2002"], "publisher-name": ["New York , Springer"]}]
BMC Bioinformatics. 2008 Aug 15; 9:342