Quality of Life Research
The mathematical relationship among different forms of responsiveness coefficients
G. R. Norman
Kathleen W. Wyrwich
Donald L. Patrick
Little consensus exists regarding the most appropriate measure of responsiveness. While most indices are variants on Cohen's effect size, the mathematical relationships among these indices have not been elucidated. Consequently, the health-related quality of life (HRQL) literature contains many publications in which a variety of different indices are computed and differences among them noted. These differences are completely predictable when the underlying analytical form of each coefficient is explicated.
In this paper, we begin with a mathematical analysis of the variance components underlying an observed change score. From this, we determine analytically the relationships among the more commonly used indices of responsiveness.
Based on this analysis, we conclude that Cohen's effect size and the Standardized Response Mean are the two most appropriate measures, as each provides unique information and each best captures an important relation between treatment effect and variability in response. However, the latter should be interpreted with caution, as under some circumstances, any measure based on variability in change scores can give misleading information. On this basis, we recommend that future analysis of responsiveness be restricted to the Cohen effect size to ensure interpretability and comparability with treatment effects in other domains.
Responsiveness HRQL Equivalence Mathematics
Appendix 1: Derivation of variance components
As described in the paper, from Classical Test Theory, any observed score is considered to have two components, a true score and an error:
$$ O_{ij} =p_i +e_{ij} $$
where O ij is the observed score, p i represents the true score of patient i, and the error term, e ij is associated with the jth observation on patient I. By definition, errors have a mean of 0, and a standard deviation, σ e 2 . From this equation, the variance of baseline scores is a sum of variances due to differences between subjects and measurement error.
$$ \sigma_{\rm baseline}^2=\sigma_p^2+\sigma_e^2 $$
If we now consider a pretest–posttest situation, where there is no treatment effect, the difference between observed pretest and posttest, from Eq. 1, is:
$$ D_i=O_{i2}-O_{i1}=(p_i-p_i)+(e_{i2}-e_{i1})=(e_{i2}-e_{i1}) $$
where the pretest corresponds to j = 1 and the posttest to j = 2. From this, the variance of the difference scores is:
$$ \sigma _{{\rm pre}\hbox{-}{\rm post}}^2 =\sigma _e^2 +\sigma _e^2 =2\sigma _e^2 $$
If there is a treatment between the pretest and the posttest, Appendix Eq. 3 contains an additional term corresponding to the effect of treatment on patient i, which we will call t i . This can be viewed in turn as the sum of the overall treatment effect, T, and the difference between the overall treatment and the response of patient, i, which amounts to a (p × T) interaction.
$$ \hbox{Change}_i=T+(t_i-T)+(e_{i2}-e_{i1}) $$
Since the variance of the overall change, T, is zero, the variance of the change score is then:
$$ \sigma _{\rm change}^2 =\sigma _e^2 +\sigma _e^2+\sigma _{p\times T}^2=2\sigma _e^2+\sigma _{p\times T}^2 $$
American Psychological Association. (1999). Standards for educational and psychological tests.Google Scholar
Anderson, J. J., Firschein, H. E., & Meenan, R. F. (1989). Sensitivity of a health status measure to short-term clinical changes in arthritis. Arthritis and Rheumatism, 32, 844–850.PubMedCrossRefGoogle Scholar
Chren, M. M., Lasek, R. J., Flocke, S. A., & Zyzanski, S. J. (1997). Improved discriminative and evaluative capability of a refined version of Skindex, a quality-of-life instrument for patients with skin diseases. Archives of Dermatology, 133, 1433–40.PubMedCrossRefGoogle Scholar
Cohen, J. J. (1988). Statistical power analysis for the behavioral sciences (p. 8). Erlbaum: Hillsdale, NJ.Google Scholar
Crosby, R. D., Kolotkin, R. L., & Williams, G. R. (2003). Defining clinically meaningful change in health related quality of life. Journal of Clinical Epidemiology, 56, 395–407PubMedCrossRefGoogle Scholar
Deyo, R. A., & Centor, R. M. (1986). Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostic test performance. Journal of Chronic Diseases, 39, 897–906.PubMedCrossRefGoogle Scholar
Deyo, R. A., Diehr, P., & Patrick, D. L. (1991). Reproducibility and responsiveness of health status measures: Statistics and strategies for evaluation. Controlled Clinical Trials, 2, 142S–158SCrossRefGoogle Scholar
Fitzpatrick, R., Ziebland, S., Jenkinson, C., & Mowat, A. (1992). Importance of sensitivity to change as a criterion for selecting health status instruments. Quality in Health Care, 1, 89–93.PubMedGoogle Scholar
Guyatt, G. H., Walter, S. D., & Norman, G. R. (1987). Measuring change over time: Assessing the usefulness of an evaluative instrument. Journal of Chronic Diseases, 40, 171–178.PubMedCrossRefGoogle Scholar
Hays, R. D., & Hadorn, D. (1992). Responsiveness to change: An aspect of validity, not a separate dimension. Quality of Life Research, 1, 73–75PubMedCrossRefGoogle Scholar
Kirshner, B., & Guyatt, G. (1985). A methodological framework for assessing health indices. Journal of Chronic Diseases, 38, 27–36.PubMedCrossRefGoogle Scholar
Liang, M. J., Fossel, A. H., & Larson, M. G. (1990). Comparison of five health status instruments for orthopedic evaluation. Medical Care, 28, 632–642.PubMedCrossRefGoogle Scholar
Lindebloom, R., Sprangers, M. A. G., & Zwinderman, A. (2005). Responsiveness: A reinvention of the wheel? Health and Quality of Life Outcomes, 3, 8.CrossRefGoogle Scholar
Norman, G. R., Stratford, P., & Regehr, G. (1997). Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach. Journal of Clinical Epidemiology, 50, 869–879PubMedCrossRefGoogle Scholar
Norman, G. R., Stratford, P., & Regehr, G. (1997) Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach. Journal of Clinical Epidemiology, 50, 869–879.PubMedCrossRefGoogle Scholar
Norman, G. R., Wyrwich, K. W., & Sloan, J. A. (2003). Interpretation of changes in health-related quality of life: The remarkable universality of half a standard deviation. Medical Care, 41, 582–592.PubMedCrossRefGoogle Scholar
O'Keeffe, S. T., Lye, M., Donnellan, C., & Carmichael, D. N. (1998). Reproducibility and responsiveness of quality of life assessment and six minute walk test in elderly heart failure patients. Heart, 80, 377–382.PubMedGoogle Scholar
Pickard, A. S., Johnson, J. A., & Feeny, D. H. (2005). Responsiveness of generic health related quality of life measures in stroke. Quality of Life Research, 14, 207–219PubMedCrossRefGoogle Scholar
Reilly, M. C., & Zbrozek, A. S. (1992). Assessing the responsiveness of a quality-of-life instrument and the measurement of symptom severity in essential hypertension. Pharmacoeconomics 2, 54–66.PubMedGoogle Scholar
Sprangers, M. A. G., Moinpour, C. M., Moynihan, T. J., Patrick, D. L., & Revecki, D. A. (2002). Assessing meaningful change in quality of life over time: A user's guide for clinicians. Mayo Clinic Proceedings, 77, 561–571PubMedCrossRefGoogle Scholar
Stratford, P. W., Binkley, J. M., & Riddle, D. L. (1996). Health status measures: strategies and analytical methods for assessing change scores. Physical Theraphy, 76, 1109–1123.Google Scholar
Streiner, D. L., & Norman, G. R. (1993). Health measurement scales: A practical guide to their development and use (p. 142). Oxford University PressGoogle Scholar
Streiner, D. L., & Norman, G. R. (2003). Health measurement scales: A practical guide to their development and use, 3rd ed. (p. 141). Oxford University PressGoogle Scholar
Terwee, C. B., Dekker, F. W., Wiersinga, W. M., Prummel, M. F., & Bossuyt, P. M. M. (2003). On assessing responsiveness of health-related quality of life instruments: Guidelines for instrument evaluation. Quality of Life Research, 12, 349–362.PubMedCrossRefGoogle Scholar
Ware, J. E., Kemp, J. P., Buchner, D. A., Singer, A. E., Nolop, K. B., & Goss, T. F. (1998). The responsiveness of disease-specific and generic health measures to changes in the severity of asthma among adults. Quality of Life Research, 7, 235–244PubMedCrossRefGoogle Scholar
Wells, G., Beaton, D., Shea, B., Boers M., Simon, L., Strand, V., Brooks, P., & Tugwell, P. (2001). Minimal clinically important differences: Review of methods. The Journal of Rheumatology, 28, 406–412.PubMedGoogle Scholar
Wright, J. G., & Young, N. L. (1997). A comparison of different indices of responsiveness. Journal of Clinical Epidemiology, 50, 239–246PubMedCrossRefGoogle Scholar
Wyrwich, K. W., Tierney, W. M., & Wolinsky, F. D. (2002). Using the standard error of measurement to identify important changes on the Asthma Quality of Life Questionnaire. Quality of Life Research, 11, 1–7.PubMedCrossRefGoogle Scholar
1.Department of Clinical Epidemiology and Biostatistics, MDCL 3519McMaster UniversityHamiltonCanada
2.Department of Research MethodologySaint Louis UniversitySt. LouisUSA
3.Department of Health ServicesUniversity of WashingtonSeattleUSA
Norman, G.R., Wyrwich, K.W. & Patrick, D.L. Qual Life Res (2007) 16: 815. https://doi.org/10.1007/s11136-007-9180-x
Received 24 January 2006