MULTI-ATTRIBUTE HEALTH STATE
VALUATIONS:
AMBIGUITIES IN MEANING
Erik Nord, PhD,National Institute of Public
Health,
PO Box 4404 Torshov,N-0403 Oslo,Norway.
Phone:(47)22042342, Fax:(47)22042595,
E-MAIL: erik.nord@folkehelsa.no
Michael Wolfson, Statistics Canada, Ottawa K1A
oT6,
Ottawa,Canada.
Summary:
In everyday language, people never refer to the
goodness or badness of health states in terms of numbers. Hence, the meaning –
and validity - of such numbers is not obvious. We asked the constructors of MAU
instruments to explain the meaning of the numbers by specifying the kind of
decision-oriented propositions that they believe can be made on the basis of
the numbers that their instruments yield. The results show that constructors of
different instruments place quite different meanings on the numbers they offer.
From a user’s perspective this is highly problematic. Conceptual clarification
is needed.
Background
Multi-attribute utility instruments (MAU-instruments) score individuals' health on a number of different dimensions or attributes and then transform the set of scores into a single index value on a scale from zero (dead) to unity (healthy) by means of some mathematical formula. The transformations are based on statistical analyses of population preference data that show how highly different dimensions of health are valued relative to each other. The instruments are potentially useful both in measuring population health and cost-effectiveness analysis of health care interventions.
Following a high-level OECD meeting on Health Care Reforms in Paris in November 1994, we were asked by the OEDC Secretariat to conduct a survey among authors/developers of MAU-instruments. The Secretariat was particularly interested in reviewing such instruments with respect to the feasibility, reliability and sensitivity of their descriptive systems, the structure and validity of the valuation algorithms which transform individuals' health profiles into summary index numbers, the existence of translated versions of the descriptive part of the various instruments, their use in actual studies and user experiences.
We were at the time aware of seven research centers in the world who had developed an MAU instruments. The instruments include the Quality of Well-Being Scale (1), the 15-D (2), the Health Utilities Index, mark II and III (3), the EuroQol Instrument (4), the Index of Health Related Quality of Life (5), the Quality of Life and Health Questionnaire (6) and the Australian Quality of Life (7). We constructed a questionnaire that was distributed in March 1996 by the OECD Secretariat to the constructors of these instruments, as well as to contact persons in Member countries' Ministries of Health. The latter were asked to forward the questionnaire to any groups in their country who might have done relevant work.
We received completed questionnaires concerning 15-D, HUI2, HUI3, EQ-5D, AQOL and QLHQ.
The full results of the survey are reported elsewhere (8). Here we wish to draw attention to one particular finding that we think gives reason for concern about the state of art in this area and calls for careful reflection about the use and further development of MAU instruments.
The problem: the meaning of numbers for health states
In everyday language, people never refer to the goodness or badness of health states in terms of numbers (e.g. ‘How are you? I feel about 0.87 today.’). Hence, there is much debate about the interpretation of health state values. In our experience there has been insufficient reflection about and too little discussion of this question among constructors of MAU instruments, sometimes with costly consequences (9).
We hypothesized that lack of effort in conceptual clarification will show up in considerable variance across constructors with respect to the interpretations they recommend to potential users. Our underlying concern is that if suppliers of health state values have trouble in agreeing on what the numbers mean, how can we fell at all confident that such numbers are helpful in monitoring population health, cost-effectiveness analysis and clinical decision making?
Method
We asked the constructors of MAU instruments to specify the kind of decision-oriented propositions that they believe can be made on the basis of the numbers that their instruments yield. Our question read as follows: "Assume two health states X and Y, to which your instrument assigns the scores of 0.8 and 0.6 respectively. Disregarding imprecision of measurement, which of the following propositions are in your view implied (at least broadly speaking) by these numbers?"
Column 1 of Table 1 shows the various propositions offered. The reader will see that they are quite different in content. Moreover, there is good evidence that these alternative ways of framing health state valuation questions do have a material impact on people’s responses (10).
The constructors were furthermore asked if the implications they claim that their numbers have, are supported by direct preference measurements. For instance, if implication C was claimed, the constructor was asked if there was evidence of the instrument's ability to predict actual chronic patients' willingness to risk death in order to get well.
Results
Table 1 shows the instruments and the propositions their values imply according to the instrument constructors. (We emphasise that these propositions are not necessarily correct. The table reports what is being claimed by the instrument developers.) The table supports our hypothesis: Constructors of different instruments place quite different meanings on the numbers they offer.
Table 2 shows the results of the question asking for empirical support for those of implications C-G which were claimed by each instrument (cfr. table 1). Generally speaking, empirical support of this kind is weak.
Conclusion¨
Different MAU instruments have previously been shown to produce a wide range of values for the same health states (11). While this certainly is an undesirable state of affairs, one might think that the variance in valuations is due to variance in the ways in which MAU developers have chosen to ask about the same underlying concept. The above results suggest that the problem is a more fundamental one, namely lack of basic conceptual clarity and agreement.
This is cause for serious concern. It suggests inappropriate use of health state valuations in informing health care decisions, since the meaning of the valuations is so ambiguous and the evidence for those meanings that are claimed is so poor. We urge the constructors of the instruments to embark on a process towards greater conceptual clarity, transparency and consensus, and towards better empirical support of the meanings that are claimed.
REFERENCES
1. Kaplan RM, Anderson JP. A general health model: Update and applications. Health Services Research, 1988,23,203-235.
2. Sintonen H, Pekurinen M. A fifteen-dimensional measure of health-related quality of life (15 D) and its applications. In Walker SR, Rosser RM (eds). Quality of life assessment. Key issues in the 1990s. Dordrecht: Kluwer Academic Publishers, 1993.
3. Feeny D, Furlong W, Boyle M, Torrance GW. Multi-attribute health status classification systems. Pharmacoeconomics 1995,7,490-502.
4. Brooks R et al. EuroQol: The current state of play. Health Policy 1996,37,53-72.
5. Rosser R, Cottee M, Rabin R, Selai C. Index of health-related quality of life. In: Hopkins A (ed). Measures of the quality of life, and the uses to which they may be put. London: Royal College of Physicians of London, 1992.
6. Hadorn D. Large scale health outcomes evaluation: How should quality of life be measured? Journal of Clinical Epidemiology 1995,48,607-618.
7. Hawthorne G, Richardson J. An Australian multi-attribute utility: Rationale and preliminary results. Working paper 49. Melbourne: Centre for Health Program Evaluation, 1996.
8. Nord E. A review of synthetic health indicators. Background paper for the OECD Directorate for Education, Employment, Labour and Social Affairs. Mimeo, June 1997.
9. Nord E. Unjustified use of the Quality of Well-Being Scale in priority setting in Oregon. Health Policy 1993,24,45-53.
10. Nord E. Methods for quality adjustment of life years. Social Science & Medicine 1992,34,559-569.
11. Nord E. Health status index models for use in resource allocation decisions. A critical review in the light of observed preferences for social choice. International Journal of Technology Assessment in Health Care 1996,12,31-44.
Table 1. Propositions that V(X) = 0.8 and
V(Y) = 0.6 purport to imply.
|
|
15-D |
EQ5D |
HUI-1 |
HUI-2 |
AQOL |
QLHQ |
person in state
Y is willing to sacrifice twice as much to become healthy as the average
person in state X. |
|
|
|
|
X |
|
|
· All else equal, the average person who is about to die is willing to sacrifice five times as much to be restored to full health as the average person in state X. |
|
|
|
|
|
|
|
C. The average person facing a life in state X is willing to take a treatment
that gives him/her at least an 80 % chance of becoming healthy and at most a 20 % chance of dying immediately
(assuming that the treatment itself causes negligible inconvenience and discomfort). |
X |
|
X |
X |
|
|
would be
indifferent between living the rest of
his/her life in this state and living
a 20 % shorter life in full health. |
X |
|
|
|
X |
|
|
· Society regards as equal in value a program that restores one person from dying to full health and (b) a program
that cures five people in state X (given equal life expectancy after the intervention). |
X |
X |
|
|
X (PTO) |
|
|
· The health related quality of life in state Y is 75 % of that of life in state X. |
|
X |
|
|
X |
X |
|
· The utility derived from state Y is 75 % of that of the utility derived from state X. |
|
(X) |
|
|
X |
X |
Table 2. Empirical support of purported implications of health states values, according to instrument constructors, cfr table 1.
|
|
15-D |
EQ5D |
HUI-2 |
HUI-3 |
AQOL |
QLHQ |
|
C |
‘some’ |
|
no |
no |
|
|
|
D |
‘some’ |
|
|
|
not yet |
|
|
E |
‘some’ |
no |
|
|
|
|
|
F/G |
|
no |
|
|
‘possible’ |
no |