What do hospital decision-makers in Ontario, Canada, have to say about the fairness of priority setting in their institutions?
© Reeleder et al; licensee BioMed Central Ltd. 2005
Received: 20 September 2004
Accepted: 21 January 2005
Published: 21 January 2005
Priority setting, also known as rationing or resource allocation, occurs at all levels of every health care system. Daniels and Sabin have proposed a framework for priority setting in health care institutions called 'accountability for reasonableness', which links priority setting to theories of democratic deliberation. Fairness is a key goal of priority setting. According to 'accountability for reasonableness', health care institutions engaged in priority setting have a claim to fairness if they satisfy four conditions of relevance, publicity, appeals/revision, and enforcement. This is the first study which has surveyed the views of hospital decision makers throughout an entire health system about the fairness of priority setting in their institutions. The purpose of this study is to elicit hospital decision-makers' self-report of the fairness of priority setting in their hospitals using an explicit conceptual framework, 'accountability for reasonableness'.
160 Ontario hospital Chief Executive Officers, or their designates, were asked to complete a survey questionnaire concerning priority setting in their publicly funded institutions. Eight-six Ontario hospitals completed this survey, for a response rate of 54%. Six close-ended rating scale questions (e.g. Overall, how fair is priority setting at your hospital?), and 3 open-ended questions (e.g. What do you see as the goal(s) of priority setting in your hospital?) were used.
Overall, 60.7% of respondents indicated their hospitals' priority setting was fair. With respect to the 'accountability for reasonableness' conditions, respondents indicated their hospitals performed best for the relevance (75.0%) condition, followed by appeals/revision (56.6%), publicity (56.0%), and enforcement (39.5%).
For the first time hospital Chief Executive Officers within an entire health system were surveyed about the fairness of priority setting practices in their institutions using the conceptual framework 'accountability for reasonableness'. Although many hospital CEOs felt that their priority setting was fair, ample room for improvement was noted, especially for the enforcement condition.
Priority setting, also known as rationing or resource allocation, occurs at all levels of every health care system, including governments, funded provincial/territorial agencies, pharmaceutical benefit-management organizations, hospitals and clinical programs . Countries with very different health care systems and levels of health care are all grappling with the problem of how to reconcile growing demands and constrained resources . Hospitals, in particular, are struggling to meet growing demands, affordably, without compromising delivery of services [3, 4].
The four conditions of 'accountability for reasonableness'
Rationales rest on evidence, reasons, and principles that fair-minded parties can agree are relevant to deciding how to meet the diverse needs of a covered population under necessary resource constraints.
Limit-setting decisions and their rationales must be publicly accessible.
There is a mechanism for challenge and dispute resolution regarding limit-setting decisions, including the opportunity for revising decisions in light of further evidence or arguments.
There is either a voluntary or public regulation of the process to ensure that the first three conditions are met.
'Accountability for reasonableness' may be an effective framework for describing the components of priority setting, evaluating the fairness of priority setting in hospitals and identifying 'good' practices and opportunities for improvement. It can help hospitals improve their priority setting practices and can be an effective driver of health care reforms [8–10].
'Accountability for reasonableness' has been used to evaluate priority setting in health systems . However, only a limited number of studies have addressed how priority setting occurs in hospitals [12–17], and no study has attempted to survey the views of hospital decision makers throughout an entire health system about the fairness of priority setting in their institutions.
The purpose of this study is to elicit hospital decision-makers' self-report of the fairness of priority setting in their hospitals using an explicit conceptual framework, 'accountability for reasonableness'.
Survey Questions – Rating* and Open Scale
Overall, how fair is priority setting at your hospital? (Rating Scale). Please explain your response (Open-ended).
How well does your hospital meet its priority setting goal(s)? (Rating Scale). Please explain your response (Open-ended).
What do you see as the goal(s) of priority setting in your hospital? (Open-ended)
How well is the relevance condition met at your hospital? (Rating Scale)
How well is the publicity condition met at your hospital? (Rating Scale)
How well is the appeals condition met at your hospital? (Rating Scale)
How well is the enforcement condition met at your hospital? (Rating Scale)
Setting and sample scope
With 12 million people, Ontario is the largest province in Canada. Like the rest of the country, it is a single payor, predominately publicly-funded health care system with some privately funded services and products (e.g. dental services, drugs). There are 160 hospitals in Ontario and all were invited to participate in this study.
160 Ontario hospital Chief Executive Officers, or their designates, were asked to participate. 86 Ontario hospitals completed this survey, for a response rate of 54%. The average bed count of the responding hospitals was 250.4, with a range of 18 to 1265 beds. The average operating budget of the hospital sample was $75.8 million, ranging from $3.3 million to $733 million. The sampled hospitals represented a blend of teaching, small, community and specialized service facilities across the province.
The survey was pre-tested and mailed to Chief Executive Officers of all Ontario hospitals in January 2001. Data from returned surveys were entered into an electronic database for further analysis. Of the 9 questions analyzed in this study, 6 asked for a rating on a 5-point scale ranging from 'Not Well' or '1' to 'Very Well' or '5', and 3 were open-ended.
Close-ended ratings were analyzed statistically , with ratings equal to and below the mid-point combined to identify proportions of respondents suggesting improvement, using a conservative bivariate cut-off point to discriminate among reported responses. P values for all hypothesis tests were two tailed. Open-ended responses were analyzed using a modified thematic analysis involving open and axial coding techniques [19, 20]. In open coding, data was segmented and coded with a label identifying parts of text relating to a concept or idea. For axial coding, concepts were organized into overarching themes, and compared, both within and between questions, in search of patterns in responses and to ensure consistency.
Summary of Decision Maker Responses a,b
How well does your hospital meet its priority setting goal(s)?
Overall, how fair is priority setting at your hospital?
How well is the relevance condition met at your hospital?
How well is the publicity condition met at your hospital?
How well is the appeals condition met at your hospital?
How well is the enforcement condition met at your hospital?
5 'Very Well'
1 'Not Well'
Priority setting and fairness ratings were positively correlated (r = .51, p < .01). Fairness ratings were also positively correlated with each of the 4 'accountability for reasonableness' conditions (p < .01), as were meeting priority setting goals (p < .05).
The bivariate correlation between each of the accountability variables of relevance, publicity, appeals/revision and enforcement was strongly positive (p < .01). The Pearson correlation coefficients among the 4 conditions ranged from 0.41 to 0.57. The internal consistency (Cronbach's α) of all of the 'accountability for reasonableness' conditions was very good (α = .78).
Bed size (r = -.24, p <.05) and operating budget (r = -.32, p < .01) were negatively correlated with the rating of fairness in priority setting.
Analysis of respondent comments
In this section we describe participants' responses to the open-ended questions organized according to themes. We have included verbatim responses to help illustrate the themes.
What do you see as the goals of priority setting in your hospital?
Decision makers emphasized four priority setting goals which were complex, interdependent, and required balancing. Some respondents said that (1) meeting needs existed in relation to delivering (2) quality services, or was constrained by resource availability. Examples of these goals include "quality care within available resources" and "access to high quality service in areas of greatest need". While (3) meeting budgets was identified as a goal by some decision makers, it did not exist in isolation from other goals such as meeting strategic directions or needs. One decision maker said his goal was "ensuring a balanced situation at year end while meeting the direction set out in our strategic plan". Achieving (4) organizational goals was expressed as a limiting, or organizational focusing process, which involved alignment or balancing of multiple goals and values.
How well does your hospital meet its priority setting goals? Please explain your response
Decision makers described a deliberative process by which their priority setting goals were operationalized, providing examples of good practices and challenges or barriers for achievement. Overall, decision makers appeared confident that their priority setting goals were the correct ones, in no case did they comment that expressed goals were unrealistic, so contributing to lack of achievement.
Decision makers pointed to five factors of importance in meeting their priority setting goals. Review processes (1) were described in which different review processes are brought to bear pointing to areas requiring additional resources. For example, a decision maker said "annually we review and ensure that resources are being allocated to priority programs and services". Leadership (2) was implicated as an important factor in meeting priority-setting goals. For example, a decision maker said "We do not set goals that are pie in the sky, they must be achievable". With respect to (3) stakeholder consultation, some respondents pointed to the need to involve the wider community in decision making. Decision-makers felt that improvements in priority setting were contingent upon (4) access to relevant information. Finally, some decision-makers emphasized the importance of (5) decision making tools, or benchmarking, to improve decision making.
Overall, how fair is priority setting at your hospital? Please explain your response
Respondents gave information to support their self-evaluation, and provided examples to demonstrate the fairness of priority setting processes from their perspective.
In evaluating the fairness of priority setting, decision makers emphasized five themes. Stakeholder consultation raises the level of (1) inclusivity, bringing different points of views and interests to bear in solving common problems. For example, a decision maker said "we have a service providers network in our community that has direct input into resource allocation and program planning". Respondents described review processes (2) involving the deliberation of concerned parties, using relevant data, based on need, cost and other values (e.g. evidence). Reporting systems (3) provide an institutional feedback loop, pointing to areas where limits may be fairly set. With respect to (4) revision/appeals, revising arguments or values based on iterative processes involving various stakeholders is a feature of fairness, and helps to improve buy-in, but may increase organizational tensions. Finally, (5) governance may also be involved in developing decision-making models, and in ensuring the conditions of relevancy, appeals/revision, and publicity are met. For example, a decision maker said "The Board and senior management have developed decision-making models for key program/service changes...our appeals process is informal, but open to access including the Board".
To our knowledge, this is the first published survey of hospital decision makers covering an entire health system, using Daniels and Sabin's framework of 'accountability for reasonableness'. It provides data to understand how fair priority setting processes are, what decision makers' priority setting goals are, and how well these goals are met.
Several studies, using either survey or qualitative methodologies sometimes in combination with hypothetical "tradeoffs", have examined priority setting from theoretical perspectives aimed at capturing the public or professional's preferences or values in priority setting [21–24]. Relatively few studies have described what is happening in real-world contexts with a view of evaluating and improving priority setting activities [25–31]. These have described actual priority setting in individual hospitals, focusing on strategic planning , surgery , critical care , new technologies , and the rationing of new drugs , with no study reviewing priority setting in hospitals across the health system.
This study makes five contributions to knowledge on priority setting. First, there is ample room for improvement in fair practices within Ontario hospitals as described by their Chief Executive Officers. Although many hospital CEOs felt that their priority setting was fair, ample room for improvement was noted, especially for the enforcement condition, followed by publicity, appeals/revision and relevance. While 79% of decision makers felt their hospitals met their priority setting goals, only 60.7% felt their processes were fair. According to decision makers, the perceived 'gap' was greater in meeting fairness than in priority setting goals.
Second, it is feasible for decision makers to assess the fairness of priority setting in their institutions on a quantitative basis according to the 'accountability for reasonableness' framework, with greater than half of respondents saying their processes are fair. There is a high degree of association between 'fairness' and the internal components of 'accountability for reasonableness', lending support that fairness can be operationalized through 'accountability for reasonableness'. As well, the internal conditions of 'accountability for reasonableness' are highly related, and positively correlated, suggesting relevance, publicity, appeals/revision, and enforcement are measuring comparable aspects of fairness. Consistent with this, the high Cronbach's α finding is promising for the development of an 'accountability for reasonableness' scale of perceived fairness of priority setting in health care institutions. Measuring decision makers' self-report of their perception in meeting the various conditions of 'accountability for reasonableness', the scale would provide a practical tool for decision makers to assess self-reported views over time, noting progress in meeting conditions, while providing a reference point for needed improvement (i.e. enhance meeting 'publicity' condition). Finally, the finding of a negative relationship between number of hospital beds or budgets and self-report on fairness is consistent with the view that smaller hospitals may be perceived to have fairer processes, with greater involvement of their local communities in decision making and emphasis on transparency and "trust".
Third, decision makers' views were surveyed at a health system level to determine what their actual priority setting practices were. Four complex, interrelated priority setting goals were described, suggesting a balance of need, quality, budget and organizational goals in priority determination. In addition, the study points to a close alignment of factors required in the meeting of priority setting goals and in the elements of fairness. In providing decision maker perspectives on characteristics of fairness, this study also adds to previous empirical work suggesting fair priority setting depends on a fair priority setting process . Additional research is required to understand the capacity of organizations to improve such practices.
Four, this study has shown, from a blend of qualitative and quantitative approaches, that it is feasible to operationalize the 'accountability for reasonableness' framework, through the design of survey instruments to facilitate self-evaluation, and identification of good practices. This data can be shared with decisions makers to improve the fairness of their priority setting processes in meeting the four conditions of 'accountability for reasonableness'.
Finally, this study expands on the likely relationship between leadership and priority setting in which leadership was found to contribute to perceptions of fairness in two committees engaged in priority setting for new health care technologies. Study results indicate that greatest room for improvement exists in meeting the 'enforcement' condition, with decision-makers describing critical leadership success factors, including: the role of governance in establishing policy and in meeting this condition, the need to set achievable corporate goals, and the significance of a lobbying funding function. Further study is needed to clarify the nature of this leadership contribution.
The study is limited, first, in the response rate from hospitals. Only 54% of Ontario's 160 hospital CEO's responded to the survey. It is possible that hospitals responding to this survey were those doing, or perceived doing, better on self-report than others. However, there is no evidence to suggest that this is the case, and the sample did include small, medium and larger teaching hospitals, as well as specialized facilities to mitigate against representative selection bias. Second, social desirability bias was possible in that the views of decision makers expressed in the survey may not have corresponded to what they actually believed, or did. However, unpublished data  based on in-depth interviews with Ontario hospital decision makers suggest self-reported views in relation to fairness were similar in both survey respondents and non-respondents, and that reasons for non-response were related to convenience, workload and priority, and not social desirability. Third, corroborative evidence of 'fairness', obtained through in-depth interviews with staff or community stakeholders, or through review of other relevant information (e.g. meeting minutes, publication of rationales on web sites) was not done. The study design was a survey, focusing on the self-report of fairness from the perspective of Chief Executive Officers, or delegates. In-depth interviews with Chief Executive Officers, or additional individual hospital case studies would be helpful in understanding actual priority setting practices, building on these survey results.
In this first survey of Chief Executive Officers within a health region reporting on their assessment of the fairness of priority setting practices in their institutions according to 'accountability for reasonableness', ample room for improvement was observed, especially for the enforcement condition. Additional study is required to understand how application of 'accountability for reasonableness' will vary within and across institutions, and is shaped or influenced by various parameters, including the nature of corporate leadership. These evaluations can help to identify good practice opportunities for improvement that can be shared between local institutions or used within government to drive health care reforms.
Funding for study provided by a grant from The Change Foundation and by an Interdisciplinary Capacity Enhancement grant from the Canadian Institutes of Health Research. DKM is supported by an Ontario Ministry of Health and Long-Term Care Career Scientist Award. PAS is supported by a Canadian Institutes of Health Research Distinguished Scientist Award.
- McKneally MF, Dickens B, Meslin EM, Singer PA: Bioethics for clinicians: resource allocation. Canadian Medical Association Journal. 1997, 157: 163-67.PubMedPubMed CentralGoogle Scholar
- Ham C: Priority setting in health care: learning from international experience. Health Policy. 1997, 42: 49-66. 10.1016/S0168-8510(97)00054-7.View ArticlePubMedGoogle Scholar
- Wiener CL: The Elusive Quest, Accountability in Hospitals. 2000, New York: Aldine de GruyterGoogle Scholar
- Singer PA, Mapa J: Ethics of resource allocation: Dimensions for Health Care Executives. Hospital Quarterly. 1998, 1: 29-31.PubMedGoogle Scholar
- Daniels N, Sabin JE: Limits to health care: Fair procedures, democratic deliberation and the legitimacy problems for insurers. Philosophy and Public Affairs. 1997, 26: 303-502.View ArticlePubMedGoogle Scholar
- Daniels N, Sabin JE: Ethics of accountability in managed care reform. Health Affairs. 1998, 17: 50-64. 10.1377/hlthaff.17.5.50.View ArticlePubMedGoogle Scholar
- Daniels N: Accountability for reasonableness in private and public health insurance. The Global Challenge of Health Care Rationing. Edited by: Coulter A, Ham C. 2000, Buckingham, U.K.: Open University Press, 89-106.Google Scholar
- Ham C, Pickard S: Tragic Choices in Health Care: The Story of Child B. 1998, London: King's FundGoogle Scholar
- Ham C, McIver S: Contested Decisions: Priority Setting in the NHS. 2000, London: King's FundGoogle Scholar
- Ham C, Robert G, editors: Reasonable Rationing: International Experience of Priority Setting in Health Care. 2003, London: Open University PressGoogle Scholar
- Daniels D, Sabin JE: Setting Limits Fairly: Can we learn to share medical resources?. 2002, Oxford, UK: Oxford University PressView ArticleGoogle Scholar
- Martin DK, Shulman K, Santiago-Sorrell P, Singer PA: Priority setting and hospital strategic planning: a qualitative case study. Journal of Health Services Research & Policy. 2003, 8: 197-201. 10.1258/135581903322403254.View ArticleGoogle Scholar
- Martin DK, Walton N, Singer PA: Priority setting in surgery: improve the process and share the learning. World Journal of Surgery. 2003, 27: 962-966. 10.1007/s00268-003-7100-y.View ArticlePubMedGoogle Scholar
- Mielke J, Martin DK, Singer PA: Priority setting in a hospital critical care unit: qualitative case study. Critical Care Medicine. 2003, 31: 1-5.View ArticleGoogle Scholar
- Deber R, Wiktorowicz M, Leatt P, Champagne F: Technology acquisition in Canadian hospitals: How is it done, and where is the information coming from?. Healthcare Management FORUM. 1994, 7: 18-27.View ArticlePubMedGoogle Scholar
- Kovac M: Rationing of hospital services in the Australian health system. Croatian Medical Journal. 1998, 39: 339-45.PubMedGoogle Scholar
- Bochner F, Martin E, Burgess NG, Somogyi AA: Controversies in treatment: How can hospitals ration drugs? Drug rationing in a teaching hospital: a method to assign priorities. BMJ. 1994, 308: 901-5.View ArticlePubMedPubMed CentralGoogle Scholar
- Norusis MJ: SPSS 10.0, Guide to Data Analysis. 2000, NJ: Prentice-Hall, IncGoogle Scholar
- Strauss A, Corbin J: Basics of Qualitative Research: Techniques and Procedures of Developing Grounded Theory. 1998, Thousand Oaks, CA: Sage Publications, IncGoogle Scholar
- Strauss A, Corbin J: Grounded theory methodology: An overview. Handbook of Qualitative Research. Edited by: Denzin NK, Lincoln YS. 2000, Thousand Oaks, CA: Sage Publications IncGoogle Scholar
- Ubel PA, DeKay ML, Baron J, Asch DA: Cost-effectiveness analysis in a setting of budget constraints: is it equitable?. New England Journal of Medicine. 1996, 334: 1174-77. 10.1056/NEJM199605023341807.View ArticlePubMedGoogle Scholar
- Bowling A: Health care rationing: the public's debate. BMJ. 1996, 312: 670-674.View ArticlePubMedPubMed CentralGoogle Scholar
- Stronks K, Strijbis AM, Wendte JF, Gunning-Schepers LJ: Who should decide? Qualitative analysis of panel data from public, patients, healthcare professionals, and insurers on priorities in health care. BMJ. 1997, 315: 92-96.View ArticlePubMedPubMed CentralGoogle Scholar
- Zweibel NR, Cassel CK, Karrison T: Public attitudes about the use of chronological age as a criterion for allocating health care resources. Gerontologist. 1993, 33: 74-80.View ArticlePubMedGoogle Scholar
- Martin DK, Pater JL, Singer PA: Priority-setting decisions for new cancer drugs: a qualitative case study. Lancet. 2001, 358: 1676-81. 10.1016/S0140-6736(01)06714-9.View ArticlePubMedGoogle Scholar
- Singer PA, Martin DK, Giacomini M, Purdy L: Priority-setting for new technologies in medicine: qualitative case study. BMJ. 2000, 321: 1316-18. 10.1136/bmj.321.7272.1316.View ArticlePubMedPubMed CentralGoogle Scholar
- Mitton C, Donaldson C: Setting priorities in Canadian regional health authorities: a survey of key decision makers. Health Policy. 2002, 60: 39-58. 10.1016/S0168-8510(01)00190-7.View ArticlePubMedGoogle Scholar
- Mendelssohn DC, Boon TK, Singer PA: Referral for dialysis in Ontario. Archives of Internal Medicine. 1995, 155: 2473-78. 10.1001/archinte.155.22.2473.View ArticlePubMedGoogle Scholar
- Norheim FO: Limiting access to allogenic bone marrow transplantation in five European countries: what can we learn about implicit rationing. Health Policy. 2000, 52: 149-56. 10.1016/S0168-8510(00)00079-8.View ArticlePubMedGoogle Scholar
- Hope T, Hicks N, Reynolds DJM, Crisp R, Griffiths S: Rationing and the health authority. British Medical Journal. 1998, 317: 1067-69.View ArticlePubMedPubMed CentralGoogle Scholar
- Martin DK, Giacomini M: Fairness, Accountability for Reasonableness, and the Views of Priority Setting Decision Makers. Health Policy. 2002, 61: 279-290. 10.1016/S0168-8510(01)00237-8.View ArticlePubMedGoogle Scholar
- Reeleder D, Martin DK, Goel V, Singer PA: Role of Leadership in Priority Setting Through the Eyes of Ontario Hospital CEOs.Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/5/8/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.