- Study protocol
- Open Open Peer Review
A cluster randomized controlled trial of a clinical pathway for hospital treatment of heart failure: study design and population
BMC Health Services Researchvolume 7, Article number: 179 (2007)
The hospital treatment of heart failure frequently does not follow published guidelines, potentially contributing to the high morbidity, mortality and economic cost of this disorder. Consequently the development of clinical pathways has the potential to reduce the current variability in care, enhance guideline adherence, and improve outcomes for patients. Despite enthusiasm and diffusion, the widespread acceptance of clinical pathways remain questionable because very little prospective controlled data demonstrated their effectiveness. The Experimental Prospective Study on the Effectiveness and Efficiency of the Implementation of Clinical Pathways was designed in order to conduct a rigorous evaluation of clinical pathways in hospital treatment of acute heart failure. The primary objective of the trial was to evaluate the effectiveness of the implementation of clinical pathways for hospital treatment of heart failure in Italian hospitals.
Two-arm, cluster-randomized trial. 14 community hospitals were randomized either to arm 1 (clinical pathway: appropriate use of practice guidelines and supplies of drugs and ancillary services, new organization and procedures, patient education, etc.) or to arm 2 (no intervention, usual care). 424 patients sample (212 in each group), 80% of power at the 5% significance level (two-sided). The primary outcome measure is in-hospital mortality. We will also analyze the impact of the clinical pathways comparing the length and the appropriateness of the stay, the rate of unscheduled readmissions, the customers' satisfaction and the costs treating the patients with the pathways and with the current practice along all the observation period. The quality of the care will be assessed by monitoring the use of diagnostic and therapeutic procedures during hospital stay and by measuring key quality indicators at discharge.
This paper examines the design of the evaluation of a complex intervention. Since clinical pathways are made up of various interconnecting parts we have chosen the cluster-randomized controlled trial because is widely accepted as the most reliable method of determining effectiveness when measuring cost-effectiveness in real practice.
ClinicalTrials.gov ID [NCT00519038]
In Europe approximately 5% of all acute medical admissions relate to heart failure and in the United States heart failure is responsible for almost 1 million hospitalizations annually. Almost three quarters of these admissions are unplanned and worsening heart failure is responsible for half of these admissions [1, 2].
The Acute Decompensated Heart Failure National Registry (ADHERE) showed that the hospital treatment of heart failure frequently does not follow published guidelines or conform to the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) core performance measures, potentially contributing to the high morbidity, mortality and economic cost of this disorder [3, 4]. ADHERE findings also suggested that the wide variations in conformity may reflect differences in training, guideline familiarity, and implementation of tools and systems to ensure that recommended care is provided and documented. Consequently the development of educational and quality improvement programs has the potential to considerably reduce the current variability in care, enhance guideline adherence, and improve outcomes for patients .
Clinical pathways has become a popular tool to achieve such goals [6–8]. Clinical pathways are a methodology for the mutual decision making and organization of care for a well-defined group of patients during a well-defined period with the aim to enhance the quality of care by improving patient outcomes, promoting patient safety, increasing patient satisfaction, and optimizing the use of resources. They are also developed by multi-professional teams . Despite enthusiasm and diffusion, the widespread acceptance of clinical pathways remain questionable because very little prospective controlled data demonstrated their effectiveness [10–12].
The Experimental Prospective Study on the Effectiveness and Efficiency of the Implementation of Clinical Pathways was designed in order to conduct a rigorous evaluation of clinical pathways in hospital treatment of decompensated heart failure.
The primary objective of the trial was to evaluate the effectiveness of the implementation of clinical pathways for hospital treatment of heart failure among a sample of Italian hospitals. Our hypothesis was that the clinical pathways should be more effective than usual care in treating patients admitted in hospital for heart failure and that the clinical pathways should reduce patients' mortality during the stay and that they should improve patients' outcomes at discharge.
Secondary objectives were to estimate the efficiency and the appropriateness of the use of the resources associated with the intervention and its effects on other relevant outcomes.
The third objective was to define a statistical model able to predict in-hospital death and unscheduled re-admission.
A follow up study to evaluate the effectiveness of the intervention after three years from baseline was described in a separate protocol.
The Experimental Prospective Study on the Effectiveness and Efficiency of the Implementation of Clinical Pathways was promoted and funded by the Italian Ministry of Health (Special Programs art. 12 bis D.lgs 229/99) and Marche Region. The study's Steering Committee defined the study's objectives, clinical topics, scheduling and design. The Regional Healthcare Agency of Marche Region coordinated and gave administrative support to the project and handled patients' data according to the Italian Data Protection act.
We performed a cluster multi-centre randomized controlled clinical trial to evaluate the effect of applying clinical pathways to process and outcome indicators and to the costs sustained to assist the patients with heart failure. We compared the results obtained treating the patients with clinical pathways to the results obtained with the usual care. Since a clinical pathway is not a single intervention to be compared with a placebo but its eventual benefits come from a mix of complex actions that are implemented at the institutional level (appropriate use of practice guidelines and supplies of drugs and ancillary services, new organization and procedures, patient education, etc.), we randomly assigned hospitals, rather than individual patients, to either introduce the pathway or continue usual care [13–17].
Forty hospitals based in four Italian Regions were invited to participate to the study (Figure 1). Eighteen hospitals showed interest in the implementation of the clinical pathway for the hospital treatment of heart failure and were assessed for eligibility. Of the eighteen hospitals that asked to implement the clinical pathway for heart failure we selected and randomized fourteen community hospitals. We based the selection on the comparability of their location, patient population, facilities and teaching status (mean bed size of the hospitals assigned to clinical pathways was 245, in the usual care group was 262).
One hospital was excluded because it did not match with the inclusion criteria (it was a national institute specialized in geriatric hospital care and research) and one hospital withdrew after the project kick off meeting on a decision of the hospital management. To participate to the study the administrations of the hospitals had to be allow the institution to be allocated to either of the two strategies (clinical pathway or current practice) for a 1-year period and to agree not to implement a clinical pathway for the treatment of heart failure if assigned to the usual care group. Two hospitals could not assure do not implement a pathway if assigned to the control group and therefore were not sampled.
The primary outcome measure is in-hospital mortality. We will also analyze the impact of the clinical pathways comparing the length and the appropriateness of the stay, the rate of unscheduled readmissions, the customers' satisfaction and the costs treating the patients with the pathways and with the current practice along all the observation period. The quality of the care will be assessed by monitoring the use of diagnostic and therapeutic procedures during hospital stay and by the use of key quality indicators at discharge, as reported in previous studies [18–22]. The list of process and outcome indicators is reported in Table 1.
The sample included all the patients treated by the hospitals during the experimental period with a principal diagnosis of heart failure (all ICD-9CM codes included in 428.xx code). We calculated the sample size needed to detect a statistical difference in the mortality rate. Since in Italy the in-hospital mortality rates range from 5% to 17%, we expected that clinical pathways succeeded to control mortality to 5% to be clinically relevant [23–25]. Based on this goal a sample size of 424 patients (212 in each group) was required for the study to have 80% power at the 5% significance level (two-sided). The sample size calculation was performed according to standard criteria for cluster randomized trials. We adjusted the sample size using an inflation factor of 1.51 to account for the cluster randomization: 7 clusters per trial arm, cluster size of 30 patients, ICC of 0.018 [26–29].
At baseline we verified the comparability of the two groups at the admission measuring patients' age, sex, co-morbidities, risk factors and symptoms severity (Table 2). Patients with a current AMI or unstable angina were excluded from the study.
The project started at each hospital with a ground round that showed the project protocol. Each hospital was assigned one methodological leader by the study Steering Committee (physicians or nurses with at least a two years experience with clinical pathways) that assisted local multidisciplinary teams in the development of the pathways and in the project implementation. The composition of each team was different in each institution and included general hospital-based physicians, cardiologists, epidemiologists, pathologists, psychologists, nurses, hospital-based pharmacists, social workers and administrative peoples. The teams were formed on a voluntary base, received a 3 days training in the development of clinical pathways and constructed the clinical pathways over a 6-month period.
The teams analyzed their care processes, did research for the best evidences and defined the appropriate goals to satisfy the multidimensional needs of the patients. These results were detailed into protocols and documentation, including the sequence of events and expected progress of the patients over time. The tasks for each professional were defined according to the following care categories: patients' evaluation; education of patients and families; planning of discharge; diagnostic exams; interventions and procedures; consultancies; medical treatments; nutrition; patients' safety [6, 11].
The clinical pathways were analyzed by the EBM unit of the Regional Healthcare Agency of Marche and they were judged consistent with current recommendations for the diagnosis and the treatment of heart failure. After the validation of the pathways each team educated in its hospital the staff to the use of the clinical pathway and monitored the use of the pathway.
Data were prospectively collected by local staff both in intervention and in control groups (physician and nurses who were trained in two pre-study educational events). We did not use incentives for the local staff.
The analysis will be performed by the research team. In addition to common descriptive statistics (Fisher exact and Kruskal Wallis test for categorical and continuous variables, respectively), that will be performed at the cluster level, the differences in the rate of in-hospital deaths and unscheduled admissions across groups and according to each variable under study will be evaluated using random-effects logistic regression, thus accounting for the clustering effect [30–33]. Variables will be included if significant at the 0.10 level (backward approach), with the exception of age which will be forced to entry. The presence of multicollinearity, interaction and higher power terms will be assessed to check final model validity. Patients who died during the study will be excluded from the regression model evaluating unscheduled readmissions because they could not be re-admitted.
Statistical significance will be defined as a two-sided p-value < 0.05. All analyses will be intention-to-treat and will be carried out using STATA statistical software, version 8.2 (Stata Corporation, College Station, Texas, 2003).
The project was exempt from ethical clearance according to the Italian Ministry of Health law number (ex art. 12bis D.lgs 229/99). Moreover the aim of the study is to improve quality of care through clinical pathways and thus should not imply any risk for the patients affected by the study. It is difficult to imagine that our intervention based on better evidences and appropriate use of technologies and drugs could worsen the quality of care when compared to usual care. So according to other experiences dealing with clinical pathways or implementation of evidence based guidelines in practice we think that a Committee of Research Ethic would not consider it necessary to submit the protocol for approval [34, 35].
Even though randomized controlled trials are widely accepted as the most reliable method of determining effectiveness, clinical pathways has not been studied sufficiently in this way. Typically controlled trial design is not used in evaluating clinical pathways because the context level adaptation, which is essential for pathways to work, is perceived as inappropriate in the trial design, likely for the difficulty of keeping replicable and recognizable the intervention . According to Hawe we think that a controlled trial design is appropriate to evaluate clinical pathways as well as other complex interventions and that it is possible to standardize the intervention (the clinical pathways) effectively. To this purpose we defined as standard the steps in the change process or the key functions that the elements of the intervention were meant to improve according to each context. Also the definition of the quality care indicators helped. The indicators were driven by the theory and concerned the functions provided by the key elements of the intervention that were based on expected adherence to the same evidences. We think that this strategy based on combining local change standards to the use of shared evidence based indicators kept the integrity of the intervention in each site.
Since clinical pathways are made up of various interconnecting parts we have chosen the cluster-randomized controlled trial design because is widely accepted as the most reliable method of determining effectiveness in Health Services Research . As reported in sample size literature in cluster-randomized controlled trial we had difficulties in defining the sample size for the dual nature of the trial that focused both on individuals and clusters. The sample size calculation was based on the number of individuals needed, while the randomization process was based on clusters. Since each additional cluster represented a large proportionate increase in the study size (and in associated costs) it was necessary to find the proper balance between the need to increase the number of the clusters and its organizational costs. According to cluster design the sample size calculation needed to predict not only the expected effect size, but also the anticipated cluster size and ICC and this was difficult because the lack of published data on clinical pathways. However we think that the number of clusters and of individuals included in our sample assured the viability of the trial [27, 29, 37, 38].
We had further difficulties related to current Italian healthcare information systems, above all in the method of documenting and collecting data from current sources (clinical records, paper based abstraction tools, etc.). We think that with a more comprehensive information structure for the clinical pathways, data collection would be easier even though the actual process did not affect the quality of the data but only reduced the efficiency of its collection.
Acute Decompensated Heart Failure National Registry
Joint Commission on Accreditation of Healthcare Organizations
Appropriateness Evaluation Protocol
Activity Based Costing
Left ventricular function
International Classification Diseases 9th revision Clinical Modification
Intra Cluster Correlation
Acute Myocardial Infarction
Evidence Based Medicine.
Cowie MR, Fox KF, Wood DA, Metcalfe C, Thompson SG, Coats AJ, Poole-Wilson PA, Sutton GC: Hospitalization of patients with heart failure. A population-based study. Eur Hearth J. 2002, 11: 877-885. 10.1053/euhj.2001.2973.
American Heart Association American Stroke association. Hearth Disease and Stroke Statistics. [http://www.americanheart.org/downloadable/heart/1105390918119HDSStats2005Update.pdf]
Adams KF, Fonarow GC, Emerman CL, LeJemtel TH, Costanzo MR, Abraham WT, Berkowitz RL, Galvao M, Horton DP, ADHERE Scientific Advisory Committee and Investigators: Characteristics and outcomes of patients hospitalized for heart failure in the United States: rationale, design, and preliminary observations from the first 100,000 cases in the Acute Decompensated Heart Failure National Registry (ADHERE). Am Heart J. 2005, 149: 209-216. 10.1016/j.ahj.2004.08.005.
Yancy CW, Lopatin M, Stevenson LW, De Marco T, Fonarow GC, ADHERE Scientific Advisory Committee and Investigators: Clinical presentation, management, and in-hospital outcomes of patients admitted with acute decompensated heart failure with preserved systolic function: a report from the Acute Decompensated Heart Failure National Registry (ADHERE) Database. J Am Coll Cardiol. 2006, 47: 76-84. 10.1016/j.jacc.2005.09.022. Erratum in: J Am Coll Cardiol 2006, 47: 1502
Fonarow GC, Yancy CW, Heywood JT, ADHERE Scientific Advisory Committee, Study Group, and Investigators: Adherence to heart failure quality-of-care indicators in US hospitals: analysis of the ADHERE Registry. Arch Intern Med. 2005, 165: 1469-1477. 10.1001/archinte.165.13.1469.
Pearson SD, Goulart-Fisher D, Lee TH: Critical pathways as a strategy for improving care: problems and potential. Ann Intern Med. 1995, 123: 941-948.
Campbell H, Hotchkiss R, Bradshaw N, Porteous M: Integrated care pathways. BMJ. 1998, 316 (7125): 133-137.
Every NR, Hochman J, Becker R, Lopecky S, Cannon CP: Critical Pathways. A Review. Circulation. 2000, 101: 461-465.
European Pathways Association (EPA). Clinical/Care Pathways. [http://www.e-p-a.org/000000979b08f9803/index.html]
Renholm M, Leino-Kilpi H, Suominen T: Critical pathways. A systematic review. J Nurs Adm. 2002, 32: 196-202. 10.1097/00005110-200204000-00008.
Panella M, Marchisio S, Di Stanislao F: Reducing clinical variations with clinical pathways: do pathways work?. Int J Qual Health Care. 2003, 15: 509-521. 10.1093/intqhc/mzg057.
Dy SM, Garg P, Nyberg D, Dawson PB, Pronovost PJ, Morlock L, Rubin H, Wu AW: Critical pathway effectiveness: assessing the impact of patient, hospital care, and pathway characteristics using qualitative comparative analysis. Health Serv Res. 2005, 40: 499-516. 10.1111/j.1475-6773.2005.0r370.x.
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P: Framework for design and evaluation of complex interventions to improve health. Br Med J. 2000, 321: 694-696. 10.1136/bmj.321.7262.694.
Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG: A Controlled Trial of a Critical Pathway for Treatment of Community-Acquired Pneumonia. JAMA. 2000, 283: 749-755. 10.1001/jama.283.6.749.
Loeb MB: Application of the development stages of a cluster randomized trial to a framework for valuating complex health interventions. BMC Health Serv Res. 2002, 2: 13-21. 10.1186/1472-6963-2-13.
Ferri M, De Luca A, Giorgi Rossi P, Lori G, Guasticchi G: Does a pre-hospital emergency pathway improve early diagnosis and referral in suspected stroke patients? – Study protocol of a cluster randomised trial [ISRCTN41456865]. BMC Health Serv Res. 2005, 5: 66-75. 10.1186/1472-6963-5-66.
Loeb M, Carusone SC, Goeree R, Walter SD, Brazil K, Krueger P, Simor A, Moss L, Marrie T: Effect of a clinical pathway to reduce hospitalizations in nursing home residents with pneumonia: a randomized controlled trial. JAMA. 2006, 295: 2503-2510. 10.1001/jama.295.21.2503.
Luthi JC, Flanders WD, Pitts SR, Burnand B, McClellan WM: Outcomes and the quality of care for patients hospitalized with heart failure. Int J Qual Health Care. 2004, 16: 201-210. 10.1093/intqhc/mzh040.
Bonow RO, Bennett S, Casey DE, Ganiats TG, Hlatky MA, Konstam MA, Lambrew CT, Normand SL, Pina IL, Radford MJ, Smith AL, Stevenson LW, Bonow RO, Bennett SJ, Burke G, Eagle KA, Krumholz HM, Lambrew CT, Linderbaum J, Masoudi FA, Normand SL, Ritchie JL, Rumsfeld JS, Spertus JA, American College of Cardiology, American Heart Association Task Force on Performance Measures (Writing Committee to Develop Heart Failure Clinical Performance Measures), Heart Failure Society of America: ACC/AHA clinical performance measures for adults with chronic heart failure: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures (Writing Committee to Develop Heart Failure Clinical Performance Measures) endorsed by the Heart Failure Society of America. J Am Coll Cardiol. 2005, 46: 1144-1178. 10.1016/j.jacc.2005.07.012.
Spertus JA, Eagle KA, Krumholz HM, Mitchell KR, Normand SL: American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. J Am Coll Cardiol. 2005, 45: 1147-1156. 10.1016/j.jacc.2005.03.011.
Joint Commission on Accreditation of Healthcare Organizations. Specifications Manual for National Hospital Quality Measures 2006. [http://www.jointcommission.org/PerformanceMeasurement/PerformanceMeasurement/Historical+NHQM+manuals.htm]
Fonarow GC, Abraham WT, Albert NM, Stough WG, Gheorghiade M, Greenberg BH, O'Connor CM, Pieper K, Sun JL, Yancy C, Young JB, OPTIMIZE-HF Investigators and Hospitals: Association between performance measures and clinical outcomes for patients hospitalized with heart failure. JAMA. 2007, 297: 61-70. 10.1001/jama.297.1.61.
Grigioni F, Carinci V, Favero L, Bacchi Reggiani L, Magnani G, Potena L, Barbieri A, Magelli C, Branzi A, Magnani B: Hospitalization for congestive heart failure: is it still a cardiology business?. Eur J Heart Fail. 2002, 4: 99-104. 10.1016/S1388-9842(01)00204-5.
Anselmino M, Petrelli A, Gnavi R, Bobbio M, Trevi G: [Hospital admissions for congestive heart failure in Piedmont, Italy: 1996-2001 trend]. Ital Heart J Suppl. 2005, 6 (1): 42-52. [Article in Italian]
Tavazzi L, Maggioni AP, Lucci D, Cacciatore G, Ansalone G, Oliva F, Porcu M, Italian survey on Acute Heart Failure Investigators: Nationwide survey on acute heart failure in cardiology ward services in Italy. Eur Heart J. 2006, 27: 1207-1215. 10.1093/eurheartj/ehi845.
Murray DM: Design and Analysis of Group-Randomized Trials. 1998, New York, NY: Oxford University Press Inc
Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ: Methods for evaluating area-wide and organization-based interventions in health and health care: a systematic review. Health Technol Assess. 1999, 3 (5).
Campbell MK, Mollison J, Grimshaw JM: Cluster trials in implementation research: estimation of intracluster correlation coefficients and sample size. Stat Med. 2001, 20: 391-399. 10.1002/1097-0258(20010215)20:3<391::AID-SIM800>3.0.CO;2-Z.
Yudkin PL, Moher M: Putting theory into practice: a cluster randomized trial with a small number of clusters. Stat Med. 2001, 20: 341-349. 10.1002/1097-0258(20010215)20:3<341::AID-SIM796>3.0.CO;2-G.
Medical Research Council: Cluster randomized trials: methodological and ethical considerations. 2002, London: Medical Research Council
Lee DS, Austin PC, Rouleau JL, Liu PP, Naimark D, Tu JV: Predicting mortality among patients hospitalized for heart failure: derivation and validation of a clinical model. JAMA. 2003, 290: 2581-2587. 10.1001/jama.290.19.2581.
Fonarow GC, Adams KF, Abraham WT, Yancy CW, Boscardin WJ, ADHERE Scientific Advisory Committee, Study Group,and Investigators: Risk stratification for in-hospital mortality in acutely decompensated heart failure: classification and regression tree analysis. JAMA. 2005, 293: 572-580. 10.1001/jama.293.5.572.
Gheorghiade M, Abraham WT, Albert NM, Greenberg BH, O'Connor CM, She L, Stough WG, Yancy CW, Young JB, Fonarow GC, OPTIMIZE-HF Investigators and Coordinators: Systolic blood pressure at admission, clinical characteristics, and outcomes in patients hospitalized with acute heart failure. JAMA. 2006, 296: 2217-2226. 10.1001/jama.296.18.2217.
Perria C, The IMPLEMEG Study Group: Strategies for the introduction and implementation of a guideline for the treatment of type 2 diabetics by general practitioners (GPs) of the Lazio region of Italy (IMPLEMEG study): protocol for a cluster randomised controlled trial [ISRCTN80116232]. BMC Health Serv Res. 2004, 4: 13-20. 10.1186/1472-6963-4-13.
Fretheim A, Oxman AD, Treweek S, Bjorndal A: Rational Prescribing in Primary Care (RaPP-trial). A randomised trial of a tailored intervention to improve prescribing of antihypertensive and cholesterol-lowering drugs in general practice [ISRCTN48751230]. BMC Health Serv Res. 2003, 3: 5-13. 10.1186/1472-6963-3-5.
Hawe P, Shiell A, Riley T: Complex interventions: how "out of control" can a randomised controlled trial be?. Br Med J. 2004, 328: 1561-1563. 10.1136/bmj.328.7455.1561.
Campbell MJ, Donner A, Klar N: Developments in cluster randomized trials and Statistics in Medicine. Stat Med. 2007, 26: 2-19. 10.1002/sim.2731.
Donner A, Klar N: Pitfalls of and controversies in cluster randomization trials. Am J Public Health. 2004, 94: 416-422.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/7/179/prepub
Members of The Heart Failure Study Group were Anna Apicella, Lorenzo Dardanelli, Marialuisa Demarchi, Pierluigi Fraternali, Roberto Gandolfo, Giuseppe Noto, Domenico Tangolo.
The author(s) declare that they have no competing interests.
MP conceived and developed this study and wrote the manuscript. SM assisted in the cluster creation, in defining the indicator set and contributed to the manuscript. AG helped to design the study and to the manuscript. FDS gave input to the project, overviewed all the steps of the study design and did the final review of the manuscript. All authors read and approved the final manuscript.