- Research article
- Open Open Peer Review
Insights about the process and impact of implementing nursing guidelines on delivery of care in hospitals and community settings
BMC Health Services Researchvolume 8, Article number: 29 (2008)
Little is known about the impact of implementing nursing-oriented best practice guidelines on the delivery of patient care in either hospital or community settings.
A naturalistic study with a prospective, before and after design documented the implementation of six newly developed nursing best practice guidelines (asthma, breastfeeding, delirium-dementia-depression (DDD), foot complications in diabetes, smoking cessation and venous leg ulcers). Eleven health care organisations were selected for a one-year project. At each site, clinical resource nurses (CRNs) worked with managers and a multidisciplinary steering committee to conduct an environmental scan and develop an action plan of activities (i.e. education sessions, policy review). Process and patient outcomes were assessed by chart audit (n = 681 pre-implementation, 592 post-implementation). Outcomes were also assessed for four of six topics by in-hospital/home interviews (n = 261 pre-implementation, 232 post-implementation) and follow-up telephone interviews (n = 152 pre, 121 post). Interviews were conducted with 83/95 (87%) CRN's, nurses and administrators to describe recommendations selected, strategies used and participants' perceived facilitators and barriers to guideline implementation.
While statistically significant improvements in 5% to 83% of indicators were observed in each organization, more than 80% of indicators for breastfeeding, DDD and smoking cessation did not change. Statistically significant improvements were found in > 50% of indicators for asthma (52%), diabetes foot care (83%) and venous leg ulcers (60%). Organizations with > 50% improvements reported two unique implementation strategies which included hands-on skill practice sessions for nurses and the development of new patient education materials. Key facilitators for all organizations included education sessions as well as support from champions and managers while key barriers were lack of time, workload pressure and staff resistance.
Implementation of nursing best practice guidelines can result in improved practice and patient outcomes across diverse settings yet many indicators remained unchanged. Mobilization of the nursing workforce to actively implement guidelines and to monitor the delivery of their care is important so that patients may learn about and receive recommended healthcare.
The gap between research evidence and clinical practice is one of the most persistent problems in the provision of quality health care. Approximately 30 to 40% of patients do not receive health care according to current scientific evidence and some patients receive unnecessary or harmful care [1, 2]. In primary care, only 3% to 49% of patients receive recommended care according to a systematic review of studies of quality of care . As one strategy to help health professionals and decision-makers address this gap, professional and government organizations are producing clinical guidelines with credibly appraised summaries of research results and recommendations for health care. The Guidelines International Network (GIN) was founded in 2002 and by 2004 there were 52 member organizations from 27 countries yet only two members were nursing organizations: The Joanna Briggs Institute for Evidence Based Nursing & Midwifery in Australia and the Royal College of Nursing Institute in the United Kingdom . In the United States, The Agency for Healthcare Research and Quality (AHRQ) sponsors the National Guideline Clearinghouse which contains 105 nursing guidelines developed by 14 organizations in Canada, China, the United Kingdom and the United Sates .
One might question whether or not discipline-specific guidelines are required. We take the position that while health care is delivered to patients from a multidisciplinary perspective, the responsibilities of each team member need to be informed by the best available research evidence and some of this evidence is discipline-specific. The nursing profession represents a large proportion of health care providers and a potential workforce to address gaps in the provision of recommended health care.
The effectiveness of guideline implementation in nursing and allied health professions has received relatively little research attention. A Cochrane review summarized the results of 18 intervention studies of which 13 included physicians and nurses, and only 4 specifically targeted nurses . The authors concluded that there is limited evidence supporting the effectiveness of guidelines for improved care in "professions allied to medicine" . A systematic review of research evidence in nursing was conducted but only four studies met the inclusion criteria and the authors concluded that little is known about how to increase research use in nursing .
In contrast, there is a considerable body of physician-oriented and multi-disciplinary research with 235 studies reviewed by Grimshaw and colleagues (2004) of the effectiveness and costs of guideline development, dissemination and implementation strategies . Physicians were the focus in 74% of these studies while the other studies targeted a multidisciplinary team. Results were promising in that eighty-six percent of studies reported improvements in care, however there were substantial variations in the results. Grimshaw and team concluded that further research is needed to support decisions about which guideline dissemination and implementation strategies are likely to be effective. No differences were found in effects according to the number of elements such as reminders, education, audit and feedback although Grimshaw et al. and others have suggested that using multifaceted strategies may be more effective than single interventions since they address multiple barriers to implementation [8–10]. While these systematic reviews help to advance our understanding of the potentially potent ingredients of guideline implementation programs, they summarize evidence from highly controlled research studies with targeted guidelines and pre-planned implementation settings. However, provincial government decision-makers who are determining whether or not to invest in nursing best practice guideline implementation programs, also need evaluation results from naturalistic implementation programs in diverse settings where decisions regarding which guidelines should be implemented and which recommendations from each guideline is a priority for implementation are determined by practitioners rather than by researchers.
In nursing, there are few empirical studies about the quality of care and it is crucial to examine the interrelationships between evidence-based nursing care processes and outcomes . A state of the science review of nursing-sensitive outcomes describes a rapidly developing field yet many unresolved issues including what, when and how to measure indicators of nursing care . Data about nursing practice is needed to answer questions about health human resource requirements and what nurses need to do to maximize their impact on the delivery of health care and patient outcomes.
In summary, there is a very limited evidence base for either the design of nursing guideline implementation interventions or for measuring the process and outcome of nursing care. An opportunity arose with the production of new nursing guidelines for a naturalistic study that examined implementation of various guidelines across settings to provide insights which are often absent from other studies where the protocol is tightly prescribed, only one guideline is implemented and it is implemented across homogeneous settings. The context for the study is described below.
Context: The Registered Nurses' Association of Ontario Initiative
For the past seven years, the Registered Nurses' Association of Ontario (RNAO) in Canada has been leading the development and implementation of nursing best practice guidelines funded by the provincial government. Priority topics are selected using focus groups, surveys and an open web-based invitation to nurses. An expert panel is convened for each topic with representatives from different health care sectors (hospital, public health, long-term care), and responsibilities (administration, education, practice, research). Guidelines undergo extensive review by multidisciplinary stakeholders. Further details about the methodology for guideline development are described elsewhere [13, 14]. To date, 30 guidelines have been published in English with twelve guidelines also translated into French . Each guideline includes recommendations for practice such as patient assessment, patient education, medication monitoring, supportive care and referral to community services
We report on the real-time implementation of six guidelines during 2002–2004: asthma, breastfeeding, delirium-dementia-depression (DDD), foot complications in diabetes, smoking cessation and venous leg ulcers . Specific objectives of this evaluation study were to document the process of best practice guideline implementation by topic, to describe facilitators and barriers to implementation and to determine the impact of indicators related to process and patient outcomes. The evaluation of these six guidelines with respect to patient referrals is reported elsewhere .
A one-year prospective before and after implementation project was conducted with three months allotted for the baseline data collection, six months for implementation and three months for the follow-up data collection. The one-year time-frame was determined by the requirements of the government contract funding. The approach used was to evaluate guideline implementation as it naturally occurred in clinical settings. Ethics approval was received from the University of Ottawa Health and Social Sciences Ethics Review Board and from participating organizations. Following a call for proposals and a peer review process, eleven health care organisations were selected to implement one of the six guidelines. Organizations selected included four acute care teaching hospitals, two community hospitals, one chronic care hospital, one mental health teaching hospital, two home visiting nursing agencies and one regional public health unit. Sites began implementation of four topics in 2002 (asthma, breastfeeding, delirium-dementia-depression, and smoking cessation) and two topics in 2003 (foot complications in diabetes, and venous leg ulcers).
Each site committed in-kind resources including at least 10% of an administrator's time and paid educational time for nurses. Project funding supported the release of one full-time equivalent clinical resource nurse (CRN) for a year which included the implementation as well as the pre-implementation and post-implementation data collection. Some organizations elected to allocate this funding to more than one CRN as described in Table 1.
Initial workshops coordinated by the RNAO were held with three to five key stakeholders from each site (e.g. CRN, manager, administrator). Leaders from each guideline development panel gave structured presentations about the guideline recommendations with the supporting research evidence. A Toolkit [17, 18] with a two-hour training session describing strategies for implementation of clinical practice guidelines was provided. The Toolkit was developed by an expert panel including members who have published on evidence-based nursing practice and theoretically-informed guideline implementation research [19–22]. The Toolkit was based on a model with six essential elements: guideline identification; stakeholder identification, assessment and engagement; environmental readiness; use of effective implementation strategies; evaluation of guideline implementation; and identification of resource requirements .
At each organization, a multidisciplinary steering committee reviewed the Toolkit and planned the implementation of the selected guideline in their setting. As recommended in the Toolkit, an environmental assessment of barriers and facilitators was undertaken, stakeholders were identified and specific implementation strategies were identified in a written action plan. Paid education sessions (range 2–9 hours) were provided by the CRNs for nurses and other health care providers. These sessions were interactive, incorporated problem solving, case studies and skill practice sessions. Table 2 describes implementation strategies including nurses' education, champions or local opinion leaders, reminder systems, policy review, creation of new documentation, multi-disciplinary involvement and patient education. In addition, monthly problem-solving teleconferences with all CRNs were held to discuss ongoing facilitators and barriers to guideline implementation. These teleconferences were led by experienced RNAO-based nursing consultants.
Process and Patient Outcomes
In order to select key outcomes, a rapid three-way consultation occurred with the leader of the specific guideline development panel, decision-makers from each implementation site and a member of the research team. Criteria guiding final selection of outcomes included: outcomes with the strongest research evidence, outcomes feasible to measure in a short time frame, and outcomes relevant to priorities at participating organizations. Chart abstraction forms were adapted from tools used in previous research when available [23–27] and assessed for face validity and relevance by the decision-makers and research team.
Consecutive patient charts that met the inclusion/exclusion criteria were audited pre- and post-implementation. In addition, for one topic (asthma), patient techniques for use of medication inhaler device were observed . In four out of six topics (asthma, breast-feeding, diabetes foot care, and smoking cessation), patients were interviewed in hospital or at home about recent nursing care and again, either four or eight weeks later by telephone, about their access to specialist and community resources . For the topic of delirium-dementia-depression, participating units were concerned about ethical issues and the feasibility of obtaining informed proxy consent from family members within the three-month time frame for baseline data collection. The CRNs described a vulnerable population with many clients unable to give consent, substitute decision-makers difficult to access and some clients with no substitute decision makers. For the venous leg ulcers guideline, the home visiting nurse agency was willing to assist with the chart audit process but declined to participate in obtaining client interviews due to limited time and multiple obligations.
Patient eligibility criteria were defined for each guideline (Table 3). Our goal was to enrol 50 patients on each of the participating units in an eight week time frame. Response rates were calculated using the number of eligible patient charts as the denominator for the initial patient interview in hospital/home. The number of initial patient interviews was then used as the denominator for calculating the response rate for follow-up interviews since only those completing the first interview were eligible to be re-interviewed. Overall missing data were less than 10%.
The data collection process was supervised centrally by an experienced Project Manager who frequently communicated with the CRNs to maintain the quality of data collection. The CRNs extracted pilot data from five to 10 charts at each organization to develop an effective system for obtaining charts and to assess the feasibility of coding categories. Part-time data collectors were then trained and supervised by the CRN locally to conduct the chart audits. The Project Manager reviewed data from all sites on an ongoing basis and contacted local CRNs about missing data or inconsistencies.
Descriptive statistics and bivariate statistics (independent t-tests, Pearson chi-square) were used to describe characteristics of the sample, and to compare before and after measures of clinical outcomes within study sites. Fisher's exact test was used when expected frequencies were small.
Semi-structured interview schedules were developed by the co-investigators and administered to all CRNs, administrators and selected staff nurses immediately after the implementation. The staff nurses (2–3 per unit) were identified by the CRN to include those who had a favourable as well as those who had a less favourable experience with the guideline implementation.
Questions were designed to elicit information about the strategies used as well as the perceived factors that facilitated or provided barriers to guideline implementation. In addition, CRNs were interviewed at the mid-point of implementation to describe priority recommendations selected for implementation and reasons why some recommendations were not selected. Examples of these interview schedules as well as our approach used to conduct and analyze interview data are provided in an available monograph .
The Project Manager and trained interviewers conducted the interviews which were audio-taped with the consent of each respondent. Analysis of the interviews was conducted using QSR-N5. Interviews were initially coded by one individual using a coding guide. Content analysis was conducted looking for prominent themes and patterns across respondents for each guideline .
Chart audits and patient interviews
Completion rates for chart audits were greater than 95% for all eligible cases. Response rates varied for initial and follow-up patient interviews, as displayed in Table 3, with an average of 61.1%. Language barriers and refusals lowered the response rates at the initial interviews. For the follow-up interviews, the response rates decreased due to drop-outs, difficulty contacting patients after discharge, discharge of patients to other facilities and death. In addition, for the asthma guideline, lower response rates were due to an extremely busy emergency department discharging potential patients before they could be interviewed for the study.
As described in Table 4, there were statistically significant improvements across all topics with improvements in more than half of the indicators for asthma (52.4%), diabetes foot care (82.6%) and venous leg ulcers (60.0%). Table 5 displays the details of selected outcomes pre- and post-guideline implementation according to chart audit data.
Overall, with respect to chart audit data, there were statistically significant improvements in nearly half of the indicators (46.7%, 35/75). It is interesting to note that improvements were found in items related to: patient assessment/risk classification 37%; patient education 29%; medication and treatments 14%; symptom monitoring 9%; supportive care 6%; and other 6%. Statistically significant improvements were also found in pre- and post-observation scores of asthma patients' correct use of an inhaler device (p =.014). With respect to patient interview data, it is notable that statistically significant improvements were found in all seven indicators for diabetes foot care as reported in Table 4.
No statistically significant differences were found > 80% of the indicators for three guidelines (breastfeeding, DDD, smoking cessation). In addition, two statistically significant negative results were found for two of these guidelines. Specifically, fewer mothers were exclusively breastfeeding their infants in hospital (80.0% pre-implementation, 67.1% post-implementation; p = 0.012) and fewer mental health patients reported that they had tried to quit smoking post-guideline implementation (66.7% pre-implementation, 33.3% post-implementation; p = .026).
The concordance of nursing practices with indicators measured varied considerably post-implementation. The lowest level was 17.4% for the provision of smoking cessation advice to smokers in a mental health organization and the highest level was 96.2% for the patients in the chronic care hospital and the clients receiving community-based nursing services for the care of their venous leg ulcers (Table 5).
Even in the three guidelines with statistically significant improvements in > 50% of indicators, there were still substantial gaps in the provision of evidence-based recommendations at post-intervention as follows: Only 23.9% of hospital patients with asthma had an action plan at discharge; only 60.5% of hospital and community-based patients with diabetes had a documented assessment of risk factors for foot ulcers; and only 45.7% of patients with venous leg ulcers from long-term care and a community-based nursing service received a Doppler Assessment of Ankle BrachialPressure Index (Table 5).
Participation in the qualitative interviews by the CRNs, nurses and administrators was strong from all units with an overall participation rate of 87.4% (83/95) with a range of 75%–100% per guideline. The proportion of recommendations implemented varied by guideline topic from 43% (e.g. 3/7) to 100% (e.g. 7/7). A very brief synopsis of the content of recommendations is described in Table 4 with a full description of recommendations available . For three guidelines, the CRNs reported at mid-point that their implementation action plan addressed all clinical recommendations (asthma, DDD and diabetes foot care). However, for the other three topics the CRNs explained reasons why certain recommendations were not a priority as described below.
The prenatal components which included a comprehensive assessment to facilitate a breastfeeding plan and small informal group education classes were not implemented because "these are very expensive and would require too much personnel time and it is felt not possible at this time". Also, the recommendation to conduct an in-person follow-up and assessment if a mother and baby were discharged within 48 hours of birth was not implemented due to perceived lack of available public health nurses. Controversy arose about the recommendation for exclusive breastfeeding for the first six months of life from the hospital nutrition/dietetic staff. According to the CRN, these nutritionists followed other national guidelines recommending exclusive breastfeeding for only four months.
The CRN reported that three recommendations were not implemented: provide intensive smoking cessation interventions, implement interventions with pregnant women and encourage smokers and non-smokers to have smoke-free homes. Reasons for non-implementation included lack of time for nurses to do intensive smoking cessation counselling and the availability of experts at a nicotine dependence clinic within the organization. The pregnant women population was not applicable to the participating mental health organization. With respect to the quest for smoke-free homes of patients/clients, the CRN stated "We felt that it was something that we did not have a lot of control over, some clients live in rooming houses and boarding houses and collective group settings."
Venous Leg Ulcers
Of the 56 recommendations, 52 were implemented. Recommendations not implemented related to secondary treatment options including venous surgery, electrical stimulation, hyperbaric oxygen, and therapeutic ultrasound. The CRN explained that "these are second wave treatment option, so the first part of our process has really been trying to educate the nurses around the guidelines, care and so that sort of comes after the initial care."
Participants' perceptions (CRNs, managers, staff nurses) of the most important facilitators were remarkably similar across implementation sites, with the most common including the education sessions, support from champions such as the CRN, APNs, and physicians, organizational and administrative support (e.g., staff replacement time to attend educational sessions), and involvement of multiple stakeholders (Table 6). Participants agreed that the following also facilitated BPG implementation: team work and collaboration, RNAO support, specific resources such as Asthma Education Centre, and availability of supplies (e.g., wound compression supplies, Doppler ultrasounds).
Participant perceptions of the most important barriers to implementation included some common barriers across all topics and sites such as workload and time pressures, competing demands, staff resistance and lack of buy-in, lack of support from other organizational levels, and organizational change. Other barriers were specific to the implementation sites, such as short patient stays, lack of support from physicians, and lack of changed documentation (Table 7). A need for the ongoing analysis of stakeholders' views and barriers was suggested by the CRN "Although considerable time was spent conducting a stakeholder analysis, the committee recognized that as changes occurred, more work was required in analyzing, revising and engaging stakeholders."
After the one-year implementation of newly developed nursing-oriented guidelines, a substantial proportion of patients were not receiving recommended nursing care. While our results reveal that some statistically significant positive improvements were found for all guideline topics, more than 80% of the indicators for breastfeeding, DDD and smoking cessation did not change from pre-implementation to post-implementation. However, there were statistically significant improvements in more than half of the indicators for the asthma, diabetes foot care and venous leg ulcer guidelines. These improvements are notable given the relatively short time frame of one-year for project completion.
Overall, this situation is consistent with findings of other studies of guideline implementation and assessment of the quality of care [1–3, 6, 8]. Perhaps changes in the processes of care and outcomes would have been enhanced if nurses, managers and other disciplines were aware of the small proportion of patients actually receiving guideline-based recommended care. In this study, feedback on the pre-intervention study results was not provided to the agency or staff. Comparative data of nursing practice and patient outcomes is important for health service delivery to determine the quality and quantity of care provided. Systematic reviews of audit and feedback conclude that telling people what they have been doing does impact on change and improve professional practice .
Recent studies of nursing practice including subsequent work of our own suggest that it is possible to obtain data in a reliable and valid way [31–33]. Electronic gathering and dissemination systems that document patients' responses to treatment, obtain real-time outcome feedback and support access to electronic resources including RNAO guidelines using hand-held computers at point of care are promising tools to support guideline implementation .
The implementation strategies used in this study reflect those generally classified as multi-faceted interventions by Grimshaw and colleagues . We observed two unique strategies applied to each of the three guideline implementations resulting in improvements in more than 50% of the indicators. Firstly, these nurses' received an opportunity for hands-on skill practice sessions (trial use of placebo devices for asthma medication, practice sessions for foot assessment of people with diabetes, and practice sessions for bandaging for people with venous leg ulcers). Skill practice has been identified as an element of interactive workshops which may also include role-play or case discussion. Interactive workshops can result in moderately large changes in professional practice compared to didactic lecture-only sessions which are unlikely to change behaviour . However, it is noteworthy that case study discussions were used in education sessions for the three other guidelines without subsequent changes in professional practice (breastfeeding, DDD and smoking cessation). The skill practice component appears to be a vital element and warrants attention.
The second unique implementation element used with the three guidelines with substantial practice changes was the development of patient education toolkits and brochures. The CRN for the asthma guideline suggested that the new patient information tool provided an opportunity to reinforce previous teaching provided to nurses. The CRN for diabetes foot care reported that patients appreciated the education suggesting a positive feedback process from clients may have encouraged implementation of the guideline. It is noteworthy that all seven process and outcome indicators from the patient interviews increased significantly from pre to post-implementation for the topic of diabetes foot care.
For other guideline topics, positive feedback from patient education was less likely in part due to the nature of the guideline topic and in part due to the application in the selected clinical context. While the diabetes guideline was directly applicable to the day-to-day care in the medical hospital in-patient units and the ongoing care of home visiting nurses, the smoking cessation guideline required a considerable shift in the norms at the participating mental health facility where previously, cigarettes had been provided as a positive reward for good patient behaviour. With respect to the breastfeeding guideline, the post-partum nurses did not receive feedback about the impact of their hospital-based teaching because public health nurses assumed care once the mother and infant were discharged from the hospital. The public health nurses did not receive feedback about long-term breast-feeding outcomes because their teaching was concentrated in the early post-partum period. The uptake of guidelines such as breastfeeding or smoking cessation with a preventive focus appears to be slower, a finding that is consistent with Rogers contention that preventive innovations have a particularly slow rate of adoption because individuals have difficulties perceiving the relative advantage and the consequences are distant in the future .
Management support was reported as an important facilitator for all guideline implementations except for breastfeeding. We report in this study and in previous evaluations of RNAO guidelines , that one of the most important facilitators for implementation of a guideline is management support and commitment. In addition, key barriers documented in previous work and in this report include lack of administrative support and changes in management. We note that "leadership support for evidence-based practice culture" is an explicit organizational element of Stetler's conceptual framework  and that leadership is a sub-element of "context" of the Promoting Action on Research Implementation in Health Services (PARIHS) framework . However, management support and leadership are not listed as an implementation strategy in either our implementation Toolkit [17, 18] or with the current EPOC Cochrane systematic reviews of consumer, professional or organizational change. We conceptualise "managerial leadership for research use" as a multidimensional process of influence to enable nurses to use research evidence in clinical practice [40, 41]. An integrative review of 12 published studies found that managerial support, policy revisions, auditing, role modelling and valuing research facilitated nurses' use of research evidence . We suggest that managerial leadership is an important element for guideline implementation in nursing
With respect to process and patient outcomes, we found that none of the organizations had existing patient care databases with sufficient detail or data about nursing process of care to be used in the evaluation of these RNAO guidelines. Thus, we endeavoured to rapidly adapt existing published tools and develop new tools (chart audit, observation and interview questionnaires) to measure the processes of nursing care that are meaningful and actionable by nursing managers and senior administrators. These data collection tools were assessed for content validity and applicability and are available on the web with the RNAO Cycle 3 guidelines  and may provide a launching point for other teams evaluating gaps in nursing care. Subsequent psychometric testing of the asthma observation tool found mean inter-rater reliability indicies of .82 . Other related articles, monographs and user-guides describing measures for evaluating the implementation of nursing best practice guidelines are available [27, 28, 33, 42, 43].
An environmental scan of facilitators and barriers as recommended in the implementation Toolkit [17, 18] was conducted by the CRNs for each guideline implementation in order to plan the intervention strategies in consultation with a multidisciplinary steering committee. However, knowing that a barrier(s) exists is only the first step. There are no quick fixes for many of the barriers identified by CRNs. For example, overcoming nurses' resistance to the breastfeeding guideline recommendations, addressing competing demands for time as reported for the DDD guideline and tackling long-standing issues of client resistance to smoking cessation are all complex issues. Tailored interventions to overcome identified barriers to change have been evaluated in a review of 15 studies with mixed results because it was not clear whether all barriers or important barriers were identified and addressed by pre-selected strategies . Pragmatic evaluations of guideline implementation in nursing are necessary to document important barriers for the design and evaluation of tailored strategies that fit real-world implementation.
A note of caution is required about the evaluation of recommendations that are sensitive to patient preferences and nurses' values. As one report advises , performance measures require attention to avoid defining high testing rates as good quality of care, since they may not take into consideration patient preferences for care options. Patient preferences and nursing values and beliefs are reported to influence evidence-based nursing  and may have contributed to the two statistically significant negative outcomes found in this study. With respect to breastfeeding, the CRN reported that tension existed about the extent to which staff should encourage individual mothers to breastfeed versus elicit mothers' choices to bottle-feed their infant. With respect to smoking cessation, client resistance to smoking cessation was reported as an important barrier by nurses. Evidence-based practice is defined to "integrate the best research evidence with clinical expertise and patient values ." There is a need to identify patient and provider values implicit in guideline recommendations and these values should be reported along with the research evidence for the recommendations .
Lack of time and workload considerations were reported as barriers across all guideline implementation initiatives. Cost considerations such as staffing for increased workload due to new procedures recommended in guidelines are critical for decision-makers , yet few authors have attempted to estimate these costs. In the subsequent Cycle 4 evaluation projects, cost estimates were derived for implementation of the pressure ulcer guideline and this report is available on request from the authors . The cost of implementation during the initial six month start-up phase and in the subsequent 18 months were determined using a balanced score card approach. Education and capital costs were high in the initial period, while patient operating costs accounted for the largest set of expenditures in the latter period. Future studies are needed to compare the cost implications of guideline implementation across settings and among different types of guidelines
Organizations in this evaluation study volunteered to participate and are therefore a self-selected group. Their volunteer status may have stimulated guideline recommendation uptake and we note that these organizations may reflect the early adopters and innovation champions . Indeed the CRNs in our study appeared to possess characteristics of early adopters with well developed interpersonal and negotiating skills, often holding key linking positions in their organization. Other limitations include the lack of concurrent control groups and the lack of inter-rater reliability testing of chart auditors. Thus, we cannot be certain that the observed changes would have happened without the intervention. Future studies with concurrent control groups and thorough process evaluations are recommended. Periodic feedback to CRNs, managers and staff nurses regarding the status of implementation, assessment of ongoing barriers and the design of repeated opportunities to infuse this information in a tailored implementation plan are suggested for future research.
A unique opportunity to study the process and impact of six newly developed nursing guidelines in hospital and community settings existed. We report a real-world implementation project that provides data about promising implementation strategies.
Specifically, these strategies include the opportunity to practice skills in interactive educational sessions for nurses, the development and use of patient educations tools, and the importance of managerial leadership for nurses' use of research-based guidelines. The lessons learned in this evaluation project may be helpful in planning future guideline implementations and in planning future research.
With funding from the Canadian Health Services Research Foundation, the Canadian Institutes of Health Research and the Canadian Nurses Foundation, we are currently conducting follow-up studies to examine the long-term impact of nursing guideline implementation on clinical and process outcomes. This follows from the comments of CRNs and managers who observed that it takes longer than the six months allocated to the intervention to make clinical changes. In addition, our longitudinal study examines the predictors of sustained implementation of guideline recommendations .
In Canada, there is a strong public and political agenda to continuously strive towards high quality health care. The 10-year action plan to strengthen health care includes a priority to share best practices and provide information to make progress transparent to citizens . The nursing profession, a major group of healthcare providers, needs to share intervention strategies and evaluation results to improve health care delivery. Developing, implementing and evaluating best practice guidelines in nursing are essential elements in high quality health care.
Clinical practice guidelines have the potential to improve the quality of care received by patients and thereby improve patient outcomes. Yet there is a lack of evidence about the impact of nursing best practice guidelines and the most effective strategies to implement such guidelines. We found that a multi-faceted intervention for the implementation of nursing guidelines about asthma, breastfeeding, delirium-dementia-depression, reducing foot complications in diabetes, smoking cessation and venous leg ulcers can result in some improvement in practice and patient outcomes across diverse healthcare settings yet many outcomes remained unchanged. Promising elements for future initiatives are identified including interactive education with skill practice sessions, attention to patient education, involvement of managerial leadership and electronic gathering and dissemination systems offering both real-time feedback and access to guidelines.
We acknowledge project funding from the Government of Ontario. Dr. Edwards holds a Nursing Chair funded by the Canadian Health Services Research Foundation, the Canadian Institutes of Health Research and the Government of Ontario. Dr. Davies held a Career Scientist award funded by the Ontario Ministry of Health and Long-Term Care at the time of this study. Dr. Ploeg held an Investigator Award from the Canadian Institutes of Health Research and St. Joseph's Health Care, Hamilton, Ontario. Tazim Virani holds a doctoral fellowship funded by the Ontario Ministry of Health and Long-term Care and awarded by the RNAO. The views expressed in this publication are those of the authors and not those of the RNAO or the Government of Ontario.
Grol R, Wensign M, Eccles M: Improving patient care. The implementation of change in clinical practice. 2005, Edinburgh: Elsevier
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348: 2635-2645. 10.1056/NEJMsa022615.
Seddon ME, Marshall MN, Campbell SM, Roland MO: Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand. Qual Health Care. 2001, 10: 152-158. 10.1136/qhc.0100152...
Ollenschlager G, Marshall C, Qureshi S, Rosenbrand K, Burgers J, Makela M, Slutsky J: Improving the quality of healthcare using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). Qual Saf Health Care. 2004, 13: 455-460. 10.1136/qshc.2003.009761.
Coopey M, Nix MP, Clancy CM: Translating research into evidence-based nursing practice and evaluating effectiveness. J Nurs Care Qual. 2006, 21: 195-202.
Thomas L, Cullum N, McColl E, Rousseau N, Soutter J, Steen N: Guidelines in professions allied to medicine. Cochrane Database of Systematic Reviews. 2000, 10.1002/14651858.. 2
Thompson DS, Estabrooks CA, Scott-Findlay SS, Moore K, Wallin L: Interventions aimed at increasing research use in nursing: a systematic review. Implement Sci. 2007, 2: 15-10.1186/1748-5908-2-15.
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): 1-72.
Davis DA, Thomson MA, Oxman AD, Haynes RB: Changing physician performance a systematic review of the effect of continuing medical education strategies. JAMA. 1995, 274: 700-705. 10.1001/jama.274.9.700.
Wensing M, Van der Weijden T, Grol R: Implementing guidelines and innovations in general practice which interventions are effective?. J Gen Pract. 1998, 48 (427): 991-997.
Maas ML, Delaney C: Nursing process outcome linkage research: issues, current status, and health policy implications. Med Care. 2004, 42 (Suppl 2): 40-48.
Doran D: Nursing-Sensitive Outcomes State of the Science. 2003, Boston: Jones and Bartlett
Grinspun D, Virani T, Bajnok I: Nursing best practice guidelines: The RNAO project. Hosp Q. 2001, 5 (2): 56-60.
Registered Nurses' Association of Ontario: Guideline Methodology Report. Toronto. 2006, [http://www.rnao.org/researchunit/]
Registered Nurses Association of Ontario: Nursing Best Practice Guidelines. [http://www.rnao.org/Page.asp?PageID=861&SiteNodeID=133]
Edwards N, Davies B, Ploeg J, Virani T, Skelly J: Implementing nursing best practice guidelines impact on patient referrals. BMC Nurs. 2007, 6: 4-10.1186/1472-6955-6-4.
Registered Nurses' Association of Ontario: Toolkit: Implementation of clinical practice guidelines. Toronto. 2002, [http://www.rnao.org/Page.asp?PageID=861&SiteNodeID=133]
DiCenso A, Virani T, Bajnok I, Borycki E, Davies B, Graham I, Harrison M, Logan J, McCleary L, Power M, Scott J: A toolkit to facilitate the implementation of clinical practice guidelines in healthcare settings. Hosp Q. 2002, 5: 55-60.
DiCenso A, Guyatt G, Ciliska D: Evidence-Based Nursing: A guide to clinical practice. 2005, St. Louis, MO: Elsevier
Graham ID, Logan J: Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004, 36: 89-103.
Davies B: Sources and models for moving research evidence into clinical practice. JOGN Nurs. 2002, 31: 558-562.
ICEBeRG: Designing theoretically-informed implementation interventions. Implement Sci. 2006, 1: 4-10.1186/1748-5908-1-4.
Lorimer K, Harrison M, Graham I, Friedberg E, Davies B: Venous leg ulcer care How evidence-based is nursing practice?. J Wound Ostomy Continence Nurs. 2003, 30 (3): 132-142.
Lorimer K, Harrison M, Graham I, Friedberg E, Davies B: Assessing venous leg ulcers population characteristics and practices in a home care community. Ostomy Wound Manage. 2003, 49 (5): 32-43.
World Health Organization & Wellstart International: The Baby-Friendly Hospital Initiative, monitoring and reassessment: Tools to sustain progress. 2001, Geneva: World Health Organization
Edwards N, Sims-Jones N, Hotz S, Cushman R: Development and testing components of a multifaceted intervention program to reduce the incidence of smoking relapse during pregnancy and postpartum of both women and their partners. Final Report to Health Canada. 1997, 1-112. (Prepared in both English and French)
Davies B, Danseco E, Cicutto L, Higuchi KS, McConnell H, Edwards N, MacPherson A, Clarke D: Nursing Best Practice Guidelines Evaluation User Guide: Inhaler Device Assessment Tool for Promoting Asthma Control in Children. 2006, University of Ottawa: Nursing Best Practice Research Unit, [http://www.nbpru.ca]
Edwards N, Davies B, Griffin P, Ploeg J, Skelly J, Danseco E, Helliwell B: Evaluation of nursing best practice guidelines Interviewing nurses and administrators. 2004, (CHRU Publication #M04-1), University of Ottawa: Community Health Research Unit, [http://aix1.uottawa.ca/~nedwards/chru/english/pdf/m04.01.pdf]
Miles MB, Huberman AM: Qualitative Data Analysis: An Expanded Sourcebook. 1994, Thousand Oaks, CA: Sage Publications, 2
Jamtvedt G, Young JM, Kristofferson O'Brien M, Oxman A: Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care. 2006, 15: 433-436. 10.1136/qshc.2006.018549.
Doran D, Harrison MB, Laschinger H, Hirdes J, Rukholm E, Sidani S, McGillis Hall L, Tourangeau A, Cranley L: Relationship between nursing interventions and outcome achievements in acute care settings. Res Nurs Health. 2006, 29: 61-70. 10.1002/nur.20110.
Doran D, Harrison MB, Laschinger HS, Hirdes JP, Rukholm E, Sidani S, McGillis Hall L, Tourangeau AE: Nursing-sensitive outcomes data collection in acute care and long-term care settings. Nurs Res. 2006, 55: S75-S81. 10.1097/00006199-200603001-00012.
Edwards N, Danseco E, Heslin K, Ploeg J, Santos J, Stansfield M, Davies B: Development and testing of instruments to assess physical restraint use. Worldviews Evid Based Nurs. 2006, 3: 73-85. 10.1111/j.1741-6787.2006.00056.x.
Doran D, Sidani S: Outcomes-focused knowledge translation A framework for knowledge translation and patient outcomes improvement. Worldviews Evid Based Nurs. 2007, 4: 3-13. 10.1111/j.1741-6787.2007.00073.x.
O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews. 2001, 10.1002/14651858.. Art. No.: CD003030. , 2
Roger EM: Diffusion of innovations. 2003, New York: Free Press, 5
Ploeg J, Davies B, Edwards N, Gifford W, Elliot-Miller P: Factors influencing best-practice guideline implementation: Lessons learned from administrators, nursing staff and project leaders. Worldviews Evid Based Nurs. 2007, 4 (4): 210-219.
Stetler C: Role of organization in translating research into evidence-based practice. Outcomes Manag. 2003, 7: 97-103.
Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Tichen A, Estabrooks C: Ingredients for change revisiting a conceptual framework. Qual Saf Health Care. 2002, 11: 174-180. 10.1136/qhc.11.2.174.
Gifford W, Davies B, Edwards N, Graham I: Leadership strategies to influence the use of clinical practice guidelines. Can J Nurs Leadersh. 2006, 19: 72-88.
Gifford W, Davies B, Edwards N, Griffin P: Managerial leadership for nurses' use of research evidence An integrative review of the literature. Worldviews Evid Based Nurs. 2007, 4 (3): 126-145. 10.1111/j.1741-6787.2007.00095.x.
Edwards N, Davies B, Ploeg J, Dobbins M, Skelly J, Griffin P, Ralphs-Thibodeau : Evaluating best practice guidelines. Can Nurse. 2005, 101: 19-23.
Dobbins M, Davies B, Danseco E, Edwards N, Virani T: Changing nursing practice: Evaluating the usefulness of a best-practice guideline implementation toolkit. Can J Nurs Leadersh. 2005, 18: 34-48.
Shaw B, Cheater F, Baker R, Gillies C, Hearnshaw H, Flottorp S, Robertson N: Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane database of Systematic Reviews. 2005, 20 (3): 10.1002/14651858. Art No.: CD005470.
Walter LC, Davidowitz NP, Heineken PA, Covinsky KE: Pitfalls of converting practice guidelines into quality measures. Lessons learned from a VA performance measurement. JAMA. 2004, 291: 2466-2470. 10.1001/jama.291.20.2466.
Sackett DL, Straus S, Richardson W, Rosenberg R: Evidence-based medicine: How to practice and teach EBM. 2002, London: Harcourt
Schunemann HJ, Fretheim A, Oxman AD: Improving the use of research evidence in guideline development 10. Integrating values and consumer involvement. Health Res Policy Syst. 2006, 4: 22-10.1186/1478-4505-4-22.
Edwards N, Downey A, Griffin P, Lang A, Eldred S, Danseco E, Perrier A: Costing Models for the Implementation of RNAO Best Practice Guidelines. Cycle 4 Technical Report. 2005, University of Ottawa
Davies B, Edwards N, Ploeg J, Virani T, Skelly J, Dobbins M: Determinants of the sustained use of research evidence in nursing. 2006, Final Report: Canadian Health Services Research Foundation & Canadian Institutes for Health Research. Ottawa
Health Canada Health Care Renewal. A 10-year plan to strengthen health care. [http://www.hc-sc.gc.ca/hcs-sss/delivery-prestation/fptcollab/2004-fmm-rpm/index_e.html]
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/29/prepub
We acknowledge the leadership of Doris Grinspun at RNAO for the nursing best practice guideline initiative. We thank evaluation team collaborators Maureen Dobbins and Jennifer Skelly from McMaster University and Heather McConnell from RNAO. We are grateful for assistance from research staff including Diana Ehlers, Barbara Helliwell, Marilyn Kuhn, Elana Ptack, and Isabelle St. Pierre.
We especially thank the participating health care organizations and the following clinical resource nurses: Genevieve Courant, Sandy Haist, and Karen Ray.
All authors declare that they have no competing interests. Tazim Virani was contracted by the RNAO to lead the overall initiative and work in collaboration with the other authors in pilot testing and evaluating the best practice guidelines.
BD and NE were Co-Principal Investigators and conceived and designed the evaluation of this project. TV led the implementation strategies, reviewed the action plans of the Clinical Resource Nurses and was involved in providing input in draft measures and protocols. BD, NE, JP designed data collection measures and protocols. All authors contributed to the interpretation of data and the preparation of this manuscript.