- Research article
- Open Access
- Open Peer Review
Assessing feasibility and acceptability of study procedures: getting ready for implementation of national stroke guidelines in out-patient health care
BMC Health Services Researchvolume 15, Article number: 517 (2015)
Even though Swedish national guidelines for stroke care (SNGSC) have been accessible for nearly a decade access to stroke rehabilitation in out-patient health care vary considerably. In order to aid future interventions studies for implementation of SNGSC, this study assessed the feasibility and acceptability of study procedures including analysis of the context in out-patient health care settings.
The feasibility and acceptability of recruitment, observations and interviews with managers, staff and patients were assessed, as well as the feasibility of surveying health care records.
To identify patients from the the hospitals was feasible but not from out-patient care where a need to relieve clinical staff of the recruitment process was identified. Assessing adherence to guidelines and standardized evaluations of patient outcomes through health care records was found to be feasible and suitable assessment tools to evaluate patient outcome were identified. Interviews were found to be a feasible and acceptable tool to survey the context of the health care setting.
In this feasibility study a variety of qualitative and quantitative data collection procedures and measures were tested. The results indicate what can be used as a set of feasible and acceptable data collection procedures and suitable measures for studying implementation of stroke guidelines in an out-patient health care context.
Stroke care and rehabilitation
The Swedish National Guidelines for Stroke Care (SNGSC) targeted in this study are based on the principles of equal care, greatest support to those in greatest need and cost effectiveness, and provide a systematic review of current scientific knowledge for policy makers, managers and health practitioners [1, 2]. Thus, the SNGSC may facilitate the equal allocation of health care resources and to support a high standard of health care . Nevertheless, a national survey of stroke care in Sweden identified geographical inequalities in access to stroke care . Although the SNGSC states that home based rehabilitation is a top priority , in some regions less than 10 % received rehabilitation interventions in their home after stroke onset . Moreover, the mean percentage of patients in Sweden that expressed unmet needs of rehabilitation during the first year after stroke was as high as 41 % and reached 53 % in certain regions . These findings indicate the necessity of bridging the gap between evidence-based knowledge and practice, and the need to explore ways to successfully facilitate the implementation of the national guidelines in clinical out-patient settings.
Implementing guidelines in clinical practice
Multiple strategies directed at health professionals, such as written information, education, and audit and feedback are often used to facilitate implementation of guidelines in the clinical context, however their results are generally moderate . In England and Canada, with a health care organisation comparable to the Swedish, user involvement was used to support implementation of stroke guidelines. A Delphi process used to develop a consensus document for implementation and focus groups and interviews preceded the development of user-friendly information regarding interventions in clinical practice. Semi-structured interviews were used to assess the context where the guidelines, e.g. a complex pathway for stroke rehabilitation, were to be implemented . Individuals within an organization can positively or negatively affect an implementation process  but it has been argued that the context in which the evidence is to be implemented plays a crucial role in implementation of evidence . In the conceptual framework Promoting Action on Research Implementation in Health Services (PARIHS), context is included as a key factor comprising aspects such as organizational culture, evaluation of performance, and leadership [11, 12]. The extent to which leaders can facilitate change has been found to be related to their ability to tailor actions to fit the nature of the evidence and contextual conditions . Sweden is structured in local authorities, comprising 20 regions and 290 municipalities responsible for providing health care and welfare for the residents . The regional organization of health care may differ as the local and regional health care authorities are mandated to allocate resources and organize health care autonomously . At the Swedish health care units, the senior managers are responsible for enabling provision of safe and cost effective health care of good quality . Another commonality is that outpatient rehabilitation post stroke should be initiated through a referral, by the stroke unit at the hospital to out-patient units. So far, little is known about how such contextual factors impact the implementation of guidelines in health care. Thus, to capture the process of implementing guidelines within out-patient health care, aspects of context such as managers’ leadership, the use of evaluation to assess performance and the culture of the health care setting need to be studied. Other crucial factors to be targeted, recognized by the PARIHS framework, are the nature of the evidence and how the implementation of evidence is facilitated .
Implementation studies and other complex interventions should be preceded by pilot studies [16, 17] and as proposed by Feeley et al. , feasibility and acceptability with regard to means and measures in data collection and analysis need to be captured. Thus, prior to a proposed full-scale study on a leadership intervention supporting managers in implementing guidelines, such as the SNGSC, the aim of this study was to explore the feasibility and acceptability of study procedures in data collection including a survey of the context of the health care setting. In detail this paper reports on:
the feasibility of:
recruiting units, managers and staff
surveying health care contexts
surveying rehabilitation interventions
the acceptability of data collection procedures, as perceived by managers, staff and patients.
In addition, a parallel paper reports on the results of piloting an intervention supporting the managers in the implementation of SNGSC.
This paper reports on a feasibility study, including both qualitative and quantitative data collection and analysis.
Five out-patient units were identified to assess the feasibility of surveying diverse health care settings: 2 units were operating in a geographical area within a southern urban region (with a population of 225 000 inhabitants) and 3 units were operating in a geographical area within a mid-Sweden rural region (with a population of 10 000 inhabitants). The provision of out-patient stroke rehabilitation after discharge was organized differently in these settings. The included rehabilitation units were a) part of a hospital organization (2 units), b) part of a district health care unit, c) part of the municipality health and welfare and d) a self-sufficient rehabilitation unit. Four units provided home-based rehabilitation only and 3 units provided both home- and clinic-based rehabilitation.
Recruitment of units, managers, staff and patients
First, the senior managers at the identified units were contacted for informed consent. After approval, each unit was requested to include a senior manager and a front line manager. Moreover, occupational therapists and physiotherapists providing rehabilitation interventions for patients with stroke (the 2 latter hereafter referred to as “staff”) were contacted as this feasibility study focused on 3 recommendations in the SNGSC involving interventions made by occupational therapists and/or physiotherapists (Table 1). All managers and staff received written and verbal information about the study and gave their informed consent.
Within a given time period of 2 months (Fig. 1), staff was instructed to initiate a consecutive recruitment of all patients who were referred to the units due to stroke-related needs for rehabilitation interventions within their first year after stroke onset. In close conjunction to their initial meeting with a patient, the staff was responsible for briefly presenting the study and for asking the patient if a data collector from the research team might contact him/her. Contact was made only by approval, and included verbal and written information. In case of limitations in communication due to aphasia a significant other was contacted and in case of language barriers registered interpreters were available. After consent, the patients were included in the study to be assessed with standardized measures. Among the included patients the staff identified patients who were planned for interventions by an occupational therapist and/or a physiotherapist and who could communicate in Swedish, the official language in Sweden. These patients were asked to participate in interviews and observations of provided rehabilitation interventions. The study was approved by the Regional Ethical Review Board in Stockholm (Regionala Etikprövningsnämnden i Stockholm).
The process applied to collecting data is presented in Fig. 1. Data collection followed a standard procedure and was performed by 2 registered physiotherapists/researchers (1 in the rural and 1 in the urban area) who communicated throughout the data collection process to assure consistency.
Surveying the health care contexts and participation in data collection procedures through interviews
All interviews were semi-structured and based on revised versions of previously tested interview guides . To survey the health care context, from a manager and staff perspective, the PARIHS framework’s elements ‘leadership’, ‘culture’, ‘evaluation’, ‘evidence’ (in general and the SNGSC in particular), and ‘facilitation’ of knowledge implementation , were included in the interview guides used in a first interview with managers and staff. A second interview included questions addressing the feasibility and acceptability of the data collection procedures  used in this study (Fig. 1). Interview guides are provided in additional files (staff, Additional file 1 and Additional file 2, managers, Additional file 3).
To survey perceived conditions for implementation from a patient perspective, the interview guides for patients included questions related to their perceived functioning and the rehabilitation interventions they were receiving. Additional questions addressed the feasibility and acceptability of participating in the data collection including observations during a training session and in the research interview (Fig. 1). An interview guide is provided inadditional file (patients, Additional file 4).
Assessing the patient recruitment process using data from the stroke units at the hospitals
To assess the feasibility of the recruitment of patients, the proportion of patients referred to outpatient rehabilitation after stroke was surveyed by collecting data retrospectively, regarding patients discharged from the stroke units at the emergency hospitals in the 2 areas targeted in this feasibility study. The survey was based on data reported by the hospitals to Riksstroke, the Swedish Stroke Register , and included patients treated at the hospitals during the same period that patients at the out-patient units were included in the study to allow a comparison of the number of patients referred from the hospital to out-patient care after stroke with the number of patients identified at the units. The data consisted of: the number of patients treated and discharged from the stroke units at the hospitals; destination after discharge from the hospital and rehabilitation unit; and referrals made to rehabilitation units.
Surveying rehabilitation interventions using observations
Overt non participant observations were performed during rehabilitation sessions to survey interventions provided for patients in the context of the health care setting (Numbered 1, in Fig. 1). During these observations, an observation guide was used , also previously applied .
Surveying rehabilitation interventions through health care records at the rehabilitation units
Data from the patients’ health care records were retrieved to survey the rehabilitation interventions provided at the units. The data included rehabilitation interventions performed, the number and type of visits and duration of rehabilitation periods (Numbered 2, in Fig. 1).
Assessing patients’ functioning and disability with standardized assessment tools
Standardized assessment tools comprising aspects of the patients’ functioning and disability specified in the 3 recommendations in the SNGSC were tested (Fig. 1). The assessments were made face to face in the patients’ homes. While the aim was to test the procedures used in this feasibility study (rather than to evaluate patient outcome before and after an intervention), patients were assessed on one occasion using the tools outlined in Table 1. In addition, in order to to provide a more full-bodied picture of the patient’s functioning and disability, cognitive function was assessed with the screening tool Mini Mental Test examination . The assessment tools were tested for floor and ceiling effects, the feasibility of the assessments, and acceptability as perceived by the patients.
Descriptive statistics were used for quantitative data. The interviews were digitally recorded and transcribed verbatim. The observations were documented as field notes and reflections, all transcribed into a Word format . Qualitative content analysis was used to analyze the observations and interviews with managers, staff and patients . In the analysis of the first interviews with managers and staff, addressing the context of the health care setting, first, a naïve understanding was sought. Then, the texts were condensed, grouped and coded into subcategories and categories . The subcategories that emerged after analysis of 5 interviews with managers were used to inform a matrix, including categories identified as corresponding to leadership, culture, evaluation, facilitation and evidence, in line with the PARIHS framework , and additionally the organizational structure and rehabilitation process. This matrix was then applied to the analysis of the remaining interviews with managers and staff . In the analysis of the second interview addressing the feasibility and acceptability of the data collection procedures as proposed by Freeley et al.  a similar process was used. Here, the texts were condensed, grouped and coded into subcategories and categories corresponding to the feasibility and acceptability aspects of the recruitment and data collection. Trustworthiness was established by recurrent dialogues within the research team regarding the most valid understanding of the data and the rigor of the analysis .
The observations during rehabilitation sessions and interviews with patients were analyzed using another matrix, based on the 3 recommendations in the SNGSC (Table 1), also including the patient’s participation in the rehabilitation intervention . The same matrix was also used to analyze the health care records where, in addition, documented use of standardized assessment tools was surveyed. The analyzed observations were compared to the notes in the health care records regarding the same rehabilitation session.
Recruiting units, managers and staff
All senior and front-line managers approached gave their consent to include their unit in the study and the front-line managers did not report any difficulties related to the recruitment of staff members. Their numbers, years in position and sex is presented in Table 2.
Surveying the health care contexts
The analyses of the initial interview with the managers showed that aspects related to leadership, culture and evaluation, facilitation and evidence had been captured, as well as aspects related to the organizational structure and the rehabilitation process. Findings are presented in Table 3.
In the initial interviews with staff, results from the interviews with managers relating to leadership, evaluation and the rehabilitation process were confirmed. Again, the financial conditions were found to direct rehabilitation interventions and evaluation focused on productivity. Moreover, the leadership decision-making process described by the managers was confirmed by the staff. In addition, the staff interviews confirmed that the content of the rehabilitation intervention was directed by the staff and patient’s needs and that the planning and execution of rehabilitation interventions differed between staff members. Multiple ways to stay informed about new evidence were described, but this was not done in a systematic manner. In terms of facilitation, staff members pointed to the necessity of involving staff members in changing practice in order to be successful in implementation.
Recruiting patients at the units and the assessment of the recruitment process
Eighteen patients in the urban area and 6 patients in the rural area were approached by staff at the units. In 1 unit in the rural area no eligible patients were identified by staff during the time of inclusion. In the 2 remaining units, 1 patient at each unit were identified and included after given consent. To increase the number of included patients, another 4 patients were approached in an adjacent geographical area. Nine patients in the urban area and 2 in the rural area declined participation leaving 13 patients to be included in the study. Reasons for declining were: other dominating disease, the patient’s social situation, limitation in communication due to aphasia, or language barriers.
At the 2 local hospitals, one serving each area, patient data (over the course of the same 8 weeks as data was collected in the out-patient clinic) could be successfully retrieved retrospectively. During the recruitment period in the urban area, 64 patients were referred from the hospital to out-patient rehabilitation after stroke. Over the same period in the rural area 5 patients were referred from the hospital to out-patient rehabilitation after stroke.
In total, 13 patients were included and assessed by the data collectors (mean age: 73 years, SD 8, range 61–86; sex: 9 men, 4 women; time since stroke onset: 2 months, SD 2, range 1–6 months). All but 1 patient gave consent to access their health care records.
The staff reported that 7 of these 13 patients were planned for rehabilitation interventions according to the 3 recommendations in the SNGSC and were thus eligible for interviews and participation in the observations.
Surveying rehabilitation interventions provided at the units
The notes from the observations were compared to the notes in the health care records regarding the same rehabilitation session. The observations made at the rehabilitation session were, in general, found to correspond to the notes in the health care records, in terms of documented activities performed during the sessions. However, contrary to the detailed observations, the health care records provided a summarized description of the interventions provided in the rehabilitation sessions. The observations generated information regarding the interaction between the patient and the staff, e.g. if and how the patient was involved in the choice of interventions provided by staff and the patient’s self-sufficiency in performing the exercises during the sessions. Moreover, the observations provided more detailed information regarding the type and intensity of the exercises and whether the interventions were task-oriented or not.
When all the notes in the health care records from the rehabilitation period were analyzed, a more comprehensive picture of the interventions provided emerged:
All but 1 patient were found to be in need of and had received rehabilitation interventions according to the 3 recommendations in the SNGSC.
Disability and task specific training according to the 3 recommendations in the SNGSC were documented in the health care records.
All but 1 patient received all their rehabilitation interventions in the home and all but 1 patient received visits from both an occupational therapist (mean 5 visits, SD 3, range 2–10), and a physiotherapist (mean 5 visits, SD 5, range 0–17).
The duration of the rehabilitation periods varied between 5 and 45 weeks (with a mean of 15 weeks).
Use of standardized assessments tools to assess patient’s functioning, disability and outcome of the interventions according to the 3 recommendations were not found in the health care records, with the exception of 1 assessment of limitations in ADL.
All patients who participated in the interviews and were observed experienced disability of relevance to the 3 recommendations in the SNGSC. The interventions that the patients recalled and described were also found in the health care records. However, the health care records included information on additional interventions that were not described by the patients.
Assessing patients’ functioning and disability with standardized assessment tools
During the patient assessments performed by the data collectors, no adverse events were reported. All items included in the assessment tools could be completed in the patients’ homes, while the 6 min walking test was performed in the area surrounding the patients’ homes.
None of the included patients suffered from cognitive decline according to the Mini Mental Test examination (mean 26, SD 2) but impaired memory was self reported in the Stroke Impact Scale (mean 80, SD 13). Moreover, according to the results from the assessment with the Barthel Index, the included patients were all in the higher range (median 95, IQR 90–100), indicating a high proportion of patients who were independent in personal care activities and mobility. However, according to SIS the patients experienced limitations in personal care and domestic life (mean 78, SD 14) as well as in mobility (mean 79, SD 19) and in participation in meaningful activities (mean 68, SD 18). Thus, aspects of the patients’ functioning and disability not captured by the Mini Mental Test examination and the Barthel Index were captured by the Stroke Impact Scale. During the assessments with the Katz Extended ADL-Index, difficulties in responding to the questions regarding dependence in domestic activities or public transportation were identified for 5 patients who, by choice rather than disability, did not perform these activities.
Data collection procedures as perceived by managers, staff and patients
All the managers, staff and patients took part in the semi-structured interviews. One manager did not wish to be digitally recorded, but agreed to notes being taken during the interview. No burden relating to the interviews was reported by the managers, staff or patients. All included patients gave their consent to being observed and assessed with the standardized tools for evaluation of patient outcome and completed their participation during these data collection procedures. Again, no perceived burden was reported by the patients.
According to the follow-up interviews, staff members had gained a good understanding of the patient recruitment process through the written information combined with the telephone conversation. However, in terms of the actual recruitment process, a more diverse picture evolved: while some found the recruitment process to be effortless, others had problems with e.g. finding words to describe the study to the patients. In addition, difficulties with developing good routines to execute the recruitment process were reported. Some found it hard to remember to inform the patients due to either a high work load or too few eligible patients. Also, the staff applied different interpretations of the inclusion criteria, and as a result they did not always inform all eligible patients about the study.
The staff’s perceived burden of participating in the recruitment of patients was affected by the working conditions at the units. In 1 unit, a high work load negatively affected staff members’ compliance with following the study procedures and participation was perceived as a burden. In other units, the staff’s participation was limited or could not be evaluated because few or no patients with stroke had been referred during the recruitment period.
In the following section, the feasibility of surveying health care contexts, in connection to implementation of guidelines such as the SNGSC, is discussed, as well as the feasibility and acceptability of the data collection procedures used in this study. The knowledge gained can be used in the design of full scale studies focusing on the implementation of guidelines in out-patient health care settings.
Recruitment of managers, staff and patients
The units included represented rural and urban areas, different organizational structures, and with different financial conditions. A spread in sex and number of years in held positions was found among managers and staff as well as a spread in the characteristics of the patients included. Thus, results in this pilot are derived from a limited but presumably representative sample of Swedish out-patient health care settings where stroke related rehabilitation interventions are provided . A limitation was the low number of inhabitants in the rural area which restricted the number of patients eligible for inclusion in the study. This finding indicates the need of surveying the demographics and the number of eligible patients in a standardized manner before inclusion.
The feasibility of recruiting patients from out-patient units can be questioned. When data regarding the number of discharged patients from the stroke units at the hospitals was compared to data regarding the number of patients approached at the out-patient units, results indicated that numerous patients were lost in the referral to the out-patient units and/or in the inclusion process at the units. It must be noted that not all patients discharged after stroke onset are in need of rehabilitation interventions. However, results from the Swedish national stroke register’s evaluation of the provision of home-base rehabilitation  and patients’ satisfaction with care  indicate that a substantial number of patients do not receive adequate rehabilitation interventions. These findings indicate that when designing a study regarding implementation of guidelines in out-patient care, the collaboration between the hospital and out-patient units need to be targeted, in order to minimize the potential risk of patients in need of rehabilitation interventions after discharge being lost in the referral process. This knowledge may be generalized to other diagnoses and different care trajectories. Overall, a crucial issue in implementation studies is how to optimize the identification of patients that will benefit from the recommended clinical interventions .
In the present study, the identification of eligible patients discharged to their homes and to rehabilitation was feasible when data was provided from the hospital. Another advantage of using this procedure is that eligible patients may be identified without burdening health practitioners. Notably, the recruitment process was found not to be acceptable due to the perceived burden, identified by the staff in 1 unit. These findings indicate a need to relieve the staff from of the responsibility of identifying, approaching, and informing the patients. Moreover, the fact that staff members at the units used different interpretations of the inclusion criteria illustrates the need to only use data collectors with full knowledge of inclusion and exclusion criteria for identifying and approaching patients. These results are in line with findings in a systematic review where issues related to the clinicians’ participation in the studies were reported. The review concludes that the patient-related recruitment process should be piloted, that demands on the clinicians should be minimized and that their participation needs to be supported by dedicated research staff .
Surveying out-patient health care settings
Interviews with managers and staff were considered feasible and generated valuable knowledge regarding the context of the out-patient health care setting. However, interviews with front-line managers may be considered sufficient to gain knowledge of the context as the interviews with staff provided no additional information. As time is often limited for study participants, the necessity of collecting affirmative data needs to be balanced with the burden that participants may experience .
Given the leadership focus, the PARISH framework that contains a specified leadership element was thought to be an appropriate knowledge translation model to be used as a framework in the current study . Support for using PARIHS as a frame can be found in Damschroder et al’s overview of theories in the knowledge translation field, where key constructs regarding implementation were found to include the PARIHS elements evidence, culture, leadership and evaluation . This overview also support the additional categories identified in the analyses of the interviews in our pilot, i.e. organizational structure and the rehabilitation process related to patient needs, and resources . Thus, we suggest that using PARIHS as an implementation study framework is feasible and relevant.
Surveying rehabilitation interventions provided at the units
The subgroup of 7 patients who participated in the interviews and observations were identified by staff, which is a limitation due to the risk of bias. Instead, in a full scale study, to prevent bias, eligible patients should be identified through health care records, following a standardized procedure performed by trained data collectors. The interviews with patients were found to be acceptable and provided information on the patients’ perception of the rehabilitation interventions, but were not a feasible tool to survey the rehabilitation interventions provided. Instead it was feasible to identify these interventions through the notes in the health care records. The notes from the entire rehabilitation period also provided information on whether the patients were receiving interventions according to the 3 guideline recommendations. Moreover, the type of interventions provided, the number of visits, and the duration of the rehabilitation period could also be identified from the records. Retrospective data collections from health care records to survey adherence to clinical stroke guidelines is a common approach; the results in terms of such adherence have been found to vary .
One barrier to guideline use is that national guidelines are considered by practitioners not to be sufficiently concise and specific . One example of a reported barrier to adoption is lack of information on frequency, duration, and intensity of mobility interventions . In the present study the results from the assessment of data from the health care records indicated that the provision of out-patient care comprised rehabilitation interventions according to the SNGSC. However, as the outcomes of the interventions were not assessed in a standardized manner by the health practitioners, the sufficiency of the interventions described in the SNGSC cannot be fully evaluated. With a mean value of only 5 home visits per patient, and the financial restraints reported in the interviews that directed the outline of the interventions, the conditions for achieving a favorable patient outcome can be questioned. The SNGSC provides only a brief description of what the recommended interventions should entail, but defines the expected outcome of an intervention (e.g. as specified in Table 1). Thus, the assessment of outcome is crucial when evaluating adherence to guidelines such as the SNGSC. It is evident that managers and health practitioners will not know if the frequency, duration and intensity of an intervention provided are sufficient unless the presumed positive effects of an intervention are evaluated. Relevant, valid and structured assessments of patient outcome may provide valuable feed back to managers and health practitioners on the quality of the rehabilitation intervention and enhance the use of standardized updated clinical procedures.
Use of standardized assessment tools
The assessment tools tested in this pilot were chosen for their suitability to assess patient outcomes after stroke rehabilitation in out-patient health care in accordance with the 3 recommendations in the SNGSC. The majority of these assessments were found to be feasible and acceptable to the patients when performed in the patients’ homes. However, the use of the Mini Mental Test Examination to assess cognitive impairments among patients receiving rehabilitation in out-patient care after stroke should be questioned. When compared to self-perceived cognitive ability in the Stroke Impact Scale, the results of the Mini Mental Test Examination indicate a ceiling effect. Instead the The Montreal Cognitive Assessment (MoCa) could be considered, as MoCa is more suitable for patients with milder cognitive impairments .
According to the results from using the Barthel Index to assess dependence in personal care and mobility , a ceiling effect was also indicated when compared to self-rated limitation in mobility and activities of daily living rated in the Stroke Impact Scale . These findings are in line with results in a previous study . Moreover, the use of Katz Extended ADL-Index to assess dependence in domestic life and transportation was not found to be feasible. Questions regarding dependence in ADL, in the Katz Extended, were described as hard to respond to by patients who did not perform the task before stroke while the questions in the Stroke Impact Scale regarding self-perceived experiences of limitations in ADL were less difficult to respond to. These results indicate that both Barthel Index and Katz Extended can be excluded in favour of the Stroke Impact Scale. These findings also point to a general need for testing assessment tools on the targeted patient group in the planning stage of an intervention study, as valid and reliable assessment tools can show floor or ceiling effects in different phases of recovery or progress of a disease.
In this feasibility study a variety of qualitative and quantitative data collection procedures and measures were tested. The results indicate what can be used as a set of feasible and acceptable data collection procedures and measures suitable for studying implementation of stroke guidelines in an out-patient health care context. When planning a full scale study, procedures and measures to be considered are:
Using interviews with managers to survey the health care context at the units.
Minimizing the involvement of clinical staff in recruitment and data collection in favour of trained data collectors to ensure consistency and to ease the burden on clinical staff.
Identifying eligible patients from local hospitals rather than from out-patient units to survey the full reach of guideline use.
Using piloted measures of patients’ functioning and disability, assessed by trained data collectors, to evaluate results of guideline use.
Retrieving data from health care records to assess the health practitioners’ adherence to guidelines and their use of standardized assessments to evaluate patient outcome.
Norrving B, Wester P, Sunnerhagen KS, Terent A, Sohlberg A, Berggren F, et al. Beyond conventional stroke guidelines: setting priorities. Stroke. 2007;38:2185–90.
The Swedish National Audit Office. The Swedish Parliament. http://www.riksrevisionen.se/PageFiles/560/RiR_2004_9.pdf. Accessed Aug 2015.
The National Board of Health and Welfare. Nationella riktlinjerna för strokesjukvård http://www.socialstyrelsen.se/nationellariktlinjerforstrokesjukvard. Accessed Aug 2015.
The National Board of Health and Welfare. Nationell utvärdering 2011-Strokevård http://www.socialstyrelsen.se/publikationer2011/2011-11-2. Accessed Aug 2015.
Riksstroke The Swedish Stroke Register. Annual report 2011. http://www.riks-stroke.org/content/analyser/Rapport07.pdf. Accessed Aug 2015.
Riksstroke The Swedish Stroke Register. One year follow up 2012. http://www.riksstroke.org/wp-content/uploads/2014/05/Strokerapport_2012_1års-web.pdf. Accessed Aug 2015.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.
Walker MF, Fisher RJ, Korner-Bitensky N, McCluskey A, Carey LM. From what we know to what we do: translating stroke rehabilitation research into practice. Int J Stroke. 2013;8:11–7.
Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A. An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs. 2004;13:913–24.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, et al. Ingredients for change: revisiting a conceptual framework. Qual Saf Health Care. 2002;11:174–80.
Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci. 2008;3:1.
Ovretveit J. Improvement leaders: what do they and should they do? A summary of a review of research. Qual Saf Health Care. 2010;19:490–2.
Stig K, Paulsson Lutz I, The National Board och Health and Welfare. Hälso- och sjukvårdens finansiering i Norden. Nordisk Medicinalstatistisk kommitté 2013. http://www.nowbase.dk/~/media/Projekt%20sites/Nowbase/Publikationer/Andre/Finansiering.ashx. Accessed Aug 2015.
The National Board of Health and Welfare: SOSFS 1997: 8 (M) Allmänna råd. Verksamhetschef inom hälso- och sjukvård. 1997. http://www.socialstyrelsen.se/Lists/Artikelkatalog/Attachments/12721/1997-10-8.pdf. Accessed Aug 2015.
Whitehead AL, Sully BG, Campbell MJ. Pilot and feasibility studies: is there a difference from each other and from a randomised controlled trial? Contemp Clin Trials. 2014;38:130–3.
Richards DA. Complex interventions in health. An overview of research methods. London & New York: Routledge; 2015.
Feeley N, Cossette S, Cote J, Heon M, Stremler R, Martorella G, et al. The importance of piloting an RCT intervention. Can J Nurs Res. 2009;4:85–99.
Seers K, Cox K, Crichton NJ, Edwards RT, Eldh AC, Estabrooks CA, et al. FIRE (Facilitating Implementation of Research Evidence): a study protocol. Implement Sci. 2012;7:25.
Riksstroke The Swedish Stroke Register. http://www.riksstroke.org/eng/. Accessed Aug 2015.
Spradley JP. Participant observation. USA: Wandsworth Thompson Learning; 1980.
Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–98.
Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008;62:107–15.
Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24:105–12.
Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol. 1999;52:1143–56.
Donnellan C, Sweetman S, Shelley E. Health professionals’ adherence to stroke clinical guidelines: a review of the literature. Health Policy. 2013;11:245–63.
Donnellan C, Sweetman S, Shelley E. Implementing clinical guidelines in stroke: a qualitative study of perceived facilitators and barriers. Health Policy. 2013;111:234–44.
Arias M, Smith LN. Early mobilization of acute stroke patients. J Clin Nurs. 2007;16:282–8.
Blackburn DJ, Walters S, Harkness K. Letter by Blackburn et al regarding article, “Is the Montreal cognitive assessment superior to the mini-mental state examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation”. Stroke. 2011;42:e582. author reply e583.
Mahoney FI, Barthel DW. Functional evaluation: the Barthel index. Md State Med J. 1965;14:61–5.
Duncan PW, Bode RK, Min Lai S, Perera S. Rasch analysis of a new stroke-specific outcome scale: the Stroke Impact Scale. Arch Phys Med Rehabil. 2003;84:950–63.
Duncan PW, Lai SM, Bode RK, Perera S, DeRosa J. Stroke impact scale-16: a brief assessment of physical function. Neurology. 2003;60:291–6.
Berg K, Wood-Dauphinee S, Williams JI. The balance scale: reliability assessment with elderly residents and patients with an acute stroke. Scand J Rehabil Med. 1995;27:27–36.
Hsieh CL, Hsueh IP, Mao HF. Validity and responsiveness of the rivermead mobility index in stroke patients. Scand J Rehabil Med. 2000;32:140–2.
Kosak M, Smith T. Comparison of the 2-, 6-, and 12-min walk tests in patients with stroke. J Rehabil Res Dev. 2005;42:103–7.
Borg GA. Psychophysical bases of perceived exertion. Med Sci Sports Exerc. 1982;14:377–81.
Sonn U, Asberg KH. Assessment of activities of daily living in the elderly. A study of a population of 76-year-olds in Gothenburg, Sweden. Scand J Rehabil Med. 1991;23:193–202.
Duncan PW, Wallace D, Lai SM, Johnson D, Embretson S, Laster LJ. The stroke impact scale version 2.0. Evaluation of reliability, validity, and sensitivity to change. Stroke. 1999;30:2131–40.
Financial support was provided by the Vårdal Foundation and the Swedish Stroke Association (Stroke-Riksförbundet). We are thankful to Disa Sommerfeld who contributed with her extensive knowledge on stroke rehabilitation during the intervention.
The authors declare that they have no competing interests.
SP, MT, ACE and LW have made substantial contribution to conception, design and analysis and interpretation of data. SP and MT have been responsible for the data collection. SP has been writing the manuscript and MT, ACE and LW have revised the manuscript critically for important intellectual content. LWH, AE, GT and CO have made substantial contribution to the conception and design and have been involved in revising the manuscript critically for important intellectual content. SP, MT, ACE, LW, LWH, AE, GT and CO have all given final approval of the version to be published and agree to be accountable for all aspects of the work.