- Research article
- Open Access
Identifying factors associated with experiences of coronary heart disease patients receiving structured chronic care and counselling in European primary care
BMC Health Services Researchvolume 12, Article number: 221 (2012)
Primary care for chronic illness varies across European healthcare systems. In patients suffering from coronary heart disease (CHD), factors associated with patients’ experiences of receiving structured chronic care and counselling at the patient and practice level were investigated.
In an observational study comprising 140 general practices from five European countries (Austria, Germany, the Netherlands, Switzerland and the United Kingdom), 30 patients with Coronary Heart Disease (CHD) per practice were chosen at random to partake in this research. Patients were provided with a questionnaire and the Patient Assessment of Chronic Illness Care (PACIC-5A) - instrument. Practice characteristics were assessed through a practice questionnaire and face to face interviews. Data were aggregated to obtain two practice scores representing quality management and CHD care, respectively. A hierarchical multilevel analysis was performed to examine the impact of patient and practice characteristics on PACIC scores.
The final sample included 1745 CHD-patients from 131 general practices with a mean age of 67.8 (SD 9.9) years. The overall PACIC score was 2.84 (95%CI: 2.79; 2.89) and the 5A score reflecting structured lifestyle counselling was 2.75 (95% CI: 2.69; 2.79). At the patient level, male gender, more frequent practice contact and fewer related or unrelated conditions were associated with higher PACIC scores. At the practice level, performance scores reflecting quality management (p = 0.013) and CHD care (p = 0.009) were associated with improved assessment of the structured chronic care and counselling received.
Patients’ perceived quality of care varies. However, good practice management and organisation of care were positively reflected in patients’ assessments of receiving structured chronic illness care. This highlights the importance of integrating patient experiences into quality measurements to provide feedback to health care professionals.
Cardiovascular disease (CVD) and notably coronary heart disease (CHD) are important causes of morbidity, contributing substantially to escalating healthcare costs . Cardiovascular risk management (CVRM), the majority of which is provided in primary care, includes counselling on lifestyle, preventive medication and continuous monitoring [2, 3]. Not all patients suffering from CVD receive optimal care [4, 5], which may be related to a range of factors including practice characteristics, and the structure of the care and counselling, which differs considerably across Europe .
Chronic Care Model (CCM)
The Chronic Care Model (CCM) is a widely adopted framework to enhance evidence based chronic care [7, 8]. It describes a proactive patient centred care approach that is planned and includes goal setting, problem-solving and follow-up support. The principles of the CCM are included in disease management programs in various health-care systems. Previous research demonstrated that elements of the CCM have been associated with improved quality of care and patient outcomes [9, 10]. Furthermore, there is evidence that a strong primary health-care orientation, with general practitioners acting as gatekeepers providing the first contact with patients within the health care system and ensuring continuity of care, is linked to the adoption of the CCM  and improved chronic illness care and outcomes .
5A Approach of behaviour change counselling
The 5A approach concerns behaviour change counselling, which is integral to the CCM, and provides a sequence of evidence-based brief intervention steps (Assess, Advise, Agree, Assist and Arrange). These steps are recommended for behavioural counselling and self-management support in primary care settings to address a broad range of behaviours and health conditions .
The Patient Assessment of Chronic Illness Care (PACIC)- General Plus 5A
The Patient Assessment of Chronic Illness Care (PACIC) questionnaire [14, 15] allows patients to assess whether the care they have received is congruent to the principles of the CCM. The 20-item PACIC questionnaire was developed and validated by Glasgow et al  and was later revised to include six new items assessing self-management support according to the 5A approach . The instrument has been translated, validated and used by patients suffering from various conditions in several countries [17–20]. Recently, the PACIC was used to evaluate case -and disease-management interventions [21, 22].
Quality measurement in primary care
Since 2001, attempts to assess and improve the quality of primary care have led to the development of two instruments as part of the “European Task Force for Methods of Assessment, Quality Management and Certification in Health Care” (TOPAS Europe)1 research project. These instruments are based on quality indicators, and were developed using validated systematic consensus techniques, expert panels and empirical testing . The European Practice Assessment (EPA)- practice management instrument was developed to measure the quality of practice management [24, 25]. During the autumn of 2003, the EPA practice management instrument came into operation in nine countries . EPA’s key aspects of activity are the development and validation of a set of indicators and tools describing the organisational aspects of primary care practices. The EPA indicators were incorporated into questionnaires, interview guides and check-lists based on an extended review of the international literature concerning assessment models for primary care, with special attention being paid to the Dutch model of practice visits. The indicator-based instrument has been demonstrated to provide valid, reliable and feasible results for quality improvement in primary health care . Since 2005 this instrument has been revised every three years, and this cycle will continue, as the instrument is used for benchmarking ambulatory care practices . The second instrument developed was the EPA-Cardio instrument, which assesses the quality of cardiovascular care and risk management [27, 28], and was developed between 2006 and 2008 .
Quality measurement from the patients’ perspective
Quality of care is multidimensional and there are various aspects of quality and several methods relating to quality measurement [29, 30]. Patient involvement and engagement are imperative to achieving good outcomes during chronic care , and patient experience is an integral part of quality measurement and improvement . Recently, international studies have demonstrated a poor association between practice characteristics and patient satisfaction [32–34]. However, the performance of health care providers may not be reflected in patients’ assessments of quality if general summarising satisfaction scores are used . Reports of patient experience are increasingly replacing assessments of patient satisfaction, as they can highlight processes of care in detail including providing self-management support, and can be helpful for providing feedback to health care professionals .
Therefore, the aim of this international observational study, conducted as part of the development of the EPA-Cardio instrument, was to estimate patients’ experience of receiving structured chronic care and counselling, and to examine the extent to which quality processes reported in general practices and individual patient characteristics were associated with patients’ experience of structured chronic care and counselling. As the PACIC instrument, our primary outcome, questions care processes received by patients within a six month time period, we hypothesised that this process-oriented instrument would be associated with practice reported process measures rather than outcome-oriented instruments used to assess patient satisfaction.
This study was part of the European Practice assessment (EPA) - Cardio project, which focused on the assessment of cardiovascular prevention and management in European primary care. In the first stage of the 4-year EPA-Cardio project (which began in 2006) we developed quality indicators to measure cardiovascular prevention and care , and identified measures for use in a subsequent observational study . The international cross-sectional observational study was conducted in 10 European countries between 2008 and 2009, i.e. Austria, Belgium, Finland, France, Germany, the Netherlands, Slovenia, Spain, Switzerland and the United Kingdom . As the English, Dutch and German versions of PACIC - instrument were validated; only the UK, Germany, Austria, Switzerland and the Netherlands data were included in this study. Ethics committees in all participating countries approved the study.
Recruitment of participants
The recruitment of countries, practices and patients has been described in detail elsewhere : In summary, general practices were randomly sampled by each national research team according to the national distribution of general practices related to practice size and location, with the intention of recruiting a representative sample of 36 practices per country.
We included patients with coronary heart disease (CHD), e.g. myocardial infarction, angina pectoris or vascular surgery, who were identified from the practice records by the presence of a diagnosis code or active current medication. Patients with terminal illness or significant cognitive impairments were excluded. Of the eligible patients, a sample of 30 patients per practice was randomly selected to be posted a questionnaire, with an expected attrition rate of 50%.
The patient questionnaire included demographic items (e.g. education), two questions derived from the quality indicators developed during the first phase of the project , and the PACIC-5A instrument (Additional file 1) as the main outcome measure. PACIC is a validated measure of patients’ perception of chronic illness care that questions elements of care and counselling received by patients. We used the 26-item instrument that asked patients to indicate how often they received a care element within the last six months, e.g. “being asked how chronic illness affects one’s life”. Each item can be answered with “almost never”, “generally not” “sometimes”, “most of the time” or “almost always”, with a score of one for the first answer (almost never) up to five (almost always). The item scores aggregate into five subscales that align with components of the CCM. In addition, there are subscales for the five steps of the 5A counselling model (Assess, Advise, Agree, Assist, and Arrange).
Two quality indicators that were developed during the first stage of the EPA Cardio project were included in the questionnaire : “All patients with established CVD should be offered referral to a supervised cardiac exercise rehabilitation program (e.g. referred by either GP or hospital specialist)” and “All patients with established CVD should be asked about the quality of care they receive from their practice”. These indicators were paired to questions that could be answered with “yes”, “no” or “don’t know”.
Researchers collated data concerning practice characteristics and quality measures through practice questionnaires and face-to-face or telephone interviews with leading GPs using standardized interview guides. These instruments contained questions to characterise the practice according to size, location and practice staff, and quality indicators that were developed during the EPA-Cardio project  and those derived from the EPA practice management instrument [24, 25]. The quality indicators covered CVD care (33 indicators)  and organizational aspects of the practice management in the three dimensions of ‘information process and technology’ (11 indicators), ‘organization of chronic care and prevention’ (19 indicators) and ‘quality improvement’ (13 indicators) . To score practice quality indicators, items of the practice questionnaires were aggregated using the homogeneity analysis by alternating least squares (HOMALS). With this factor analysis, 32 binary items with discrimination measures over 0.4 were identified, loading on two dimensions “quality management” (15 items) and “CVD care (17 items) (Table 1). Scores were calculated through summation of the number of ‘yes’- answers, resulting in a range from 0 to 15 for the quality-management score, and from 0 to 17 for the CVD-care score.
The main outcome measure was the overall PACIC score, ranging between one and five, with higher scores indicating better patient perceived quality of chronic illness care. We calculated mean overall PACIC scores, overall 5A scores and subscale scores, following the scoring instructions of Glasgow et al (Additional file 1) .
Due to the hierarchical structure of the data, multilevel analysis was applied, which takes into account the non-independence of patient observations (level 1) nested within practices (level 2), and these nested within countries (level 3). Several models were evaluated:
Multilevel linear analysis began with a three-level null (empty) model with no predictor variables in the fixed part and only the intercepts in the random part of the model (M1). This model can be used as a reference for comparing the size of contextual (practice or country) variations in PACIC scores in subsequent models.
The next model (M2) included patient-level variables as fixed effects only. Finally, explanatory variables at the patient and practice level were added as fixed effects resulting in the best model fit (M3). Variance partition coefficients at each level were calculated using the restricted maximum likelihood (REML). The random-part results of the null model (M1) are reported together with the corresponding intra-class correlations (ICC) at the practice and country level . Finally, the fixed-part results of the full 3-level random-intercept model (M3) are presented. Variance partition coefficients with corresponding two-sided 95% confidence intervals (CI) are provided.
The significance level was set to 5% (two-sided). All statistical analyses were carried out using SPSS version 18.0 (SPSS Inc., Chicago, IL, USA), with the exception of the multilevel analysis. This analysis was conducted by using the procedure PROC MIXED in SAS version 9.2 (SAS Institute, Cary, NC).
Practice and Patient Characteristics
The total sample comprised 2152 patients with CHD in 140 primary care practices in five countries: Austria (28 practices), Germany (24 practices), the Netherlands (34 practices), Switzerland (22 practices) and the UK (32 practices). Four practices (16 patients) were excluded as they had enrolled fewer than five patients. For 1958 individuals (91%), PACIC sum scores could be calculated. Only patients with complete data concerning all explanatory variables were considered in the final model of the multilevel hierarchical analysis, reducing the sample size to 1745 patients from 131 practices (81.1%). A non-responder analysis demonstrated no clinically important differences. On average, excluded patients were 1.7 years older, had less yearly practice contacts and slightly less medical conditions, compared with patients included in analyses (Table 2). Of the final patient sample, 31.6% were female and the mean age was 67.8 years (SD 9.9). The majority of patients had been in school for more than nine years (68.7%) and had been visiting their practices for more than seven years (83.7%). Approximately 75% of the sample had never been asked to assess the quality of care of their practice previously. On average, patients self-reported as having three medical conditions (Table 2).
The mean number of patients included per practice was 13, ranging from 6-37. The mean full time equivalent GPs reported working at each practice was 1.8 (SD 1.6). The mean practices’ scores were 9.8 (SD 3.8) for “quality management”, ranging from 0-17, and 8.5 (SD 4.7) for “CVD care” (range: 0-15). The included practices (n = 131) did not differ from the excluded practices (n = 9) in relation to any variables.
PACIC and 5A scores
The mean overall PACIC score (range 1-5), reflecting elements of the chronic care model (CCM), was 2.84 [CI: 2.79; 2.89]. The mean 5A score (range 1-5), reflecting structured behaviour change counselling, was 2.75 [CI: 2.69; 2.79]. Mean sum scores and subscale scores differed from one another, with higher scores for chronic care processes such as ‘patient activation’ than for counselling steps such as ‘assess’ or ‘advise’. Highest scores were calculated for the subscale ‘delivery system/practice design, i.e. 3.35 [95% CI: 3.30; 3.40], representing organizational aspects of care. Lowest scores were found in the subscales on follow up management reflecting continuity of care: The subscale ‘Follow up/coordination’ scored 2.51 [95% CI: 2.46; 2.57] and the subscale ‘arrange’ achieved 2.38 [95% CI: 2.32; 2.43] (Table 3).
Factors associated with patients’ assessment of receiving structured chronic care and counselling
Random part results
The random-part results indicated that variation in the PACIC scores was predominantly due to variations in individual characteristics or differences between countries, rather than differences between practices. The total variance in the null model was 1.235. The proportion of variance (ICC) was 2.7% at the practice level and 26.0% at the country level. The residual proportion of variance including variance at patient level plus random was 71%. Including explanatory variables at patient and practice level explained the variance between countries to 10% and between practices to 22%.
Fixed part results
Table 4 presents the results for the overall PACIC score: Adjusted for all other variables included in the model, age and educational level were not significantly associated with patients’ assessment of receiving structured care and counselling (PACIC score). Being female was associated with a 0.06 point lower PACIC score compared with males (p = 0.012). Higher PACIC scores (0.22) were associated with patients who had been visiting their practice for less than two years compared with the reference category of visiting a practice for more than seven years (p = 0.001). Attending a practice three times or fewer per year was associated with lower PACIC scores (-0.16; p = 0.007) compared with attending more than seven times per year. Lower PACIC scores were achieved if patients were asked to assess the received care for the first time (-0.48) compared with patients that had been asked to assess this previously (p = 0.000). Patients’ assessment was associated with the number of medical conditions, resulting in a 0.01 decrease of the PACIC score with each additional condition (p = 0.008).
At the practice level, “CVD care” (0.01) and “quality management” (0.02) scores were associated with increasing PACIC scores. The regression coefficient indicates that for instance, each additional self-reported quality item of the CVD care score results in a 0.01 increase in the PACIC score. As practices included in the sample achieved the whole range of quality items from 0 to 17 (Table 1), the difference between the PACIC scores of practices with the lowest scores compared with those with the highest scores was 17 × 0.01 = 0.17. The coefficients for the quality management score require comparable interpretation, resulting in a maximum difference of 15 × 0.02 = 0.3 points in PACIC scores. The number of full time equivalent GPs, reflecting the practice size, was not significantly associated with PACIC scores.
This study has three main findings: Firstly, patients with coronary heart disease from European general practices perceive that the quality structured chronic care and counselling is not optimal. During this research the mean overall PACIC score was calculated as 2.84 (maximum = 5). During previous research, the PACIC instrument has been applied to a wide range of chronic conditions and populations [38, 39] including individuals suffering from diabetes , coronary heart disease (CHD) , osteoarthritis  and mental health , with overall scores reported being between 2.49  and 3.80. Compared to diabetes care , in the present study goal setting and follow-up support activities were less often provided during CHD care. Furthermore, key elements of the CCM, namely assisting patients with self management and arranging follow-up support, were provided significantly less often, indicating possible quality deficiencies and areas for quality improvement, particularly in relation to continuity of care .
The second and third findings are related to factors associated with patients’ assessments of receiving structured chronic care and counselling. At the patient level, being male, having more frequent practice contacts and having fewer other conditions were associated with higher PACIC scores. Other studies reported higher PACIC scores for younger patients  or did not demonstrate significant associations with patients’ characteristics [16, 19]. We applied the PACIC instrument to routine primary care settings in various European health care systems. Within these settings, patients who had been included in quality assessment previously scored significantly higher. This finding resonates with previous research, which demonstrated that patients who were enrolled in interventions and were cared for more intensively scored higher, especially when interventions were tailored to special elements of the CCM. For example, patients participating in disease-management programs have been reported as having higher PACIC scores than those receiving usual care . Furthermore, PACIC scores of patients participating in patient-centred case-management interventions can be improved from baseline to post-interventional measurement . These findings are consistent with the necessity to align evaluation research with care improvement strategies [42, 43].
At the practice level, there was a positive association between patients’ evaluations of the quality of care they received and quality scores reflecting quality-management and cardiovascular-care processes of general practices. However, the variance proportion at this level was less than the variance proportion at patient or country level. The variance caused by country specific factors can only be explained marginally in this study, as explanatory variables at the country level were not available. Although the variance between practices relating to the patients’ evaluations was relatively small, practice quality scores explained a significant portion of these variance. It had been presumed that larger practices with more full time equivalent (FTE) GPs would provide good quality of care . However, the number of FTE GPs was not associated with patients’ evaluations during this study.
There is not one standardised way in which to measure quality of care, and each method has strengths and limitations. Ideally, it would be desirable that patients’ evaluation of their experiences with health care were congruent to other quality measures of practice care, to provide feedback to health care providers. Previous research has demonstrated few associations between objective quality measures and patients’ satisfaction [32–34, 45]. Wensing et al. stated that patients may assess care differently from recommended care strategies . Furthermore, it has been argued that patients value humanistic and affective items (e.g. staff’s friendliness) more than items concerning organization and governance , and that patient assessments depend on the personal relationship to the practice team where trust, loyalty and positive regard may influence the assessment [34, 47]. In addition, several processes and structures of care are outside the direct experience or observation of patients, and not all patients are capable of understanding the risks and benefits of clinical choices [35, 48]. The quality of providers’ performance may not be reflected in patients’ perceptions if general satisfaction scores that summarize the assessment of different health care processes into one global score are used . The lack of positive association between quality measurement from the patients’ perspective and other quality measures of practice performance may be due to different underlying theoretical constructs of various instruments being used to assess patient perspectives [42, 49]. Patient satisfaction may reflect the relationship between patient and practice team, which is dependent on patient and practice characteristics, whereas patient experiences with care may focus on organisational and procedural aspects of care . The PACIC instrument predominantly questions the receipt of specific clinical processes of care. The underlying theoretical construct of the PACIC is the CCM; a patient centred care approach that is proactive, planned and includes goal setting, problem-solving and follow-up support. Therefore, positive association of patients’ perspectives with practice quality measures can be explained, as the quality indicators of practice organisation and chronic care used during this study could be related to these proactive CCM elements . As patient experience is an important component of quality of care, and patient involvement is central to achieving good outcomes, future research is necessary to specify the constructs of patient assessment instruments that are required to obtain valid and reliable patient judgements of health care processes, particularly if patient experience is used within pay for performance systems .
Strengths and limitations
The EPA cardio study is one of the largest international studies concerning management of cardiovascular care in European primary care . To eliminate different health care effects, countries with strong primary health care systems (UK, the Netherlands) and countries characterized by a weaker primary care orientation  were included, and multilevel modelling was used to adjust for these differences. We used validated patient measures and assessed practice quality indicators through well-proven means [24, 25]. All measures were pilot tested before being included in this study .
However, in Germany, Austria and Switzerland it was difficult to enrol 36 practices per country, as intended in the study protocol. As the PACIC instrument was only validated in three languages, countries such as France and Slovenia were excluded, decreasing the number of eligible patients. As we used an observational design, it was not possible to demonstrate a causal relationship between included characteristics or measures and patients’ perceived quality of care.
The results of this study demonstrate that from the perspective of patients, the delivery of structured chronic care and counselling could be improved in general practices, particularly the arrangement of follow-up contacts. This indicates possible quality deficiencies in the continuity of care. Although patients’ perspective is an important component of the multidimensional nature of quality, evaluations based on patient experience should be adjusted according to relevant patient characteristics including the severity of illness and gender. Our findings indicate that the quality of practice management and care is positively reflected in patients’ assessment of receiving structured chronic care and counselling. This provides evidence for integrating patients’ evaluations into quality improvement strategies to identify quality deficiencies and to provide feedback to health care providers. However, there is an argument that instruments to assess quality of care from patients’ perspective and other quality measures of health care may not be generally congruent.
Ethics committee approval
The corresponding author explicitly states that any necessary ethics committee approval was secured for the study reported. Names and locations of the approving ethics committees are listed below:
Austria: Med Uni Graz, Auenbruggerplatz 2, A-8036 Graz, Prof Peter Rehak
Belgium: Commissie Medische Ethiek UZ Leuven, Campus Gasthuisberg, Herestraat 49, B-3000 Leuven
Finland: Ethics committee of the Kuopio University Hospital, Kuopio, Finland
France: Ethic Commitee (Comié de Protection des Personnes Île-de-France V, Hôpital Saint-Antoine 184, rue du Faubourg Saint-Antoine 75012 Paris)
Germany: Ethics committee University Hospital: “Medizinische Fakultät Heidelberg“, Alte Glockengießerei 11/1, D-69115 Heidelberg (S-413/2007)
Netherlands: Ethics committee CMO Region Arnhem-Nijmegen
Slovenia: Slovenian national committee on medical ethics (No 87/11/07)
Switzerland: KEK - Kantonale Ethikkommission, Bern
United Kingdom: North West Research Ethics Committee: 07/H1010/83
Conflict of interests
The authors declare that they have no conflicts of interests.
Leal J, Luengo-Fernandez R, Gray A, Petersen S, Rayner M: Economic burden of cardiovascular diseases in the enlarged European Union. Eur Heart J. 2006, 27: 1610-1619. 10.1093/eurheartj/ehi733.
Graham I, Atar D, Borch-Johnsen K, Boysen G, Burell G, Cifkova R, Dallongeville J, De Backer G, Ebrahim S, Gjelsvik B, Herrmann-Lingen C, Hoes A, Humphries S, Knapton M, Perk J, Priori SG, Pyorala K, Reiner Z, Ruilope L, Sans-Menendez S, Op Reimer WS, Weissberg P, Wood D, Yarnell J, Zamorano JL, Walma E, Fitzgerald T, Cooney MT, Dudina A, Vahanian A, et al: European guidelines on cardiovascular disease prevention in clinical practice: full text. Fourth Joint Task Force of the European Society of Cardiology and other Societies on Cardiovascular Disease Prevention in Clinical Practice. Eur J Cardiovasc Prev Rehabil. 2007, 14: S1-S113.
Ferket BS, Colkesen EB, Visser JJ, Spronk S, Kraaijenhagen RA, Steyerberg EW, Hunink MG: Systematic review of guidelines on cardiovascular risk assessment: Which recommendations should clinicians follow for a cardiovascular health check?. Arch Intern Med. 2010, 170: 27-40. 10.1001/archinternmed.2009.434.
Nolte E, McKee M: Caring for people with chronic conditions. A health system perspective. 2008, World Health Organization, http://www.euro.who.int/__data/assets/pdf_file/0006/96468/E91878.pdf.
Kotseva K, Wood D, De Backer G, De Bacquer D, Pyorala K, Keil U: Cardiovascular prevention guidelines in daily practice: a comparison of EUROASPIRE I, II, and III surveys in eight European countries. Lancet. 2009, 373: 929-940. 10.1016/S0140-6736(09)60330-5.
Van Lieshout J, Goldfracht M, Campbell S, Ludt S, Wensing M: Primary care characteristics and population-orientated health care across Europe: an observational study. Br J Gen Pract. 2011, 61: e22-e30. 10.3399/bjgp11X548938.
Bodenheimer T, Wagner EH, Grumbach K: Improving primary care for patients with chronic illness. JAMA. 2002, 288: 1775-1779. 10.1001/jama.288.14.1775.
Bodenheimer T, Wagner EH, Grumbach K: Improving primary care for patients with chronic illness: the chronic care model, Part 2. JAMA. 2002, 288: 1909-1914. 10.1001/jama.288.15.1909.
Nutting PA, Dickinson WP, Dickinson LM, Nelson CC, King DK, Crabtree BF, Glasgow RE: Use of chronic care model elements is associated with higher-quality care for diabetes. Ann Fam Med. 2007, 5: 14-20. 10.1370/afm.610.
Tsai AC, Morton SC, Mangione CM, Keeler EB: A meta-analysis of interventions to improve care for chronic illnesses. Am J Manag Care. 2005, 11: 478-488.
Schmittdiel JA, Shortell SM, Rundall TG, Bodenheimer T, Selby JV: Effect of Primary Health Care Orientation on Chronic Care Management. Ann Fam Med. 2006, 4: 117-123. 10.1370/afm.520.
Starfield B, Shi L, Macinko J: Contribution of primary care to health systems and health. Milbank Q. 2005, 83: 457-502. 10.1111/j.1468-0009.2005.00409.x.
Goldstein MG, Whitlock EP, DePue J: Multiple behavioral risk factor interventions in primary care. Summary of research evidence. Am J Prev Med. 2004, 27: s61-s79. 10.1016/j.amepre.2004.04.023.
Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM: Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005, 43: 436-444. 10.1097/01.mlr.0000160375.47920.8c.
Glasgow RE, Emont S, Miller DC: Assessing delivery of the five 'As' for patient-centered counseling. Health Promot Int. 2006, 21: 245-255. 10.1093/heapro/dal017.
Glasgow RE, Whitesides H, Nelson CC, King DK: Use of the Patient Assessment of Chronic Illness Care (PACIC) With Diabetic Patients: Relationship to patient characteristics, receipt of care, and self-management. Diabetes Care. 2005, 28: 2655-2661. 10.2337/diacare.28.11.2655.
Wensing M, Van Lieshout J, Jung HP, Hermsen J, Rosemann T: The Patients Assessment Chronic Illness Care (PACIC) questionnaire in The Netherlands: a validation study in rural general practice. BMC Health Serv Res. 2008, 8: 182-10.1186/1472-6963-8-182.
Rosemann T, Laux G, Droesemeyer S, Gensichen J, Szecsenyi J: Evaluation of a culturally adapted German version of the Patient Assessment of Chronic Illness Care (PACIC 5A) questionnaire in a sample of osteoarthritis patients. J Eval Clin Pract. 2007, 13: 806-813. 10.1111/j.1365-2753.2007.00786.x.
Gensichen J, Serras A, Paulitsch M, Rosemann T, König J, Gerlach F, Petersen J: The Patient Assessment of Chronic Illness Care Questionnaire: Evaluation in Patients with Mental Disorders in Primary Care. Com Ment Health J. 2010, 47: 1-7.
Aragones A, Schaefer EW, Stevens D, Gourevitch MN, Glasgow RE, Shah NR: Validation of the Spanish translation of the Patient Assessment of Chronic Illness Care (PACIC) survey. Prev Chronic Dis. 2008, 5: A113.
Szecsenyi J, Rosemann T, Joos S, Peters-Klimm F, Miksch A: German Diabetes Disease Management Programs Are Appropriate for Restructuring Care According to the Chronic Care Model: An evaluation with the Patient Assessment of Chronic Illness Care instrument. Diabetes Care. 2008, 31: 1150-1154. 10.2337/dc07-2104.
Peters-Klimm F, Campbell S, Hermann K, Kunz C, Muller-Tasch T, Szecsenyi J: Case management for patients with chronic systolic heart failure in primary care: The HICMan exploratory randomised controlled trial. Trials. 2010, 11: 56-10.1186/1745-6215-11-56.
Campbell SM, Braspenning J, Hutchinson A, Marshall MN: Research methods used in developing and applying quality indicators in primary care. BMJ. 2003, 326: 326-10.1136/bmj.326.7384.326.
Engels Y, Campbell S, Dautzenberg M, Van den HP, Brinkmann H, Szecsenyi J, Falcoff H, Seuntjens L, Kuenzi B, Grol R: Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract. 2005, 22: 215-222. 10.1093/fampra/cmi002.
Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R: Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract. 2006, 23: 137-147.
Szecsenyi J, Broge B, Willms S, Brodowski M, Goetz K: Benchmarking in ambulatory care practices–The European Practice Assessment (EPA)] [Article in German. Z Evid Fortbild Qual Gesundh Wesen. 2011, 105: 404-407. 10.1016/j.zefq.2011.05.007.
Campbell SM, Ludt S, Van Lieshout J, Boffin N, Wensing M, Petek D, Grol R, Roland MO: Quality indicators for the prevention and management of cardiovascular disease in primary care in nine European countries. Eur J Cardiovasc Prev Rehabil. 2008, 15: 509-515. 10.1097/HJR.0b013e328302f44d.
Ludt S, Campbell S, Van Lieshout J, Grol R, Szecsenyi J, Wensing M: Development and pilot of an internationally standardized measure of cardiovascular risk management in European primary care. BMC Health Serv Res. 2011, 11: 70-10.1186/1472-6963-11-70.
Campbell N, Roland MO, Buetow SA: Defining quality of care. Soc Sci Med. 2000, 51: 1611-1625. 10.1016/S0277-9536(00)00057-5.
Campbell SM, Hann M, Hacker J, Burns C, Oliver D, Thapar A, Mead N, Safran DG, Roland MO: Identifying predictors of high quality care in English general practice: observational study. BMJ. 2001, 323: 784-10.1136/bmj.323.7316.784.
Institute of Medicine: Crossing the Qualiy Chasm: a new Health System for the 21st Century. 2001, National Academy Press, 1-337. http://www.nap.edu/openbook.php?isbn = 0309072808.
Wensing M, Vedsted P, Kersnik J, Peersman W, Klingenberg A, Hearnshaw H, Hjortdahl P, Paulus D, Kunzi B, Mendive J, Grol R: Patient satisfaction with availability of general practice: an international comparison. Int J Qual Health Care. 2002, 14: 111-118. 10.1093/oxfordjournals.intqhc.a002597.
Salisbury C, Wallace M, Montgomery AA: Patients' experience and satisfaction in primary care: secondary analysis using multilevel modelling. BMJ. 2010, 341: c5004-10.1136/bmj.c5004.
Sixma HJ, Spreeuwenberg PM, van der Pasch MA: Patient satisfaction with the general practitioner: a two-level analysis. Med Care. 1998, 36: 212-229. 10.1097/00005650-199802000-00010.
Elwyn G, Buetow S, Hibbard J, Wensing M: Respecting the subjective: quality measurement from the patient's perspective. BMJ. 2007, 335: 1021-1022. 10.1136/bmj.39339.490301.AD.
Wensing M, Ludt S, Campbell S, Van Lieshout J, Volbracht E, Grol R: European Practice Assessment of Cardiovascular risk management (EPA Cardio): protocol of an international observational study in primary care. Implement Sci. 2009, 4: 3-10.1186/1748-5908-4-3.
Davis PD, Scott AJ: The effect of interviewer variance on domain comparisons. Survey Methodol. 1995, 21: 99-106.
Schmittdiel J, Mosen D, Glasgow R, Hibbard J, Remmers C, Bellows J: Patient Assessment of Chronic Illness Care (PACIC) and Improved Patient-centered Outcomes for Chronic Conditions. J Gen Int Med. 2008, 23: 77-80. 10.1007/s11606-007-0452-5.
Taggart J, Chan B, Jayasinghe UW, Christl B, Proudfoot J, Crookes P, Beilby J, Black D, Harris MF: Patients Assessment of Chronic Illness Care (PACIC) in two Australian studies: structure and utility. J Eval Clin Pract. 2011, 17: 215-221. 10.1111/j.1365-2753.2010.01423.x.
Guthrie B, Saultz JW, Freeman GK, Haggerty JL: Continuity of care matters. BMJ. 2008, 337: 337-10.1136/bmj.a337.
Wood DA: Clinical reality of coronary prevention guidelines: a comparison of EUROASPIRE I and II in nine countries. Lancet. 2001, 357: 995-1001. 10.1016/S0140-6736(00)04235-5.
Roland M, Elliott M, Lyratzopoulos G, Barbiere J, Parker RA, Smith P, Bower P, Campbell J: Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England. BMJ. 2009, 339: 339.
Salisbury C: Using patient experience within pay for performance programmes. BMJ. 2009, 339: 339.
Ashworth M, Armstrong D: The relationship between general practice characteristics and quality of care: a national survey of quality indicators used in the UK Quality and Outcomes Framework, 2004-5. BMC Fam Pract. 2006, 7: 68-10.1186/1471-2296-7-68.
Wensing M, Baker R, Szecsenyi J, Grol R: Impact of national health care systems on patient evaluations of general practice in Europe. Health Policy. 2004, 68: 353-357. 10.1016/j.healthpol.2003.10.010.
Greenhalgh T, Heath I: Measuring quality in the therapeutic relationship - Part 1: objective approaches. Qual Saf Health Care. 2010, 19: 475-478. 10.1136/qshc.2010.043364.
Ridd M, Shaw A, Lewis G, Salisbury C: The patient-doctor relationship: a synthesis of the qualitative literature on patients' perspectives. Br J Gen Pract. 2009, 59: e116-e133. 10.3399/bjgp09X420248.
Chapple A, Campbell S, Rogers A, Roland M: Users understanding of medical knowledge in general practice. Soc Sci Med. 2002, 54: 1215-1224. 10.1016/S0277-9536(01)00091-0.
Evans RG, Edwards A, Evans S, Elwyn B, Elwyn G: Assessing the practising physician using patient surveys: a systematic review of instruments and feedback methods. Fam Pract. 2007, 24: 117-127. 10.1093/fampra/cml072.
Van Lieshout J, Wensing M, Campbell SM, Grol R: Primary care strength linked to prevention programs for cardiovascular disease. Am J Manag Care. 2009, 15: 255-262.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/12/221/prepub
Bertelsmann Foundation (Gütersloh, Germany) initiated, coordinated and funded the EPA Cardio project, in collaboration with IQ Healthcare of the Radboud University Nijmegen (The Netherlands). The following individuals were part of the EPA Cardio group: Ester Cornelis (Belgium), Eva Frigola (Spain), Glyn Elwyn (UK), Reinhold Glehr (Austria), Margalit Goldfracht (Israel), Hector Falcoff (France), Lilian Michlig (Switzerland), Esko Kumpusalo (Finland), Janko Kersnik (Slovenia), Kati Kettunen (Finland), Solene Martin (France), Ingrid Pichler (Austria), Martin Roland (UK), Marianne Samuelson (France), Veerle van der Stighelen (Belgium), Trudy van der Weijden (Netherlands).
MW developed the overall outline of the EPA-Cardio project. SC co-coordinated the selection of performance indicators, on which the EPA-Cardio audit instrument is based. SL co-coordinated the development and pilot testing of the measures, carried out data analysis and wrote the draft and final version of this paper. JvL contributed to the development and selection of measures and the draft revisions. JR provided biometrical support and contributed to analyses. DO and TF contributed to analyses and methods. JS supervised the research reported in this paper. All authors critically assessed and approved this paper.