This article has Open Peer Review reports available.
Development and pilot of an internationally standardized measure of cardiovascular risk management in European primary care
- Sabine Ludt†1Email author,
- Stephen M Campbell†2,
- Jan van Lieshout†3,
- Richard Grol3,
- Joachim Szecsenyi1 and
- Michel Wensing3
© Ludt et al; licensee BioMed Central Ltd. 2011
Received: 9 December 2010
Accepted: 7 April 2011
Published: 7 April 2011
Primary care can play an important role in providing cardiovascular risk management in patients with established Cardiovascular Diseases (CVD), patients with a known high risk of developing CVD, and potentially for individuals with a low risk of developing CVD, but who have unhealthy lifestyles. To describe and compare cardiovascular risk management, internationally valid quality indicators and standardized measures are needed. As part of a large project in 9 European countries (EPA-Cardio), we have developed and tested a set of standardized measures, linked to previously developed quality indicators.
A structured stepwise procedure was followed to develop measures. First, the research team allocated 106 validated quality indicators to one of the three target populations (established CVD, at high risk, at low risk) and to different data-collection methods (data abstraction from the medical records, a patient survey, an interview with lead practice GP/a practice survey). Secondly, we selected a number of other validated measures to enrich the assessment. A pilot study was performed to test the feasibility. Finally, we revised the measures based on the findings.
The EPA-Cardio measures consisted of abstraction forms from the medical-records data of established Coronary Heart Disease (CHD)-patients - and high-risk groups, a patient questionnaire for each of the 3 groups, an interview questionnaire for the lead GP and a questionnaire for practice teams. The measures were feasible and accepted by general practices from different countries.
An internationally standardized measure of cardiovascular risk management, linked to validated quality indicators and tested for feasibility in general practice, is now available. Careful development and pilot testing of the measures are crucial in international studies of quality of healthcare.
Cardiovascular disease (CVD) is a major cause of premature death in Europe and also an important cause of morbidity, contributing substantially to escalating healthcare costs . Cardiovascular risk management (CVRM) is targeted at three broad populations: those with established and diagnosed disease (e.g. angina, stroke), those at high risk of developing CVD due to known risk factors (e.g. hypertension, smoking), and individuals at low risk for developing CVD, but with unhealthy lifestyles. CVRM involves risk assessment and communicating to patients the risk of developing CVD [2, 3], providing relevant guideline based treatments, such as counselling on lifestyle as well as cardio-protective medication and follow-up where appropriate [4, 5]. Primary care has an important role in the delivery of CVRM and quality improvement programs have been developed to further strengthen this role .
Previous research has emphasised the importance of addressing global CVD risk assessment rather than focusing on individual risk factors such as smoking . Various global CVD risk assessment and communication instruments for use in primary care have been developed across Europe [7–12], but there are differential and inconsistent use in routine clinical practice between and within countries [13–15]. The EUROASPIRE studies have advocated the need for more effective lifestyle management of patients at high risk and medication management of patients with coronary heart disease (CHD) . These findings were derived from patients receiving treatment from medical specialists, whereas most patients received treatment in primary care, particularly in countries with a strong primary care system [17, 18].
Quality indicators (QI) are used widely across Europe and internationally as part of quality improvement and pay-for-performance schemes [19–21], but research suggests that not all patients received the necessary evidence-based care that underpin these indicators [22, 23].
We started a large project (EPA-Cardio) to assess CVRM in primary care across Europe. The participating countries were recruited from the TOPAS Europe Association, which is a collaboration between researchers of quality improvement in healthcare, founded in January 2005 http://www.epa-cardio.eu. In the first stage of the EPA-Cardio-project, we developed quality indicators for the prevention and management of cardiovascular disease in primary care . Careful development and testing of measures linked to these quality indicators were deemed crucial for valid international comparisons. This paper describes the methods used to develop the set of measures, the results of a pilot in different countries of these measures, and the resulting final standardized EPA-Cardio instrument.
The development of the data collection instruments and piloting took place between March 2007 and January 2008. A core group of researchers (SC, MW, JvL, and SL) developed the prototypes of the instruments, and discussed and refined it in meetings with the project partners. Pilot tests were done with GPs volunteering to participate.
Development of the EPA-Cardio instrument
Allocation of the quality indicators (QI) - Data collection methods and patient groups
Medical record audit
Quality of care domains*
Patients with diabetes**
Patients with CVD
Patient at high risk for CVD
Primary prevention group
Overall number of QI
Patient populations (table 2)
Inclusion- and exclusion criteria for different patient groups
1. Patients with CHD
2. Patients at high risk
3. Primary-prevention group
- Documented diagnosis
- Men ≥ 60 and
- Age: 18-45 years
- ICD 10 code: I20-I25 or
- Registered or regular visitors in practice
- ICPC-2 code: K74-76
- ICD 10: F17 or
- ICPC-2: P17
Exclusion criteria for all 3 groups:
- terminal illness
- cognitive disorders (e.g. dementia)
- psychiatric diseases (e.g. schizophrenia)
- lack of language knowledge
Exclusion criteria for patient groups 2 and 3
- established CVD (Angina, Myocardial infarction, stroke)
Patients with established/diagnosed CHD
Some indicators related to CVD overall, and others in particularly to coronary heart disease (CHD). To facilitate data collection and homogenise our study group, we decided to focus on the patient group to whom all these indicators apply, i.e. patients with CHD. We excluded patients with diabetes because our primary focus was on patients with CVD.
Patients 'at high risk'
While we used widely accepted risk-calculation models to define high risk, for pragmatic reasons we defined 'high risk' for the EPA-Cardio pilot as a 'male smoker over 60 years of age. For these individuals (without other risk factors), the risk of fatal or nonfatal CVD events within 10 years was estimated at 14.4%, using a German electronic risk calculation instrument , meeting the definition-criteria of high risk. We expected to facilitate patient sampling by focusing on one risk factor other than age and gender.
Adults with unhealthy life styles
Only a few indicators related to this population. In EPA-Cardio, we focused on all individuals between 18 and 45 years. As risk and risk factors for CVD increases with age, we used this age limit with the intention of separating this group from patients at high risk of CVD.
Data collection procedures differed between the participating countries, due to ethical and legal regulations, and the number of available practice team members. The research team provided guidance for patient sampling and data collection as part of the EPA-Cardio instrument.
Established CHD and 'at high risk' groups
In Germany, Slovenia, Switzerland, and the Netherlands, patient selection and data extraction of the medical record audit was conducted by a practice nurse (guided by the GP). Support by telephone and email was provided by a researcher in the relevant country. In Belgium, the GP took over this task, and, in the UK, a researcher visited the practice to conduct the searches and the audit in collaboration with practice staff.
Where there identified more than 30 patients in a practice, we used random numbers to select patients for inclusion. We included all identified patients where there were less than 30 patients. We used consecutive sampling if the practice conducted manual searches. All patients selected for the medical audit (established CHD and 'at high risk') per practice were included in the patient survey (maximum of 30 patients per group per practice). As in most countries (except the UK), informed consent was requested for auditing medical records, questionnaires with informed consent sheets were sent after selecting eligible patients, and data were abstracted only for those who had sent back written consents. We intended to include at least 15 patients of each group per practice.
Primary prevention group (aged 18-45)
It was planned to select the primary prevention group by electronic searches (considering age group and exclusion criteria) followed by random sampling of 40 individuals from the search list. Where an electronic search was not feasible, the practice approached regular visitors who met the inclusion criteria consecutively. As we aimed to include 15 individuals per practice, we planned to send 40 questionnaires per practice in cases of electronic selection, or to hand out 15 questionnaires to practice visitors ready to participate in the study.
We allocated each indicator to the most appropriate instrument: data abstraction forms for the medical record for patients with CHD and patients at high risk respectively, patient questionnaires for all 3 groups as appropriate, an interview guide for the lead GP, and a questionnaire for practice members. The designing of the abstraction forms for the medical record followed that of Banks , in order to facilitate data encoding and analysis.
Questionnaire for patient survey
In all cases, selected patients were sent a questionnaire and letter explaining the aims of the study along with an informed consent sheet, a questionnaire (relevant to the appropriate group), and a self-addressed prepaid envelope to return the questionnaire to the research group and the written consent sheet to the practice team. The questionnaires included questions derived from the quality indicators, questions about sociodemographic issues and other conditions. Validated instruments to evaluate the quality of primary care from the patient perspective were also included for all 3 patient groups . Other validated instruments were included to evaluate the quality of chronic care for patients with CHD [28–30] and instruments to assess lifestyle behaviour for patients at high risk and the primary prevention-group) [31–34].
A researcher conducted the interview with the lead GP or practice manager, either by telephone or face to face and recorded the answers in the interview questionnaire. The interview questionnaire contained questions representing indicators on organisational issues and risk assessment methods. In addition, practice engagement in cardiovascular quality improvement projects and public health projects on cardiovascular health were assessed, and also GP's views and activities on primary prevention.
Practice members completed a practice questionnaire sent by the researchers. This questionnaire contained questions to identify the practice according to size, location, number and function of practice staff and number of listed or yearly attending patients. Other items were derived from the EPA practice management  and represented the EPA dimensions "information process and technology", "organisation of chronic care and prevention", and "quality improvement". These items were included to obtain an insight into the quality management activities of included practices considered to have an influence on study results.
The aim of the pilot study was to assess the acceptability of the developed data collection instruments and feasibility of the methods, irrespective of the difference of nationally adapted methods. We conducted the pilot study at the end of 2007 in 2 practices in 6 of the participating countries Belgium, Germany, the Netherlands, Slovenia, Switzerland and the United Kingdom. Not all countries had the materials translated and ready in time for the pilot study. We focused only on patients with CHD and patients at high risk in the pilot, because most data-collection instruments and procedures were related to these groups and the primary-prevention group was easier to select. Researchers chose the practices out of national GP-networks, approaching them by phone or mail. In each of the participating countries, pilot practices received between 100€ and 400€ as an incentive to take part.
Data collection procedures included patient selection, sending of patient questionnaires, data abstraction from the medical records, interview with the lead GP and completing of the practice questionnaire.
At the end of the pilot study, researchers interviewed the lead nurse by phone (face-to-face in the UK at the time of the visit), using an interview guide, and asked questions about the acceptability and feasibility of the instruments as well as the data-collection procedures. The pilot results were discussed at a meeting of all EPA-Cardio country co-ordinators, common standards were agreed, as also national adaptations to revise the instruments and procedures for subsequent use in the project. Ethical approval was sought as appropriate in the different participating countries.
The instrument was piloted in 12 European general practices, 2 general practices per country, in Belgium, Germany, the Netherlands, Slovenia, Switzerland and the UK. These practices selected a total of 268 patients with established CHD and 127 patients at high risk.
The practice teams rated study materials and instructions as easily understandable. Despite differences in data-collection procedures between the countries, all were capable of completing the assessment. It was possible to complete all procedures within 4-8 weeks. In the UK, an external reviewer completed the assessment within one working day per practice (without waiting for the replies from the patient's questionnaires).
Selection of patient groups, data abstraction
The group of patients with CHD was easier to select than the group of patients at high risk, because smoking status was not well documented in the medical records, and small practices could not select enough patients in this group. Therefore, it was possible to identify 30 patients with CHD in most practices, whereas the number of patients selected at high risk was between 8 and 30 (in larger practices with patient lists of >10000 patients).
The patients had some difficulties in completing the questionnaires, especially the PACIC-instrument, often asking nurses for explanations. Patient lists were used as check lists to follow the audit procedures, control the questionnaire responses, and send a reminder if necessary. Most of the countries sent reminders by mail. In Germany, the practice nurses reminded patients by telephone. Sending reminders was not permissible in the UK. The response rates were between 50% (UK) and 100% (Slovenia).
Interview with the lead GP
The researcher carried out the interview with the lead GP either in a face-to-face meeting (UK) or, in most countries, by telephone. The interview lasted 15-20 min.
Questionnaire for the practice team
Practice teams had no problems with completing the questionnaire for practice members. They needed 20-30 minutes to answer all questions.
Factors that contributed to the success or failure of data collection
There was a good level of practice co-operation with the research team of each country. Practices considered the instructions for patient selection and data-collection procedures as helpful, as well as the possibility to contact the research team if problems arose with study procedures. The practices reported that the selection of the eligible study patients from the medical records raised the major problem and consumed the most time of the audit. In most countries, nurses needed the help of the GP because they were not familiar with ICD or ICPC codes and searches run with the practice software.
The final EPA-Cardio instrument
The final EPA-Cardio instrument
Number of items
Data abstraction tools
1 and 2
- Sex and age
- 2 items
- Quality indicators (QI) on documentation of:
- 16 items for patient group 1 (18 QI)**
- Patient's lifestyle (smoking, physical activity and BMI)
- lifestyle advice (smoking, physical activity and diet advice)
- levels of blood pressure, cholesterol and blood glucose
- 14 items for patient group 2 (12 QI)
Patient group 1
- QI on documentation of pharmaceutical treatment (statins, anti-platelet therapy and influenza vaccination)
- 3 items (3 QI)
- QI on documentation an medical record
- 8 items (8 QI)
- QI on CVD risk assessment
- 27 items (20 QI)
- Practice engagement in CVD-quality improvement (programs) or
- 7 items (5 QI)
- Practice engagement in CVD-related public health programs+
- 1 item
- Views on primary prevention (lifestyle, lifestyle advice and support)+
- 11 items
All patient groups: 1,2,3
- Patient demographics and conditions+
- 11 items
- Quality of primary care delivery (EUROPEP)+
- 23 items
Patient group 1
- QI on lifestyle advice and patient perspective
- 2 items (2 QI)
- Quality of chronic illness care (PACIC)+ [28;29]
- 26 items
1 and 2
- Quality of life / health state (EQ5D)+
- 5 items and VAS
- Medication/ Medication adherence (Morisky score) +
- 5 items
2 and 3
- Lifestyle: Rapid assessment of physical activity+ (RAPA) 
- 14 items
- Lifestyle: Rapid Eating and Activity Assessment for participants - shortened version (REAP-S)+
- 16 items
- Lifestyle: Behavior Change Consortium: Smoking Assessment (Mid-Sized model - baseline measurements)+
- 10 items
Patient group 3
- Views on primary prevention (lifestyle, lifestyle advice and support) +
- 11 items (6 QI)
Questionnaire for the practice team +
- Information process and technology
- 11 items
- Organisation of chronic care and prevention
- 20 items
- Quality improvement activities
- 13 items
- Practice size and location
- 5 items
- Practice staff (number and function)
- 7 items
This study provided an internationally standardized measure of cardiovascular risk management in primary care, linked to previously selected quality indicators and pilot tested for feasibility.
Strengths and limitations
The EPA-Cardio project was conducted by the same team that developed the EPA-practice management accreditation tool (European Practice Assessment). Lessons learned in the process were applied to EPA-Cardio [35, 36]. The basis of the EPA-Cardio instrument were quality indicators developed using a validated Delphi procedure. However, this methodology is did not cover all aspects of cardiovascular prevention and care as described and recommended in guidelines . The assessment of important activities of primary care, such as communication, empathy, teamwork, consultation time, counselling and continuity of care, require other methodologies, such as in-depth interviews e.g. [37, 38]. To complete the assessment of cardiovascular prevention and management, we supplemented the EPA-Cardio instrument with patient evaluation instruments. We therefore chose the EUROPEP-instrument because it is validated, used in many surveys and exists in the languages of all participants . As patients' perspectives contribute considerably to the improvement in the quality of care , we also included the PACIC instrument to reflect the view of patients with established CHD . This project aims to provide better cover of primary prevention, and we therefore included validated lifestyle-assessment instruments  and self-developed questions on attitudes towards primary prevention for individuals and GPs.
We included a convenience sample of countries out of the TOPAS Europe Association http://www.topaseurope.eu in the study for feasibility reasons and good experiences with collaboration in past projects [35, 36]. This begs the question of whether EPA-Cardio be different if, for example, Greece and Denmark and Portugal had taken part in the study.
In the pilot, we used a small convenience sample of 2 practices in 6 of the 9 participating countries, as some countries were not able to have the study materials ready in time. The different methods used to select patient groups may lead to different populations (with limited generalisability) and different data-collection procedures, and therefore may restrict the comparability. The results of the subsequent observational study should therefore be interpreted carefully.
Methods of data collection in general practices
The systematic collection of healthcare data across different countries in primary practice is difficult because of differences in levels of computerisation and different practice software within and between European countries . The use of computers to record clinical data, and also the content and quality of electronic patient records, varies significantly [41, 42]. In order not to exclude countries or practices with lower levels of computerisation, it is important to apply methodologies that prevent research only from being conducted selectively in technologically highly developed practices [22, 43]. To make data collection feasible for practices with different technological levels in all countries, we developed paper-based abstraction tools. However, electronic data abstraction in general practice reduces the associated workload and is a pre-condition for the implementation of quality assessment in routine care .
The challenge of cross-national assessment
The aim of the study was to develop standardized data collection procedures in all countries. However, it is also necessary to adapt to differences in national procedures to conduct cross-country collaborations. Perfect standardisation of data collection methods in cross-national studies is not possible due to differences in legislation and practice resources. For example, whether national data protection acts allow anyone not employed by the practice to look at patient medical records . Another difficulty is the different standards of the ethics committees between, and even within, countries [46, 47]; for example, in some countries practice staff are allowed to collect patient data for quality improvement purposes without the permission of patients (e.g. UK, the Netherlands), whereas in others (e.g. Germany, Belgium) the informed consent of the patient is required.
The future of the EPA-Cardio instrument
We used the final EPA-Cardio instrument in the EPA-Cardio project within 9 participating countries between May 2008 and August 2009 . The results of this large observational study may be used to compare countries with different quality improvement strategies, such as disease management programmes in Germany , indicator-based incentive contracts in the UK , or practice-based support with outreach visits in the Netherlands . This may identify best practice examples and derive quality improvement recommendations for the prevention and management of cardiovascular diseases in Europe.
An internationally standardized measure of CVRM in primary care is now available, linked to previously selected quality indicators. It has been pilot tested for feasibility. Careful development and pilot testing were deemed crucial for international comparisons. Detailed assessment, analysis and feedback can be the starting point for developing quality improvement activities and for deriving recommendations at a national and pan-European level.
The Bertelsmann Foundation (Gütersloh, Germany) initiated and co-ordinated the EPA-Cardio project, in collaboration with IQ Healthcare of the Radboud University Nijmegen (Nijmegen, The Netherlands). The Bertelsmann Foundation also supported the project financially (without restrictions on the scientific work), and each of the project participants invested additional resources in the project. The following individuals were part of the EPA-Cardio group, in addition to the authors of this study protocol: Ester Cornelis (Belgium), Eva Frigola (Spain), Glyn Elwyn (UK), Reinhold Glehr (Austria), Margalit Goldfracht (Israel), Hector Falcoff (France), Lilian Michlig (Switzerland), Esko Kumpusalo (Finland), Janko Kersnik (Slovenia), Kati Kettunen (Finland), Beat Künzi (Switzerland), Solene Martin (France), Davorina Petek (Slovenia), Ingrid Pichler (Austria), Martin Roland (UK), Marianne Samuelson (France), Veerle van der Stighelen (Belgium), Trudy van der Weijden (Netherlands).
- Leal J, Luengo-Fernandez R, Gray A, Petersen S, Rayner M: Economic burden of cardiovascular diseases in the enlarged European Union. Eur Heart J. 2006, 27 (13): 1610-1619. 10.1093/eurheartj/ehi733.View ArticlePubMedGoogle Scholar
- Sheridan SL, Viera AJ, Krantz MJ, Ice CL, Steinman LE, Peters KE, et al: The effect of giving global coronary risk information to adults: a systematic review. Arch Intern Med. 2010, 170 (3): 230-239. 10.1001/archinternmed.2009.516.View ArticlePubMedGoogle Scholar
- Waldron CA, van der WT, Ludt S, Gallacher J, Elwyn G: What are effective strategies to communicate cardiovascular risk information to patients? A systematic review. Patient Educ Couns. 2010.Google Scholar
- Graham I, Atar D, Borch-Johnsen K, Boysen G, Burell G, Cifkova R, et al: European guidelines on cardiovascular disease prevention in clinical practice. Fourth Joint Task Force of the European Society of Cardiology and other Societies on Cardiovascular Disease Prevention in Clinical Practice. Eur J Cardiovasc Prev Rehabil. 2007, 14 (S2): S1-S113. 10.1097/01.hjr.0000277983.23934.c9.View ArticlePubMedGoogle Scholar
- Ferket BS, Colkesen EB, Visser JJ, Spronk S, Kraaijenhagen RA, Steyerberg EW, et al: Systematic review of guidelines on cardiovascular risk assessment: Which recommendations should clinicians follow for a cardiovascular health check?. Arch Intern Med. 2010, 170 (1): 27-40. 10.1001/archinternmed.2009.434.View ArticlePubMedGoogle Scholar
- Van Lieshout J, Wensing M, Campbell SM, Grol R: Primary care strength linked to prevention programs for cardiovascular disease. Am J Manag Care. 2009, 15 (4): 255-262.PubMedGoogle Scholar
- D'Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al: General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study. Circulation. 2008, 117 (6): 743-753.View ArticlePubMedGoogle Scholar
- Conroy RM, Pyorala K, Fitzgerald AP, Sans S, Menotti A, De Backer G, et al: Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003, 24 (11): 987-1003. 10.1016/S0195-668X(03)00114-3.View ArticlePubMedGoogle Scholar
- Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P: Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007, 335 (7611): 136-10.1136/bmj.39261.471806.55.View ArticlePubMedPubMed CentralGoogle Scholar
- Krones T, Keller H, Sonnichsen A, Sadowski EM, Baum E, Wegscheider K, et al: Absolute Cardiovascular Disease Risk and Shared Decision Making in Primary Care: A Randomized Controlled Trial. Ann Fam Med. 2008, 6 (3): 218-227. 10.1370/afm.854.View ArticlePubMedPubMed CentralGoogle Scholar
- van Steenkiste B, van der WT, Stoffers HE, Kester AD, Timmermans DR, Grol R: Improving cardiovascular risk management: a randomized, controlled trial on the effect of a decision support tool for patients and physicians. Eur J Cardiovasc Prev Rehabil. 2007, 14 (1): 44-50. 10.1097/01.hjr.0000239475.71805.1e.View ArticlePubMedGoogle Scholar
- Bonnevie L, Thomsen T, Jorgensen T: The use of computerized decision support systems in preventive cardiology--principal results from the national PRECARD survey in Denmark. Eur J Cardiovasc Prev Rehabil. 2005, 12 (1): 52-55. 10.1097/00149831-200502000-00008.PubMedGoogle Scholar
- Getz L, Kirkengen AL, Hetlevik I, Romundstad S, Sigurdsson JA: Ethical dilemmas arising from implementation of the European guidelines on cardiovascular disease prevention in clinical practice. Scandinavian Journal of Primary Health Care. 2004, 22 (4): 202-208. 10.1080/02813430410006693.View ArticlePubMedGoogle Scholar
- Mosca L, Linfante AH, Benjamin EJ, Berra K, Hayes SN, Walsh BW, et al: National Study of Physician Awareness and Adherence to Cardiovascular Disease Prevention Guidelines. Circulation. 2005, 111 (4): 499-510. 10.1161/01.CIR.0000154568.43333.82.View ArticlePubMedGoogle Scholar
- Chauhan U: Cardiovascular disease prevention in primary care. Br Med Bull. 2007, ldm002Google Scholar
- Kotseva K, Wood D, De Backer G, De Bacquer D, Pyorala K, Keil U: Cardiovascular prevention guidelines in daily practice: a comparison of EUROASPIRE I, II, and III surveys in eight European countries. The Lancet. 2009, 373 (9667): 929-940. 10.1016/S0140-6736(09)60330-5.View ArticleGoogle Scholar
- WHO: World Health Report 2008. 2008, WHO, GenevaGoogle Scholar
- Starfield B, Shi L, Macinko J: Contribution of primary care to health systems and health. Milbank Q. 2005, 83 (3): 457-502. 10.1111/j.1468-0009.2005.00409.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Burge FI, Bower K, Putnam W, Cox JL: Quality indicators for cardiovascular primary care. Can J Cardiol. 2007, 23 (5): 383-388.View ArticlePubMedPubMed CentralGoogle Scholar
- Perera R, Dowell T, Crampton P, Kearns R: Panning for gold: an evidence-based tool for assessment of performance indicators in primary health care. Health Policy. 2007, 80 (2): 314-327. 10.1016/j.healthpol.2006.03.011.View ArticlePubMedGoogle Scholar
- Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M: Effects of pay for performance on the quality of primary care in England. N Engl J Med. 2009, 361 (4): 368-378. 10.1056/NEJMsa0807651.View ArticlePubMedGoogle Scholar
- Kirk SA, Campbell SM, Kennell-Webb S, Reeves D, Roland MO, Marshall MN: Assessing the quality of care of multiple conditions in general practice: practical and methodological problems. Qual Saf Health Care. 2003, 12 (6): 421-427. 10.1136/qhc.12.6.421.View ArticlePubMedPubMed CentralGoogle Scholar
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348 (26): 2635-2645. 10.1056/NEJMsa022615.View ArticlePubMedGoogle Scholar
- Campbell SM, Ludt S, Van Lieshout J, Boffin N, Wensing M, Petek D, et al: Quality indicators for the prevention and management of cardiovascular disease in primary care in nine European countries. Eur J Cardiovasc Prev Rehabil. 2008, 15 (5): 509-515. 10.1097/HJR.0b013e328302f44d.View ArticlePubMedGoogle Scholar
- Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46 (12): 1417-1432. 10.1016/0895-4356(93)90142-N.View ArticlePubMedGoogle Scholar
- Banks NJ: Designing medical record abstraction forms. Int J Qual Health Care. 1998, 10 (2): 163-167. 10.1093/intqhc/10.2.163.View ArticlePubMedGoogle Scholar
- Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, et al: Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000, 50 (460): 882-887.PubMedPubMed CentralGoogle Scholar
- Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM: Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005, 43 (5): 436-444. 10.1097/01.mlr.0000160375.47920.8c.View ArticlePubMedGoogle Scholar
- Glasgow RE, Emont S, Miller DC: Assessing delivery of the five 'As' for patient-centered counseling. Health Promot Int. 2006, 21 (3): 245-255. 10.1093/heapro/dal017.View ArticlePubMedGoogle Scholar
- Morisky DE, Green LW, Levine DM: Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care. 1986, 24 (1): 67-74. 10.1097/00005650-198601000-00007.View ArticlePubMedGoogle Scholar
- Glasgow RE, Ory MG, Klesges LM, Cifuentes M, Fernald DH, Green LA: Practical and relevant self-report measures of patient health behaviors for primary care research. Ann Fam Med. 2005, 3 (1): 73-81. 10.1370/afm.261.View ArticlePubMedPubMed CentralGoogle Scholar
- Topolski TD, LoGerfo J, Patrick DL, Williams B, Walwick J, Patrick MB: The Rapid Assessment of Physical Activity (RAPA) among older adults. Prev Chronic Dis. 2006, 3 (4): A118..PubMedPubMed CentralGoogle Scholar
- Segal-Isaacson CJ, Wylie-Rosett J, Gans KM: Validation of a Short Dietary Assessment Questionnaire: The Rapid Eating and Activity Assessment for Participants Short Version (REAP-S). The Diabetes Educator. 2004, 30 (5): 774-781. 10.1177/014572170403000512.View ArticlePubMedGoogle Scholar
- Yaroch AL, Nebeling L, Thompson FE, Hurley TG, Hebert JR, Toobert DJ, et al: Baseline Design Elements and Sample Characteristics for Seven Sites Participating in the Nutrition Working Group of the Behavior Change Consortium. J Nutr. 2008, 138 (1): 185S-192.PubMedGoogle Scholar
- Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, et al: Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract. 2006, 23 (1): 137-147. 10.1093/fampra/cmi091.View ArticlePubMedGoogle Scholar
- Engels Y, Campbell S, Dautzenberg M, Van den HP, Brinkmann H, Szecsenyi J, et al: Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract. 2005, 22 (2): 215-222. 10.1093/fampra/cmi002.View ArticlePubMedGoogle Scholar
- Campbell SM, Hann M, Hacker J, Burns C, Oliver D, Thapar A, et al: Identifying predictors of high quality care in English general practice: observational study. BMJ. 2001, 323 (7316): 784-10.1136/bmj.323.7316.784.View ArticlePubMedPubMed CentralGoogle Scholar
- Carpiano RM, Flocke SA, Frank SH, Stange KC: Tools, teamwork, and tenacity: an examination of family practice office system influences on preventive service delivery. Prev Med. 2003, 36 (2): 131-140. 10.1016/S0091-7435(02)00024-5.View ArticlePubMedGoogle Scholar
- Rao JK, Weinberger M, Kroenke K: Visit-specific expectations and patient-centered outcomes: a literature review. Arch Fam Med. 2000, 9 (10): 1148-1155. 10.1001/archfami.9.10.1148.View ArticlePubMedGoogle Scholar
- Green LA, Fryer GE, Froom P, Culpepper L, Froom J: Opportunities, challenges, and lessons of international research in practice-based research networks: the case of an international study of acute otitis media. Ann Fam Med. 2004, 2 (5): 429-433. 10.1370/afm.224.View ArticlePubMedPubMed CentralGoogle Scholar
- Thiru K, Hassey A, Sullivan F: Systematic review of scope and quality of electronic patient record data in primary care. BMJ. 2003, 326 (7398): 1070-10.1136/bmj.326.7398.1070.View ArticlePubMedPubMed CentralGoogle Scholar
- Heidenreich R, Himmel W, Böckmann H, et al: Documentation of electronic patient records in German general practices: a telephone survey [article in German]. German Journal for Quality in Health Care. 2005, 99 (9): 573-580.Google Scholar
- Chan SC, Lee TW, Teoh LC, Abdullah ZC, Xavier G, Sim CK, et al: Audit on cardiovascular disease preventive care in general practice. Singapore Med J. 2008, 49 (4): 311-315.PubMedGoogle Scholar
- de Lusignan S: An educational intervention, involving feedback of routinely collected computer data, to improve cardiovascular disease management in UK primary care. Methods Inf Med. 2007, 46 (1): 57-62.PubMedGoogle Scholar
- Iversen A, Liddell K, Fear N, Hotopf M, Wessely S: Consent, confidentiality, and the Data Protection Act. BMJ. 2006, 332 (7534): 165-169. 10.1136/bmj.332.7534.165.View ArticlePubMedPubMed CentralGoogle Scholar
- Hearnshaw H: Comparison of requirements of research ethics committees in 11 European countries for a non-invasive interventional study. BMJ. 2004, 328 (7432): 140-141. 10.1136/bmj.328.7432.140.View ArticlePubMedPubMed CentralGoogle Scholar
- Byrne G, Morgan J, Kendall S, Offredy M: A critical account of obtaining research governance approval. Nurse Res. 2005, 13 (2): 7-18.View ArticlePubMedGoogle Scholar
- Miksch A, Laux G, Ose D, Joos S, Campbell S, Riens B, et al: Is there a survival benefit within a German primary care-based disease management program?. Am J Manag Care. 2010, 16 (1): 49-54.PubMedGoogle Scholar
- Ashworth M, Jones RH: Pay for performance systems in general practice: experience in the United Kingdom. Med J Aust. 2008, 189 (2): 60-61.PubMedGoogle Scholar
- Hulscher ME, van Drenth BB, van der Wouden JC, Mokkink HG, van Weel C, Grol RP: Changing preventive practice: a controlled trial on the effects of outreach visits to organise prevention of cardiovascular disease. Qual Health Care. 1997, 6 (1): 19-24. 10.1136/qshc.6.1.19.View ArticlePubMedPubMed CentralGoogle Scholar
- Rabin R, de Charro F: EQ-5D: a measure of health status from the EuroQol Group. Ann Med. 2001, 33 (5): 337-343. 10.3109/07853890109002087.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/11/70/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.