Research article | Open | Open Peer Review | Published:
Patient-physician agreement on tobacco and alcohol consumption: a multilevel analysis of GPs’ characteristics
BMC Health Services Researchvolume 15, Article number: 110 (2015)
Data about tobacco and alcohol consumption are essential in many types of studies. These data can be obtained by directly questioning patients or by using the information collected from physicians. Agreement between these two sources varies according to the characteristics of patients but probably also those of physicians. The purpose of this study was to analyze the characteristics of general practitioners (GPs) associated with agreement between them and their patients about the patients’ consumption of alcohol and tobacco.
Data came from an observational survey among GPs who were internship supervisors in the Paris metropolitan area. Fifty-two volunteer GPs completed a self-administered questionnaire about the organization of their practice and their training. For each GP, a random sample of 70 patients, aged 40 to 74 years, answered questions about their personal tobacco and alcohol consumption. GPs simultaneously answered similar questions about each patient. We used a mixed logistic model to assess the association between physicians’ characteristics and agreement for patients’ smoking status and alcohol consumption.
Data were collected from both patient and physician for 2599 patients. The agreement between patients and their physicians was 60.4% for smoking status and 48.7% for alcohol consumption. Physicians with continuing medical education in management of smokers and those reporting specific skill in managing hypertension had the best agreement for smoking. Physicians who taught courses at the university medical school and those reporting specific skill in managing alcoholism had the best agreement for alcohol consumption.
Agreement increases with physicians’ training and skills in management of patients with tobacco and alcohol problems. It supports the importance of professional training for improving the quality of epidemiologic data in general practice. Researchers who use GPs as a source of information about patients’ tobacco and alcohol consumption must assess the physicians’ characteristics.
Tobacco and alcohol consumption are two major causes of morbidity and mortality. Worldwide, the mortality attributable to smoking has been estimated at 12%  and that attributable to alcohol at 3.8% . Data about consumption of these products, which are risk factors for numerous diseases, are essential in many types of studies, especially epidemiologic, as a variable of interest or for adjustment . These data can be obtained by directly questioning patients or by collecting the information from physicians [4,5]. General practitioners (GPs) are often questioned about this [6,7]. The information collected from physicians and patients is not always identical. They agree more closely for smoking than for alcohol consumption [7,8]. In addition, agreement for smoking varies according to patients’ characteristics: it is poorer for patients who are younger, male, or more socially disadvantaged .
Few studies [9,10] have examined the variations in agreement according to physicians’ characteristics. Nonetheless, when data are collected from physicians (for reasons of simplicity or if the patients cannot be questioned directly), knowing these characteristics would make it possible to identify the GPs with the most reliable information about consumption (that is, the closest to the patient-reported data). Some characteristics of practice organization, such as the number and types of professionals in the office, have already been studied, but no significant associations have been identified . Moreover, agreement differed according to type of practice organization (medical group vs independent practice association) in a US study, although this study did not consider tobacco and alcohol consumption . To the best of our knowledge, other important organizational aspects, including the duration of consultations, have not been analyzed. Moreover, we might suppose that physicians’ training, especially in managing smoking and alcoholism, might affect the agreement rate, but this hypothesis has never been tested.
The objective of this study was to analyze whether and if so what GP characteristics (related to their organization of work and their initial and continuing training) are associated with agreement between patients and physicians about the patients’ use of tobacco and alcohol.
This study is an ancillary analysis of data from an observational survey named Prev Quanti . This survey was initially designed to document social inequalities in preventive care (screening for breast, cervical, and colorectal cancer, tobacco and alcohol consumption, and cardiovascular risk) provided by GPs. To have a sample size large enough to be able to study the cancer screening tests recommended for patients aged 50–74 years, while still being able to analyze young patients at low cardiovascular risk, we chose to include only patients aged 40 to 74 years. A power calculation determined that we would require 50 GPs and 70 patients per GP to be able to demonstrate social gradients for the types of preventive care studied. The Prev Quanti study was conducted in 2008–09 among GPs who supervised students training in general practice during internships at their offices. We used email and telephone to recruit from among the 215 GPs working with two medical school departments of general practice in the Paris metropolitan area (who were paid 300 € for work estimated to take around 10 hours). For each participating GP, a random sample of 35 men and 35 women aged 40 to 74 years was drawn from their patient list (patients who had reported them to be their regular GP), furnished by the national health insurance fund. There were no exclusion criteria.
The GPs’ characteristics were collected by a self-administered questionnaire. The first group of characteristics concerned the organization of their practices: fixed fees or authorization for fees beyond the amount reimbursed by the national health insurance fund, group or solo practice, mean number of consultations per week, mean duration of consultations, percentage of consultations by appointment (compared with open hours), use of computerized medical files, and use of automatic reminders. The second group of characteristics concerned their initial and continuing training: time since completion of medical school, teaching medical school courses (above and beyond supervising internships), participation in peer groups, responsibility in an organization offering continuing medical education (CME), participation in a CME course or a study on any or more of six prevention themes (hypertension, smoking, alcoholism, and screening for breast cancer, colon cancer, and cervical cancer), and reporting specific skill for each of these themes.
All of the included patients received a questionnaire from their GP, which asked about their smoking status, alcohol consumption, and social situation. They were asked to complete and return it to the GP. At the same time, the GPs were asked to complete a form covering the same data for each patient, using the information in their medical files or what they knew about the patient, even if not mentioned in the medical file.
Patients’ smoking status, collected from the patient and the physician, was classified in three categories: current smoker, ex-smoker and non-smoker. Neither group’s report of smoking status was considered a gold standard.
The binary variable of agreement about smoking status was defined as positive when both the patient and the physician placed the patient in the same category. The physician, unlike the patient, could answer “do not know” about the patient’s smoking status. In this case, the response was considered discordant regardless of the patient’s response.
The variable of agreement about alcohol consumption was defined similarly to that for smoking but depended on the patient’s level of alcohol consumption, classified in four categories: abstinent, moderate consumption (≤210 g weekly for men and ≤ 140 g for women), occasional excessive consumption (≥50 g on a single occasion) and regular excessive consumption (>210 g weekly for men and > 140 g for women). This classification was determined by the alcohol consumption reported, and the cut-off values are those set by the French Society of Alcohol Studies .
We describe below only the analyses for smoking but the same analytic strategy was used for alcohol consumption. We first calculated the agreement among all patients for whom this calculation was possible (that is, in including all the physicians with “do not know” responses). Next we calculated the agreement and a weighted kappa coefficient (quadratic weighting) , only among patients whose physicians did not answer “do not know”, as is usual in the literature [7,8].
We used a mixed logistic model  with a random intercept [15,16] to assess the association between physicians’ characteristics and agreement for smoking status (the dependent variable). The structure of our data (that is, that patients are grouped by physicians) mandates the use of these statistical models  to obtain unbiased estimators . These models simultaneously include fixed indicators that enable ORs to be calculated, as in the standard models, and a random indicator, inter-GP variance, which allows us to calculate variations in the agreement rates between GPs .
We performed a univariate and then a multivariate analysis, adjusted for patient’s sex, age (40–50 years, 51–60, 61 to >70), duration of time seeing this GP (<1 year, 1–5, >5), number of consultations with the GP during the past year (<3, ≥3) and occupation, as well as for the physician’s sex  and age (in quartiles). We included in a multivariate model all of the GPs’ characteristics significantly associated with agreement for smoking status in the univariate analysis, at a threshold of 20%. We then removed the GP characteristics according to a backward stepwise strategy, with a threshold of 5%, to obtain the final model.
At the end, to evaluate the percentage of inter-GP variance (s2) explained by the GP characteristics in the final model, we constructed another multivariate model (called the reduced final model) containing all patient characteristics and the GPs’ age and sex but without any of the other GP characteristics present in the final model. The percentage of reduction of the inter-GP variance between these two models was calculated as follows: Δs2 = 100.(s2reduced final model - s2final model)/s2reduced final model.
This calculation allowed us to quantify the effect of GPs’ characteristics on the variations in the agreement rate between GPs, independently of any effect due to the composition of their practice (patient load) .
All statistical analyses were performed with R software, version 2.14.1, and SAS software, version 9.3. The study was approved by the advisory committee on information treatment for health research (Commission nationale de l’informatique et des libertés), and all patients signed informed consents.
The first 52 GPs who volunteered to participate were included in the study. Of the 3640 randomly selected patients, the return rate for physician questionnaires was 98.9% (n = 3600), while the patient participation rate was 71.6% (n = 2605). Finally, data were collected from both patient and physician for 71.4% (n = 2599) of the patients included.
The physicians’ mean age was 55 years (SD = 6), and 63% of them were men (Table 1). The mean duration of consultations was 21 minutes (SD = 4.8), and physicians saw 92 (SD = 23) patients weekly, on average.
The patients’ mean age was 54 years (+/−9) (Table 1). The median duration of the physician-patient relationship was 7 years (interquartile range (IQR) = 3-11) and the median number of consultations during the past year 3 (IQR = 1-5).
The patients’ non-response rate to the smoking questions was 0.3% (n = 8), and to the drinking questions, 23% (n = 596).
For smoking, the mean agreement rate was 60.3%. This rate varied significantly between physicians, ranging from 41.2% to 77.8% for the 10th and 90th percentiles of the distribution (s2 = 0.63). Among patients whose physician did not answer “do not know”, agreement was 75.6% and the weighted kappa coefficient was 0.70. Agreement for smoking status was higher among physicians who had done a CME course about managing smoking cessation and those who reported specific skill in managing hypertension (Table 4). The reduction in the inter-GP variance because of these two GP characteristics was 21%.
For drinking, the mean agreement rate was 48.7%. This rate again varied significantly between physicians, from 36.0% to 60.8% for the 10th and 90th percentiles of the distribution (s2 = 0.40). Among patients whose physician did not answer “do not know”, the agreement was 58.6% and the weighted kappa coefficient was 0.33. Agreement for alcohol consumption was higher among GPs teaching courses at the medical school and those GPs reporting specific skill in managing alcoholism (Table 5). The reduction in the inter-GP variance because of these two GP characteristics was 29%.
In our study, the agreement between patients and their physicians was about 60% for smoking status and almost 50% for alcohol consumption. Both varied significantly between the GPs. Physicians with a CME course in management of smoking cessation and those reporting specific skill in managing hypertension had the best agreement for smoking. Physicians who taught courses at the university medical school and those reporting specific skill in managing alcoholism had the best agreement for alcohol consumption. No characteristic of the organization of their practice was significantly associated with agreement.
To our knowledge, this study is the first to have tested the association between a large group of physicians’ characteristics related to their practice organization and medical education and agreement about smoking and alcohol consumption.
Another strength of our study is the good level of patient participation. This is explained in part by specific characteristics of the physicians who recruited them. The fact that the request for participation came from their regular GP  and that this physician had an association with the university  probably gave them confidence.
To assess the physicians’ knowledge of their patients’ smoking status and alcohol use, we used what the GPs actually knew about their patients, even if it was not written in the medical files. That is, GPs in France mainly use their files as practice reminders rather than exhaustive data collections or protection against malpractice suits: they frequently do not include everything they know about the patient (especially the things they know well and are unlikely to forget) . We also calculated agreements based only on what was recorded in the medical files. These agreement rates were lower (39.4% for smoking status and 13.3% for alcohol use, results not shown) than those we obtained in the principal analyses.
In addition to the portion of disagreement resulting from our treating physicians’ “do not know” responses as disagreement, several other points may explain some of this discordance between patients and physicians. Patients generally underestimate their consumptions during questionnaire surveys . But because the patients had to send the questionnaire back to the GP, they probably responded as they would have during a consultation with him or her. From their perspective, some physicians might also have assumed under-reporting by their patients and accordingly estimated consumption at a level higher than reported.
Moreover, some patients might have modified their consumptions between their last consultation and the time that they completed the questionnaire, in particular those whom the doctors had not seen recently. To try to take time since the last consultation into account (and thus limit the measurement bias for the agreement), we adjusted our analyses for the number of consultations during the previous year (which was significantly associated with agreement, results of adjustment not shown). We also conducted a sensitivity analysis, excluding patients who had not been seen in the past year (16.7% of patients); its results were nearly identical to those presented. Agreement was 62.7% for smoking status and 50.0% for alcohol consumption. The results of the final models also remained very similar in terms of the significance of the variables, and the direction and intensity of the associations.
Another particularity of our study is that we collected the doctor’s opinion (as well as the file information). This opinion might be affected by memory errors or based on prejudices .
Finally, some part of the disconcordance may be due to the fact that physicians do not regularly collect updated information on these topics, although such updating is recommended .
For some GPs, agreement might have been better because they had patients with the characteristics associated with better agreement and not because of their own characteristics. Agreement is best, for example, for the most advantaged patients . Physicians whose patients were largely well-off could thus have had better agreement that was not associated with their own characteristics. We took this confounding into account by using hierarchical models adjusted for different patient characteristics (such as occupation) that are known to be associated with agreement.
Two hypotheses might explain why we found so few GP characteristics associated with agreement. Our sample of physicians might be too homogeneous. All the participating GPs also supervise internships, which tends to encourage them to follow the clinical practice guidelines  that mandate the inclusion of smoking and alcohol status in medical files. Nonetheless this theory seems to be contradicted by the great variability observed between the GPs. More probably, the relatively low number of physicians included in the study limited the power of our statistical tests: in hierarchical models with random physician effects, power is linked above all else to the number of physicians analyzed [24,26].
The generalizability of our results might also be questioned, especially because all the participating GPs supervise interns. However, the characteristics of the participating GPs are not very different from those of all French GPs, particularly with regard to age, sex, fixed fees or not, and solo or group practice , and the agreement we observed for smoking status and alcohol consumption was quite similar to the agreement reported in the literature (as discussed below). Moreover, the association measures between agreement and GPs’ characteristics reported here do not appear likely to be influenced by internship supervision.
The kappa for smoking status that we observed (0.70) corresponds to substantial agreement .
This result is consistent with previous studies that have found kappa coefficients between 0.46 and 0.83 [7,8,29,30]. The kappa for alcohol consumption (0.33) corresponds to fair agreement and is lower than that reported in the study by Mant et al. (0.53) . This difference is probably due to the existence of the category of occasional excessive consumption, which the physicians selected rather rarely: only 9.5% of the patients who reported such consumption were so classified by their GPs (Table 3).
We found few reports to compare with our results about the physicians’ characteristics associated with agreement. The study by Ferrante et al., which is the closest to ours, did not look at alcohol consumption, nor did it find the organizational characteristics of the practice to be associated with agreement about smoking .
Physicians trained in managing patients who smoke had the best agreement rate, probably because they are more involved in screening and helping their patients to stop smoking . Similarly, physicians skilled in managing patients with drinking problems feel that they can appropriately be interested in their patients’ consumption. It is well known that one of the major obstacles to screening for alcoholism is physicians’ feeling of impotence in this field and their lack of training .
Agreement increases with physicians’ training and skills in management of patients with tobacco and alcohol problems. Our results thus underline the importance of professional training, which still requires improvement. Our results should also induce researchers to look at physicians’ characteristics when they ask GPs about their patients’ smoking status and alcohol consumption. The data in studies of phenomena associated with use of these substances would probably be of higher quality if the researchers recruited as investigators physicians trained or specifically skilled in these domain, for example, through a CME organization or a relevant care network.
Advisory committee on data processing in health research (Comité consultatif sur le traitement de l’information en matière de recherche dans le domaine de la santé)
Continuing medical education
French society of generalist treatment (Société de formation thérapeutique du généraliste)
World Health Organization
World Health Organization. WHO global report on mortality attributable to tobacco. Geneva, Switzerland: World Health Organization; 2012.
World Health Organization, Management of Substance Abuse Team. Global Status Report on Alcohol and Health. Geneva, Switzerland: World Health Organization; 2011.
Newell SA, Girgis A, Sanson-Fisher RW, Savolainen NJ. The accuracy of self-reported health behaviors and risk factors relating to cancer and cardiovascular disease in the general population: a critical review. Am J Prev Med. 1999;17:211–29.
Pakhomov SV, Jacobsen SJ, Chute CG, Roger VL. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care. 2008;14:530–9.
Murray RL, Coleman T, Antoniak M, Fergus A, Britton J, Lewis SA. The potential to improve ascertainment and intervention to reduce smoking in Primary Care: a cross sectional survey. BMC Health Serv Res. 2008;8:6.
Simpson CR, Hippisley-Cox J, Sheikh A. Trends in the epidemiology of smoking recorded in UK general practice. Br J Gen Pract J R Coll Gen Pract. 2010;60:e121–7.
Mant J, Murphy M, Rose P, Vessey M. The accuracy of general practitioner records of smoking and alcohol use: comparison with patient questionnaires. J Public Health Med. 2000;22:198–201.
Wilson A, Manku-Scott T, Shepherd D, Jones B. A comparison of individual and population smoking data from a postal survey and general practice records. Br J Gen Pract. 2000;50:465.
Ferrante JM, Ohman-Strickland P, Hahn KA, Hudson SV, Shaw EK, Crosson JC, et al. Self-report versus Medical Records for Assessing Cancer-Preventive Services Delivery. Cancer Epidemiol Biomarkers Prev. 2008;17:2987–94.
Tisnado DM, Adams JL, Liu H, Damberg CL, Hu A, Chen W-P, et al. Does concordance between data sources vary by medical organization type. Am J Manag Care. 2007;13(part 1):289–96.
Société de formation thérapeutique du Généraliste. Mobiliser les médecins traitants franciliens pour réduire les inégalités de prévention et dépistage. Paris, France: Société de formation thérapeutique du généraliste; 2010.
Société française d’alcoologie. Alcohol misuse except alcohol addiction: use and misuse. Alcoologie Addictologie. 2003;25:75S–87.
Cohen J. A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas. 1960;20:37–46.
Coughlin SS, Pickle LW, Goodman MT, Wilkens LR. The logistic modeling of interobserver agreement. J Clin Epidemiol. 1992;45:1237–41.
Snijders TAB, Bosker R. Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Secondth ed. Los Angeles: SAGE Publications Ltd; 2011.
Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. Thousand Oaks: SAGE Publications, Inc; 2001.
Diez R. A glossary for multilevel analysis. J Epidemiol Community Health. 2002;56:588–94.
Moerbeek M, van Breukelen GJP, Berger MPF. A comparison between traditional methods and multilevel regression for the analysis of multicenter intervention studies. J Clin Epidemiol. 2003;56:341–50.
Merlo J, Chaix B, Yang M, Lynch J, Råstam L. A brief conceptual tutorial of multilevel analysis in social epidemiology: linking the statistical concept of clustering to the idea of contextual phenomenon. J Epidemiol Community Health. 2005;59:443–9.
Soto CM, Kleinman KP, Simon SR. Quality and correlates of medical record documentation in the ambulatory care setting. BMC Health Serv Res. 2002;2:22.
Rigal L, Saurel-Cubizolles M-J, Falcoff H, Bouyer J, Ringa V. Do social inequalities in cervical cancer screening persist among patients who use primary care? The Paris Prevention in General Practice survey. Prev Med. 2011;53:199–202.
Kaguelidou F, Amiel P, Blachier A, Iliescu C, Rozé J-C, Tsimaratos M, et al. Recruitment in pediatric clinical research was influenced by study characteristics and pediatricians’ perceptions: a multicenter survey. J Clin Epidemiol. 2013;66:1151–7.
Agence nationale d’accréditation et d’évaluation en santé (ANAES): La Tenue Du Dossier Médical En Médecine Générale: État Des Lieux et Recommandations. Paris, France; 1996
Snijders TAB, Bosker RJ. Standard errors and sample sizes for two-level research. J Educ Behav Stat. 1993;18:237–59.
Van Ryn M, Burke J. The effect of patient race and socio-economic status on physicians’ perceptions of patients. Soc Sci Med 1982. 2000;50:813–28.
Kreft IGG, de Leeuw J. Introducing Multilevel Modeling. London; Thousand Oaks, Calif: Sage; 1998.
Institut national de prévention et d’éducation pour la santé. Baromètre santé médecins généralistes 2009. Saint-Denis: INPES éd; 2011.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.
Eze-Nliam C, Cain K, Bond K, Forlenza K, Jankowski R, Magyar-Russell G, et al. Discrepancies between the medical record and the reports of patients with acute coronary syndrome regarding important aspects of the medical history. BMC Health Serv Res. 2012;12:78.
Tisnado DM, Adams JL, Liu H, Damberg CL, Chen W-P, Hu FA, et al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44:132–40.
Stead M, Angus K, Holme I, Cohen D, Tait G. Factors influencing European GPs’ engagement in smoking cessation: a multi-country literature review. Br J Gen Pract. 2009;59:682–90.
Blanquet M, Peyrol MF, Gerbaud L, Morel F, Maradeix B, Llorca P-M, et al. Tackling the alcohol issue in France. Br J Gen Pract. 2014;64:16.
The authors are extremely grateful to all the GPs and patients who took part in this study, as well as to the Society for Training of General Practitioners (Société de Formation Thérapeutique du Généraliste, SFTG) for its logistic support and to the general medicine department of the Faculté de médecine Pierre and Marie Curie for its active collaboration in the performance of this study. The authors thank the Groupement régional de santé publique d'Ile de France, Conseil régional d’Ile de France and the National Institute for Prevention and Health Education (INPES, Institut national de prévention et d’éducation pour la santé) for their financial support.
The authors declare that they have no competing interests.
All authors participated in the development of the research question and the study design. FN, MF, HF and LR participated in the recruitment of physicians, data collection and data entry. JLT and LR performed the statistical analysis. JLT and LR drafted the manuscript, which was reviewed and approved by all authors.