This article has Open Peer Review reports available.
Quality and correlates of medical record documentation in the ambulatory care setting
© Soto et al; licensee BioMed Central Ltd. 2002
Received: 9 September 2002
Accepted: 10 December 2002
Published: 10 December 2002
Documentation in the medical record facilitates the diagnosis and treatment of patients. Few studies have assessed the quality of outpatient medical record documentation, and to the authors' knowledge, none has conclusively determined the correlates of chart documentation. We therefore undertook the present study to measure the rates of documentation of quality of care measures in an outpatient primary care practice setting that utilizes an electronic medical record.
We reviewed electronic medical records from 834 patients receiving care from 167 physicians (117 internists and 50 pediatricians) at 14 sites of a multi-specialty medical group in Massachusetts. We abstracted information for five measures of medical record documentation quality: smoking history, medications, drug allergies, compliance with screening guidelines, and immunizations. From other sources we determined physicians' specialty, gender, year of medical school graduation, and self-reported time spent teaching and in patient care.
Among internists, unadjusted rates of documentation were 96.2% for immunizations, 91.6% for medications, 88% for compliance with screening guidelines, 61.6% for drug allergies, 37.8% for smoking history. Among pediatricians, rates were 100% for immunizations, 84.8% for medications, 90.8% for compliance with screening guidelines, 50.4% for drug allergies, and 20.4% for smoking history. While certain physician and patient characteristics correlated with some measures of documentation quality, documentation varied depending on the measure. For example, female internists were more likely than male internists to document smoking history (odds ratio [OR], 1.90; 95% confidence interval [CI], 1.27 – 2.83) but were less likely to document drug allergies (OR, 0.51; 95% CI, 0.35 – 0.75).
Medical record documentation varied depending on the measure, with room for improvement in most domains. A variety of characteristics correlated with medical record documentation, but no pattern emerged. Further study could lead to targeted interventions to improve documentation.
Keywordsambulatory care documentation medical records correlation studies quality of care
Documentation in the medical record facilitates diagnosis and treatment, communicates pertinent information to other caregivers to ensure patient safety and reduce medical errors, and serves an important medical-legal function in risk management . Quality of documentation may also reflect the quality of care delivered, although recent studies have suggested that medical record documentation in the outpatient setting tends to underestimate the actual performance of preventive health care services and other indicators of quality care [2–4]. Electronic medical record (EMR) systems may improve the quality of care delivered as well as the documentation of that care in the outpatient setting, but few studies have examined this issue [5, 6].
Determining the correlates of quality medical record documentation could thus lead to educational programs and other interventions to improve documentation, but few studies have rigorously examined the correlates of quality of chart documentation . We therefore undertook the present study to measure the rates of documentation of quality of care measures in an outpatient primary care practice setting that utilizes an EMR. We also aimed to examine the physician- and patient-level variables that correlated with the quality of medical record documentation. We studied the abstracted outpatient electronic medical records of 834 patients who received care from 117 internists and 50 pediatricians at 14 practice locations in 1998.
Medical records were abstracted in 1998 at 14 sites of Harvard Vanguard Medical Associates (HVMA, formerly known as Harvard Community Health Plan, and later, the Health Centers Division of Harvard Pilgrim Health Care) in greater Boston, Massachusetts. In 1998, more than 90% of patients at HVMA had capitated health insurance coverage from a single health maintenance organization, Harvard Pilgrim Health Care.
At the time of the study, HVMA was transitioning from one electronic medical record system, the COSTAR-based Automated Medical Record System [8–10] to the current system, EpicCare® . EpicCare® records constituted the majority of those reviewed for this study. Both systems have designated, coded fields for capturing the quality measures described below, and both systems have free-text fields for clinicians to enter narrative information from patient encounters. Physicians themselves enter the majority of data in EpicCare®, either by populating specific fields (e.g., smoking status, including number of cigarettes per day and duration of smoking; alcohol consumption [yes/no and ounces per week]) or by keyboard entry of free text. Depending on the physician's style, most data entry occurs immediately upon completion of the office visit, while some occurs during the visit in the presence of the patient and some occurs at the end of a clinical session. A minority of office notes are dictated; these transcribed notes are entered only as narrative text and do not result in any coding of specific fields.
Physicians and patients
At the time of the study, 212 physicians provided primary care at HVMA, of whom 167 had completed their malpractice insurance carrier's biennial survey of clinical and teaching activities; these 167 primary care physicians (117 internists and 50 pediatricians) constitute the study sample. With three exceptions, five patient records per physician were selected at random for review; four records were reviewed for each of two physicians and six records for one physician. A total of 834 patient records were reviewed.
Data collection and physician-level variables
Data sets containing medical record review information and self-reported physician teaching and patient care activities were obtained through permission of the Harvard Risk Management Foundation. Medical record reviews were conducted as a component of quality assurance procedures to fulfill accreditation requirements of the National Center for Quality Assurance (NCQA). Trained nurse abstractors from a medical record consulting firm conducted the reviews using an abstraction instrument containing 51 questions modeled after NCQA's 1999 Health Plan Employer Data and Information Set (HEDIS) quality measures . Nurse abstractors reviewed both the free-text and coded-field components of the medical records.
Physicians reported the amount of direct patient care they provided in hours per week; they reported the amount of teaching they did as measured by the number of patients per month for whom they supervised care, as opposed to directly provided care. Physicians were classified as teachers if they reported supervising the care of any patients each month or supervising patient care in any months of the year. Specialty, gender and year of medical school graduation were obtained for each physician from the World Wide Web sites of the American Medical Association and the medical licensing boards of Massachusetts, New York, and California [13–16]. Practice site was dichotomized according to whether or not the site was located within Boston city limits, because we anticipated significant differences in patient populations and physician practices across sites in these locations.
Measures of medical record documentation quality
Before examining the data, we selected for analysis five measures from the chart review instrument that we believed were the most important indicators of quality of medical record documentation in ambulatory care. The five measures were medications, allergies, immunizations, smoking history and compliance with age-appropriate screening guidelines. All measures were dichotomous (yes/no) variables that were coded "yes" if the trained abstractors determined that the relevant information was appropriately documented. For medication documentation, we created a dichotomous (yes/no) variable that was scored "yes" if the record contained documentation of current medications either in the encounter text of the complete history and physical examination or in the medication list (i.e., medication field) of the computerized medical record. Allergy documentation was coded in a similar fashion. The other three measures – immunizations, smoking history and compliance with age-appropriate screening guidelines – are analogous to items found in the 1999 HEDIS List of Measures under "Effectiveness of Care" . For "compliance with age-appropriate screening guidelines," abstractors referred to Harvard Vanguard's clinical guidelines at the time of the study. These guidelines included colorectal cancer screening for adults age 50 years or older, mammography for women age 50 years or older, and Pap smears for all women. Prostate cancer screening was not included in this measure. For children, age-appropriate screening included sensory screening (vision and hearing), and lead testing. The "compliance with age-appropriate screening guideline" variable was coded "yes" if the abstractor determined that each relevant screening item had been documented.
There were no missing data. Chart abstractors coded "not applicable" for one or more measure on less than one percent of the records reviewed. We recoded this small number of "not applicable" values as "no" on the assumption that any items not coded as "yes" were lacking appropriate documentation of the specified measures.
Characteristics of physicians were compared by specialty using Student's t-test for continuous measures and by Fisher's exact test for categorical variables.
Analyses of documentation behavior were stratified by specialty because we anticipated differences between pediatricians and internists due to the nature of their patient populations, methods of practice, and the documentation measures we selected for analysis. No analyses were performed for immunization documentation by pediatricians since all pediatricians documented immunizations.
To account for clustering of patients within each physician, we used mixed effects (hierarchical) logistic modeling for documentation outcomes . This technique accounts for the correlation between outcomes measured on patients who share a physician . We determined whether physician teaching status, physician gender, years since medical school, clinical hours per week of direct patient care, practice site, patient age, and patient gender were correlated with better chart documentation for each of the five measures. We hypothesized a priori that physicians' teaching status would be correlated with the quality of documentation. Other physician characteristics were included because we expected that documentation practice would vary by clinician experience and by geography, although we did not have an a priori hypothesis about the direction of these effects.
The study was approved by the Human Studies Committee (Institutional Review Board) of Harvard Pilgrim Health Care.
Characteristics of physicians and patients
Characteristics of 167 HVMA physicians in a study of medical records documentation.
Internists (N = 117) N (%) or mean ± sd
Pediatricians (N = 50) N (%) or mean ± sd
Years Since Medical School
18.0 ± 8.2
22.3 ± 8.6
Clinical Hours Per Week†
34.3 ± 13.8
29.0 ± 9.4
Practice Site Outside Boston City Limits
The mean (± SD) age of patients was 9.3 ± 5.5 years in the pediatrics group and 57.5 ± 15.0 in the internist group. Fifty-five percent of all patients were female.
Overall rates of medical record documentation quality
Unadjusted rates of completion of medical record documentation among internists and pediatricians for each of five selected measures.*
Internists (N = 117) N (%) or mean ± sd
Pediatricians (N = 50) N (%) or mean ± sd
Correlates of medical record documentation quality
No consistent patterns emerged among the physician or patient characteristics and documentation quality measures in this study (Table 3 – see Additional File). Female internists were more likely than male internists to document smoking history (odds ratio [OR], 1.90; 95% confidence interval [CI], 1.27 – 2.83) but were less likely to document drug allergies (OR, 0.51; 95% CI, 0.35 – 0.75). With each subsequent year since completing medical school, internists were less likely to document drug allergies (OR, 0.97; 95% CI, 0.95 – 0.99) and immunizations (OR, 0.91; 95% CI, 0.87–0.96). For internists, increased clinical time was associated with better documentation for smoking history (OR, 1.02; 95% CI, 1.01 – 1.03) but not for any of the other measures. As compared with internists practicing at sites within Boston city limits, those practicing outside Boston city limits were less likely to document, medications (OR, 0.47; 95% CI, 0.26 – 0.85), and immunizations (OR, 0.19; 95% CI, 0.07–0.47) but were more likely to document smoking history (OR, 1.55; 95% CI, 1.08 – 2.20).
Internists were more likely to document drug allergy status (OR, 2.48; 95% CI, 1.65 – 3.74), medications (OR, 1.49; 95% CI, 1.18 – 1.87), and compliance with screening guidelines (OR, 1.36; 95% CI, 1.06 – 1.73) for female patients than for male patients. However, internists were less likely to document smoking history (OR, 0.51; 95% CI, 0.34 – 0.79) and immunizations (OR, 0.61; 95% CI, 0.47–0.79) for female patients than for male patients. Internists were less likely to document smoking history (OR, 0.98; 95% CI, 0.96 – 0.99) but more likely to document immunizations (OR, 1.02; 95% CI, 1.01 – 1.03) with every yearly increase in patient age.
Female pediatricians were less likely than male pediatricians to document drug allergy status (OR, 0.47; 95% CI, 0.28 – 0.79) but were more likely to document smoking history (OR, 2.78; 95% CI, 0.85 – 9.11); interpretation of this latter result is limited by imprecision of the effect size as manifested by the wide confidence interval. For pediatricians, increased clinical time was associated with better documentation for compliance with screening guidelines (OR, 1.22; 95% CI, 1.06 – 1.41) but not for any of the other measures. When compared with pediatricians practicing in the City of Boston, pediatricians practicing outside Boston city limits were more likely to document drug allergies (OR, 3.17; 95% CI, 1.05 – 9.57) and medications (OR, 7.93; 95% CI, 1.87 – 33.68) but less likely to document compliance with screening guidelines (OR, 0.08; 95% CI, 0.01 – 0.58). Pediatricians were more likely to document smoking history with increasing patient age (OR, 1.61; 95% CI, 1.12 – 2.31). Pediatricians were more likely to document drug allergy status for female patients than for male patients (OR, 1.62; 95% CI, 1.19 – 2.20) but less likely to document compliance with age-appropriate screening guidelines for female patients (OR, 0.17; 95% CI, 0.08 – 0.38).
In this study of the quality of medical record documentation among 167 primary care physicians, we found very high levels of documentation for immunizations and compliance with screening guidelines, but lower levels of documentation for medications, allergies, and smoking status. Although no characteristics consistently correlated with all measures of documentation quality, some characteristics had important relationships to certain measures. For example, female internists were more likely than male internists to document smoking history but were less likely to document drug allergies.
Pediatricians and internists excelled in different domains of documentation. For example, all 50 pediatricians fulfilled the documentation measure for immunizations for all 250 patients whose records were sampled. In comparison, internists met documentation standards for immunization for 562 of 584 patients (96%). These rates of immunization documentation are similar to previously reported rates of immunization for patients in this practice group , thereby supporting the validity of this study, and compare favorably with national benchmarks for immunization.
Pediatricians and internists also differed in their patterns of documenting smoking status. Documentation of smoking status may be considered a more appropriate measure of medical record documentation quality for internal medicine than for pediatrics, since very young children (e.g., infants and toddlers) are not at risk for smoking cigarettes themselves. However, national guidelines urge pediatricians to address children's exposure to passive smoking during routine pediatric practice, beginning at the prenatal visit . Pediatricians were more likely to document smoking status for their older patients, providing construct validity to this measure. Curiously, however, internists were less likely to document smoking status with increasing age of the patient. These findings may suggest that pediatricians and internists focus their smoking documentation, and perhaps their smoking prevention and cessation efforts, on adolescents and young adults, target populations advocated by national guidelines [22, 23].
Although no prior studies have comprehensively examined the correlates of medical record documentation quality in the primary care setting, one small study measured medical residents' chart-documentation practices . That study of 26 medical records suggested that documentation was more complete in charts of male patients than in female patients, a finding not confirmed in the present study. A recent study reported the medical record documentation practices of resident physicians and faculty members in a university-based internal medicine training program . In that chart abstraction study, which used measures similar to those in the present study, rates of documentation also varied widely, although that study emphasized performance scores for both preventive health services as well as chronic disease management. Others have found that certain educational interventions improved documentation quality in the ambulatory care setting [26, 27]. For example, clinical audit and standardized follow up improved smoking history documentation from 28% to 88% in charts of patients with diabetes . The low pre-intervention rate of smoking documentation in that study is consistent with the rates we observed among internists (38%) and pediatricians (20%) in the present study.
This study provided a novel and valuable glimpse into the quality of medical record documentation of primary care physicians using an electronic medical record system in an ambulatory care setting and enabled the assessment of the correlation between physician and patient characteristics and the quality of medical record documentation. Strengths of the study included the sampling of physicians and patients from 14 practice sites, including both pediatrics and internal medicine, in both urban and suburban locations; the systematic medical record review and abstraction process using measures of documentation quality that resonate with widely accepted NCQA measures of quality of care; and the hierarchical (multi-level) statistical modeling techniques used to control for clustering and for potential confounding by variables at the level of the physician and the patient.
Several important limitations should be considered in interpreting the results. The first is the lack of evidence of inter-rater reliability among chart abstractors. Although chart abstractors were registered nurses specially trained in medical record review, it is conceivable that different abstractors judged satisfactory documentation of a measure differently. Misclassification of documentation quality measures could have biased the results toward the null, and a true relationship between some of the physician or patient variables and documentation quality may not have been detected by this study.
A second limitation is that documentation in the medical record may not accurately reflect care delivered. One study showed that the validity of the medical record for measuring delivery of different health services depended on the service in question. Specifically, medical record review more accurately reflected the administration of immunizations than the provision of smoking cessation advice .
A third limitation is the possibility that unmeasured variation in "case-mix" could have accounted for variation in documentation quality. For example, physicians with generally sicker patients might be less likely to document health maintenance items, such as compliance with screening guidelines. Conversely, physicians might be expected to document more carefully the medications, allergies, and immunization status of more chronically ill patients.
Finally, the physicians' self report of supervising patient care may not have accurately reflected their true status as teachers. Although social desirability bias may have led those physicians who do any teaching to overestimate their actual teaching time, we believe it is less likely that physicians who do no teaching would actually report some teaching time. Dichotomizing this exposure variable should have helped to minimize the social desirability bias.
In summary, medical record documentation varied depending on the measure in question, with room for improving documentation in most domains. Despite the presence of an electronic medical record designed to facilitate documentation, rates of documentation for drug allergy and smoking status fell below desirable levels. We found that no consistent pattern of correlates of medical record documentation quality emerged in this study. Future studies could investigate whether other factors, such as socioeconomic status and insurance carrier of patients, correlate with medical record documentation quality in ambulatory care settings. Clarification of the correlates of medical record documentation quality can enhance and focus educational strategies to improve clinicians' documentation practices.
Mr. Soto's work was supported by a summer research fellowship from the Harvard Medical School Office of Enrichment Programs. The authors gratefully acknowledge the Harvard Risk Management Foundation for providing the data sets for this study. Mr. Anthony Chamberas generously and patiently helped us to understand the structure of the data sets and the definitions of the variables. We are indebted to Drs. Robert Fletcher and Antoinette Peters for helpful comments on earlier versions of the manuscript.
- Wood DL: Documentation guidelines: evolution, future direction, and compliance. American Journal of Medicine. 2001, 110: 332-334. 10.1016/S0002-9343(00)00748-8.View ArticlePubMedGoogle Scholar
- Dresselhaus TR, Peabody JW, Lee M, Wang MM, Luck J: Measuring compliance with preventive care guidelines: standardized patients, clinical vignettes, and the medical record. Journal of General Internal Medicine. 2000, 15: 782-788. 10.1046/j.1525-1497.2000.91007.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P: How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. American Journal of Medicine. 2000, 108: 642-649. 10.1016/S0002-9343(00)00363-6.View ArticlePubMedGoogle Scholar
- Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M: Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000, 283: 1715-1722. 10.1001/jama.283.13.1715.View ArticlePubMedGoogle Scholar
- Gill JM, Ewen E, Nsereko M: Impact of an electronic medical record on quality of care in a primary care office. Delaware Medical Journal. 2001, 73: 187-194.PubMedGoogle Scholar
- Brown PJ, Harwood J, Brantigan P: Data quality probes – a synergistic method for quality monitoring of electronic medical record data accuracy and healthcare provision. Medinfo. 2001, 10: 2-9.Google Scholar
- Cradock J, Young AS, Sullivan G: The accuracy of medical record documentation in schizophrenia. Journal of Behavioral Health Services & Research. 2001, 28: 456-465.View ArticleGoogle Scholar
- Barnett GO, Winickoff R, Dorsey JL, Morgan MM, Lurie RS: Quality assurance through automated monitoring and concurrent feedback using a computer-based medical information system. Med Care. 1978, 16: 962-970.View ArticlePubMedGoogle Scholar
- Barnett GO: The application of computer-based medical-record systems in ambulatory practice. New England Journal of Medicine. 1984, 310: 1643-1650.View ArticlePubMedGoogle Scholar
- Barnett GO, Justice NS, Somand ME, et al: COSTAR: a computer-based medical information system for ambulatory care. Proc IEEE. 1979, 67: 1226-1237.View ArticleGoogle Scholar
- 5-20-2002, [http://www.epicsystems.com/software/emr.php]
- 11-01-2001, [http://www.ncqa.org/communications/news/h99meas.htm]
- 11-01-2001, [http://www.ama-assn.org/aps/amahg.htm]
- 11-01-2001, [http://www.docboard.org/ma/df/masearch.htm]
- 11-01-2001, [http://www.op.nysed.gov/opsearches.htm]
- 11-01-2001, [http://www.docboard.org/ca/df/casearch.htm]
- Laird NM, Ware JH: Random-effects models for longitudinal data. Biometrics. 1982, 38: 963-974.View ArticlePubMedGoogle Scholar
- Breslow NE, Clayton DG: Approximate inference in generalized linear mixed models. J Am Stat Assoc. 1993, 88: 9-25.Google Scholar
- SAS Institute I: The GLIMMIX macro:. Cary, NC, SAS Institute, Inc. 11-01-2001, [http://ftp.sas.com/techsup/download/stat/glmm800.sas]
- SAS Institute I: SAS/STAT User's Guide, Version 8.2, vol 2. [8.2]. Cary, NC, SAS Institute. 2001Google Scholar
- 10-09-2001, [http://www.harvardpilgrim.org/employers/about/HEDIS2000.doc]
- American Academy of Pediatrics Committee on Substance Abuse: Tobacco, Alcohol, and Other Drugs: The Role of the Pediatrician in Prevention and Management of Substance Abuse. Pediatrics. 1998, 101 (1): 125-128.View ArticleGoogle Scholar
- Centers for Disease Control and Prevention: Health-care provider advice on tobacco use to persons aged 10–22 years – United States, 1993. MMWR. 1995, 44: 826-830.Google Scholar
- Moran MT, Wiser TH, Nanda J, Gross H: Measuring medical residents' chart-documentation practices. Journal of Medical Education. 1988, 63: 859-865.PubMedGoogle Scholar
- Kogan JR, Reynolds EE, Shea JA: Resident and Faculty Adherence to Common Guidelines. Acad Med. 2001, 76: S27-S29.View ArticlePubMedGoogle Scholar
- Grebe SK, Smith RB: Clinical audit and standardised follow up improve quality of documentation in diabetes care. New Zealand Medical Journal. 1995, 108: 339-342.PubMedGoogle Scholar
- Opila DA: The impact of feedback to medical housestaff on chart documentation and quality of care in the outpatient setting. Journal of General Internal Medicine. 1997, 12: 352-356. 10.1046/j.1525-1497.1997.00059.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, et al: How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Medical Care. 1998, 36: 851-867. 10.1097/00005650-199806000-00009.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/2/22/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.