Validating abortion procedure coding in Canadian administrative databases
© The Author(s). 2016
Received: 21 December 2015
Accepted: 17 June 2016
Published: 12 July 2016
The British Columbia (BC) Ministry of Health collects abortion procedure data in the Medical Services Plan (MSP) physician billings database and in the hospital information Discharge Abstracts Database (DAD). Our study seeks to validate abortion procedure coding in these databases.
Two randomized controlled trials enrolled a cohort of 1031 women undergoing abortion. The researcher collected database includes both enrollment and follow up chart review data. The study cohort was linked to MSP and DAD data to identify all abortions events captured in the administrative databases. We compared clinical chart data on abortion procedures with health administrative data. We considered a match to occur if an abortion related code was found in administrative data within 30 days of the date of the same event documented in a clinical chart.
Among 1158 abortion events performed during enrollment and follow-up period, 99.1 % were found in at least one of the administrative data sources. The sensitivities for the two databases, evaluated using a gold standard, were 97.7 % (95 % confidence interval (CI): 96.6–98.5) for the MSP database and 91.9 % (95 % CI: 90.0–93.4) for the DAD.
Abortion events coded in the BC health administrative databases are highly accurate. Single-payer health administrative databases at the provincial level in Canada have the potential to offer valid data reflecting abortion events.
KeywordsInduced abortion Clinical coding Database Data collection Reproducibility of results British Columbia
Accurate capture of abortion events within administrative data is important for both population health surveillance and for policy and program planning and evaluation. Abortion is a common procedure in Canada, with 92,524 reported in 2011 including 14,341 in the province of British Columbia (BC), and 37 % of women seeking a hospital-provided abortion in 2011 reporting having had at least one prior abortion . Interventions that assist women presenting for an abortion to avoid subsequent unintended pregnancies have the potential to address this problem, yet clinical follow up post-abortion has a very high attrition rate. Health administrative data could potentially be used to test the effectiveness of health policies and programs with potential to impact the number of abortions performed. However, we were unable to find any evidence that the validity of the data capture of these events within a single-payer health administrative data system in Canada has ever been documented.
Observation of outcomes after an index abortion presents unique challenges, due to very low rates of return for clinical follow up post abortion [2, 3]. Failure to return for follow up is frequently associated with lower socioeconomic status, of particular concern as women of low socioeconomic status are over-represented among the population presenting for abortion, and particularly for repeat abortion [2, 4, 5]. Comprehensive chart review among all clinical services offering abortion within a jurisdiction, while theoretically possible, is time-consuming, costly and impeded by the wide range of medical record systems used. Further, chart audit methodology is likely to decline in accuracy due to the recent approval in Canada of mifepristone (RU-486) for use to induce medical abortion, which may be performed in a wide range of primary care settings otherwise unrelated to settings self-identifying as abortion clinics .
Capture of abortion events using health administrative data has the potential to provide comprehensive data on both numbers and rates of abortion in relation to a wide range of determinants, as well as in relation to subsequent health system events. Administrative data capture could be associated with lower costs than chart review and with greater accuracy for subsequent events than clinical follow up. Administrative databases have the added advantage of allowing for extraction of additional information, such as delivery outcomes and hospitalization records, which could be of value to inform health policy and system decision making.
The accuracy of using health administrative data to capture abortion events is currently unknown as no validation of abortion procedure capture and coding in administrative data compared to verified clinical records has been undertaken. Previous Canadian studies on administrative data capture have shown the validity of administrative coding for cardiac and perinatal procedures [7–9]. Based on these results, we anticipate a high degree of accuracy in administrative coding for abortions.
We conducted two randomized controlled trials (RCTs) enrolling over a thousand women presenting for abortion in BC [5, 10–12]. We followed up each participant through the review of clinical chart records from all six abortion clinic facilities in BC for both index and subsequent abortions. We examined the analogous data for each participant as captured through the provincial health administrative linked databases of Population Data BC, and the BC Ministry of Health [13–16]. By comparing these different data sources, we evaluated the validity of using health administrative data to capture abortion events among residents of BC registered in the provincial health system.
From 2009 to 2012, two RCTs studying the effectiveness of intrauterine contraceptive devices enrolled a cohort of 1031 women presenting for surgical abortion at BC clinics [5, 10–12]. Women were screened using the eligibility criteria of the studies. Two main conditions for being eligible to participate were the intention not to conceive within the subsequent year and current registration with the BC provincial health plan (Medical Services Plan). Over three years, the two studies recruited 530 women undergoing a first trimester abortion and 501 women undergoing a second trimester abortion. Studies were carried out in five BC surgical abortion clinics, of which two are hospital-based and three are in the community setting. Chart review was conducted at these five clinics, as well as at an additional community based clinic which offers only medical abortion, to capture subsequent events among enrolled participants.
Women enrolling in the study gave consent for follow up clinically and through linkages using health administrative data, as well as through the completion of annual questionnaires. The participants were followed subsequently for five years including by an annual chart review at all six abortion clinics in BC. The chart review data set, including initial enrollment data as well as direct confirmation of individual initial and subsequent clinical events whenever it was required, constitutes the researcher-collected database for this study.
The trials were conducted under the supervision of an independent Data and Safety Monitoring Board. Ethics approval for the trials, including the current secondary analysis, was obtained from the University of British Columbia- Children’s and Women’s Hospital Research Ethics Board (REB) (H10-00306, H10-00798) and from the analogous REBs for all health authorities with study sites. Both study protocols, and the intake cohort characteristics for one RCT, have been published [5, 10–12].
Characteristics of administrative database capturing abortions data in BC
Medical Services Plan (MSP) Payment Information
Discharge Abstract Database
Individuals covered by the Medical Services Plan (MSP), BC’s universal insurance program
In-patients and day surgery patients in acute care hospitals in BC
Billing information for all medically required services (both procedure and diagnostic codes)
Data on discharges, transfers and deaths
Acute care hospitals
ICD-9 codes 635.x-638.x
ICD-10-CA code O04
fee item codes 4110-4114, 14545
CCI procedure codes beginning with 5CA.
The study cohort was linked to the administrative data in 2014 using personal health numbers (PHN) and date of birth as unique patient identifiers. Administrative data for all index and subsequent abortion procedures occurring for the study cohort between June 1, 2009 and September 30, 2013 were captured. We required the women’s consent during enrollment to perform this data linkage as the abortion data in these two health databases is not publicly available. Using hospital and anonymous physician codes, we excluded, for the purposes of this comparison, any procedures that did not occur at the six clinics for which we had access to clinical charts.
There are two sets of abortions events in our study that we can use to validate the procedure coding of abortions; the index abortions, occurring at the time of enrollment in either of the two RCTs; and the subsequent abortions, which occurred within the 5-year follow up timeline. All discrepancies between data sources were investigated through chart review at the relevant facility.
When comparing the administrative data with the researcher-collected medical chart data, we considered an abortion event to be matched if an abortion procedure code exists in the relevant administrative data source with a date that is accurate within ±30 days compared to the researcher-collected data. In cases with multiple abortion related procedural codes within a window of 30 days, we assumed it to be a single abortion event, an assumption that was universally supported by review of chart data. A common observation in the MSP data was the presence of abortion-related diagnostic codes (such as at the time of provision of consultation, follow up, or complications) in the absence of an abortion procedure code. We examined these cases but once all inconsistencies were resolved through specific clinical chart confirmation, we were able to restrict our consideration to cases with an abortion procedural code for the purpose of matching between databases.
We report discrepancies and the source, if known, of errors or omissions in the administrative data. In the analysis of index abortions, sensitivity is defined as the ratio of correct matches of the administrative data with the researcher-collected medical chart data. Since the abortions are recorded in two administrative databases in BC, we provide the sensitivity statistics for each data source separately as well as for the two databases combined.
In the analysis of subsequent abortions, we found abortions performed at study related facilities identified in administrative data that were not present in researcher-collected data. Therefore, for this cohort of abortions we conduct a two-direction analysis to measure the number of matched events in the administrative and researcher-collected databases. When reporting the results of our comparisons, we also include the exact binomial confidence intervals calculated using the Clopper-Pearson method. All the statistical analysis in this paper was conducted using the statistical software R.
We present the results of our analysis for index abortion events separately from our consideration of subsequent abortion events.
Sensitivity of administrative databases for the index abortions when compared to the researcher-collected database
Sensitivity (95 % CI)
MSP or DAD
Number of subsequent abortions events found in different data sources
Proportion (95 % CI)
Total subsequent abortions found in all databases
Matched in RCD
Matched in MSP or DAD
Matched in both admin and RCD
Event date difference
Event date difference among all data sources for index and subsequent abortions
This is the first study to examine the validity of administrative data capture of abortion procedure events in Canada. We found a high degree of accuracy, with over 99 % of procedures being correctly identified by at least one of the two administrative databases.
While the two administrative data sources combined had a high sensitivity (99.1 %), there was a significant difference in the concordance rates between the MSP and DAD for the index and subsequent abortion events. The accuracy of the hospital discharge data was lower compared to fee for service data (MSP), 91.9 % versus 97.7 % sensitivity among the index cases. The match rate for DAD was even less in the set of subsequent abortions, where only 61.4 % of cases were matched. The large difference in capturing the index and subsequent abortion events was expected. The majority of study participants had their index abortion in a hospital setting but only for these was the procedure captured in the hospital discharge data. Community setting clinics, where some of the index abortions and a large portion of the subsequent abortions occurred do not contribute to the hospital discharge data. Across Canada more than half of all abortions reporting location of service delivery are performed in clinic settings . Thus, the hospital discharge abstract database (DAD) is not reliable as an independent source for capturing abortions in Canada.
The high capture rate of abortions events by the combined BC administrative databases suggests that this technique can be used as a reliable source in abortion related studies in BC. Based on the access, cost and time needed to collect data from available sources, researcher-collected compared to administrative databases, the prospective data user can decide the most appropriate technique. Researcher-collected clinical chart data is likely to require individual consent as well as, or at least with, ethics review board approval. The variety of clinical data storage systems (paper and a number of different electronic medical records) may make collection of clinic-based data cumbersome and expensive. However, the clinical chart data is available immediately after the event, while data capture in administrative data sources may require 18–24 months prior to availability. Further, permission to access the data may require additional time . Thus the potential data user must weigh the various practical considerations to choose an appropriate data source, as our results do not present a significant variation in the accuracy of the data available from either source.
We were limited in this study by our inability to capture chart data from all abortion settings in BC. In 2010, less than 10 % of abortions in BC occurred in clinics not involved in this study . Among the study cohort, administrative data found only 1.7 % of women presented for a subsequent abortion at a location for which we did not have corresponding clinical chart data. Thus this limitation did not significantly affect our ability to validate the capture of subsequent abortion outcomes using these administrative data. Similarly, another limitation is the exclusion of non-BC residents among the index abortion sample. This exclusion (which was necessary as MSP information does not exist for non-residents) accounted for only 3.5 % of patients assessed for eligibility. We feel that these limitations impact neither the validity of administrative data use in the context of clinical trials, nor the capture and reporting of abortion data at the provincial level.
This analysis has focused on the validation of the capture of clinical abortion events in the BC government health administrative databases. The results may have limited application to the capture of abortion events in the administrative data of other jurisdictions. The Canadian Institute for Health Information (CIHI) has found that the use of fee for service data, analogous to MSP data in BC, to record abortions varies significantly between Canadian provinces . This accords with our findings where the combination of MSP and hospital discharge data provided a better capture of events. The ability to combine linked administrative data from both of these sources, and to compare to clinical chart recorded events, is a major strength of this study.
Abortion procedures are common and of interest to health researchers, health policy makers and health program planners. The capture of abortion procedure events in health administrative data in Canada has not been previously validated. The high degree of accurate capture (over 99 %) that we found validates the use of linked BC health administrative data to capture abortion procedure events.
BC, British Columbia; CI, confidence interval; CIHI, Canadian Institute for Health Information; DAD, Discharge Abstracts Database (representing hospital services and diagnostic data in BC); MSP, Medical Services Plan (representing physician services and diagnostic billing data in BC); PHN, personal health number; RCD, Research Collected Database; RCT, randomized controlled trial; REB, Research Ethics Board; WHRI, Women’s Health Research Institute
One RCT was supported by a Large Grant from the Society of Family Planning (SFP4-6). The other RCT was supported by an Operating Grant (MOP-106653) and a Primary Health Care Bridge Funding Grant (PCB-102963), both from the Canadian Institutes of Health Research (CIHR). Additionally, pilot study work was supported by a Sue Harris Family Practice Research Grant from the Women’s Health Research Instituted (WHRI) of the Provincial Health Services Authority of BC. Infrastructure support for this study is provided by the WHRI, the Women’s Services Clinic of Kelowna General Hospital, Kelowna, BC, and the Department of Family Practice, Faculty of Medicine, UBC. The levonorgestrel releasing IUC used in one of the RCTs were donated by Bayer Inc., Canada as their sole contribution to the study. None of the above organizations was involved in any manner with the study design, or with the collection, analysis, and interpretation of data; or in the writing of the manuscript; or in the decision to submit the manuscript for publication. Dr. Norman is supported as a Scholar of the Michael Smith Foundation for Health Research, and by the Canadian Institutes of Health Research and the Public Health Agency of Canada as a Chair in Applied Public Health Research (CPP-137903).
Availability of data and materials
All data used for this analysis are protected under the privacy policies of the Data Stewards of the BC health administrative data and Population Data BC, and within the terms of the institutional review board approval for this study, and are not publicly available.
All authors made substantial contributions to conception and design of this study. WVN contributed to acquisition of data. SP and RB helped with preparation of the analytic dataset. SS an SP carried out the analysis of data and produced the results. SS contributed to the design of methodology and drafted the manuscript. WVN, RB, and JK participated in the analysis of results and contributed to critical revision for important intellectual content. All authors have read and approved the final manuscript.
The author(s) declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Both studies received institutional review board approval from the University of British Columbia - Children’s and Women’s Research Ethics Board (H10-00798 and H10-00306), the Interior Health Authority Research Ethics Board (2010-28 and 2010-034), and Vancouver Island Health Authority Clinical Research Ethics Board (C2010-47). This analysis was additionally approved by the Data Stewards of the provincial health administrative data held within Population Data BC linked data holdings. All inferences, opinions, and conclusions drawn in this analysis and program recommendations are those of the authors, and do not reflect the opinions or policies of the Data Steward(s).
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Canadian Institute for Health Information. Induced abortions performed in Canada in 2011. 2014. https://www.cihi.ca/en/ta_11_alldatatables20130221_en.pdf. Accessed 2 Jun 2015.Google Scholar
- Stanek AM, Bednarek PH, Nichols MD, Jensen JT, Edelman AB. Barriers associated with the failure to return for intrauterine device insertion following first-trimester abortion. Contraception. 2009;79(3):216–20.View ArticlePubMedGoogle Scholar
- Norman WV. Induced abortion in Canada 1974–2005: trends over the first generation with legal access. Contraception. 2012;85(2):185–91.View ArticlePubMedGoogle Scholar
- Bednarek PH, Creinin MD, Reeves MF, Cwiak C, Espey E, Jensen JT. Immediate versus delayed IUD insertion after uterine aspiration. N Engl J Med. 2011;364(23):2208–17.View ArticlePubMedGoogle Scholar
- Norman WV, Chiles J, Turner C. The contraceptive experience among women seeking abortion. Contraception. 2011;84(3):314.Google Scholar
- Health Canada. Regulatory decision summary (SBD): MIFEGYMISO - 2015 - Health Canada. 2015. http://www.hc-sc.gc.ca/dhp-mps/prodpharma/rds-sdr/drug-med/rds_sdr_mifegymiso_160063-eng.php. Accessed 10 Jul 2016.Google Scholar
- Lee DS, Stitt A, Wang X, Yu JS, Gurevich Y, Kingsbury KJ, Austin PC, Tu JV. Administrative hospitalization database validation of cardiac procedure codes. Med Care. 2013;51(4):e22–26.View ArticlePubMedGoogle Scholar
- Joseph KS, Fahey J. Validation of perinatal data in the discharge abstract database of the Canadian Institute for Health Information. Chronic Dis Can. 2009;29(3):96–100.PubMedGoogle Scholar
- Frosst G, Hutcheon J, Joseph KS, Kinniburgh B, Johnson C, Lee L. Validating the British Columbia perinatal data registry: a chart re-abstraction study. BMC Pregnancy Childbirth. 2015;15(1):123.View ArticlePubMedPubMed CentralGoogle Scholar
- Norman WV, Kaczorowski J, Soon JA, Brant R, Bryan S, Trouton KJ, Dicus L. Immediate vs. delayed insertion of intrauterine contraception after second trimester abortion: study protocol for a randomized controlled trial. Trials. 2011;12(1):149. doi:https://doi.org/10.1186/1745-6215-12-149.View ArticlePubMedPubMed CentralGoogle Scholar
- Norman WV, Chiles JL, Turner CA, Brant R, Aslan A, Kaczorowski J. Comparing the effectiveness of copper intrauterine devices available in Canada. Is FlexiT non-inferior to NovaT when inserted immediately after first-trimester abortion? study protocol for a randomized controlled trial. Trials. 2012;13(1):147. doi:https://doi.org/10.1186/1745-6215-13-147.View ArticlePubMedPubMed CentralGoogle Scholar
- Norman WV, Brooks M, Brant R, Soon JA, Majdzadeh A, Kaczorowski J. What proportion of Canadian women will accept an intrauterine contraceptive at the time of second trimester abortion? baseline data from a randomized controlled trial. J Obstet Gynaecol Can. 2014;36(1):51–9.View ArticlePubMedGoogle Scholar
- British Columbia Ministry of Health. Medical Services Plan (MSP) Payment Information File. Population Data BC. Data Extract. MOH (2014). 2014. http://www.popdata.bc.ca/data.Google Scholar
- British Columbia Ministry of Health. Consolidation File (MSP Registration & Premium Billing). Population Data BC. Data Extract. MOH (2014). 2015. http://www.popdata.bc.ca/dataGoogle Scholar
- British Columbia Vital Statistics Agency. Vital Statistics Deaths. Population Data BC. Data Extract. BC Vital Statistics Agency (2014). 2014. http://www.popdata.bc.ca/data.Google Scholar
- Canadian Institute for Health Information. Discharge Abstract Database (Hospital Separations). Population Data BC. Data Extract. MOH (2014). 2014. http://www.popdata.bc.ca/data.Google Scholar
- Lisonkova S, Liu S, Bartholomew S, Liston RM, Joseph KS. Temporal trends in maternal mortality in Canada II: estimates based on hospitalization data. J Obstet Gynaecol Can. 2011;33(10):1020–30.View ArticlePubMedGoogle Scholar
- Smolina K, Hanley GE, Mintzes B, Oberlander TF, Morgan S. Trends and determinants of prescription drug use during pregnancy and postpartum in British Columbia, 2002–2011: a population-based cohort study. PLoS One. 2015;10(5):e0128312.View ArticlePubMedPubMed CentralGoogle Scholar
- Leung VW, Soon JA, Lynd LD, Marra CA, Levine M. Population-based evaluation of the effectiveness of two regimens for emergency contraception. Int J Gynecol Obstet. 2016;133(3):342–6.View ArticleGoogle Scholar
- Bedouch P, Marra CA, FitzGerald JM, Lynd LD, Sadatsafavi M. Trends in asthma-related direct medical costs from 2002 to 2007 in British Columbia, Canada: a population based-cohort study. PLoS One. 2012;7(12):e50949.View ArticlePubMedPubMed CentralGoogle Scholar
- Chamberlayne R, Green B, Barer ML, Hertzman C. Creating a population-based linked health database: a new resource for health services research. Can J Public Health. 1998;89(4):270.PubMedGoogle Scholar
- Liu S, Liston RM, Joseph KS, Heaman M, Sauve R, Kramer MS, Maternal Health Study Group of the Canadian Perinatal Surveillance System. Maternal mortality and severe morbidity associated with low-risk planned cesarean delivery versus planned vaginal delivery at term. Can Med Assoc J. 2007;176(4):455–60.View ArticleGoogle Scholar
- Doctors of BC. Doctors of BC Guide to Fees. 2015. https://www.doctorsofbc.ca/resource-centre/physicians/handbooks-guides. Accessed 1 Jun 2015.Google Scholar
- Population Data BC. The data access request (DAR) process. 2014. https://www.popdata.bc.ca/dataaccess/process. Accessed 10 Jul 2016.Google Scholar
- Norman WV, Soon JA, Maughn N, Dressler J. Barriers to rural induced abortion services in Canada: findings of the British Columbia abortion providers survey (BCAPS). PLoS One. 2013;8(6):e67023. doi:https://doi.org/10.1371/journal.pone.0067023.View ArticlePubMedPubMed CentralGoogle Scholar