- Research article
- Open Access
- Open Peer Review
Effect of drug utilization reviews on the quality of in-hospital prescribing: a quasi-experimental study
BMC Health Services Researchvolume 6, Article number: 33 (2006)
Drug utilization review (DUR) programs are being conducted in Canadian hospitals with the aim of improving the appropriateness of prescriptions. However, there is little evidence of their effectiveness. The objective of this study was to assess the impact of both a retrospective and a concurrent DUR programs on the quality of in-hospital prescribing.
We conducted an interrupted time series quasi-experimental study. Using explicit criteria for quality of prescribing, the natural history of cisapride prescription was established retrospectively in three university-affiliated hospitals. A retrospective DUR was implemented in one of the hospitals, a concurrent DUR in another, whereas the third hospital served as a control. An archivist abstracted records of all patients who were prescribed cisapride during the observation period. The effect of DURs relative to the control hospital was determined by comparing estimated regression coefficients from the time series models and by testing the statistical significance using a 2-tailed Student's t test.
The concurrent DUR program significantly improved the appropriateness of prescriptions for the indication for use whereas the retrospective DUR brought about no significant effect on the quality of prescribing.
Results suggest a retrospective DUR approach may not be sufficient to improve the quality of prescribing. However, a concurrent DUR strategy, with direct feedback to prescribers seems effective and should be tested in other settings with other drugs.
Prescription drugs constitute an important component of health care. However, drugs can only benefit to patients if they are used appropriately which involves that physicians prescribe them according to evidence. One common method of assessing and correcting the appropriateness of drug prescription is drug utilization review (DUR) programs [1–3].
In a retrospective DUR, observed patterns of drug prescriptions together with an evaluation of their appropriateness are sent to physicians in the form of a report of collated practice patterns. Since the prescriptions assessed were issued in the past, a retrospective DUR is relevant at preventing inappropriate prescribing in the future . A DUR can also be run concurrently to the intervention. The appropriateness of prescriptions is evaluated after the drug has been dispensed but while the patient is still hospitalized. The patient may then benefit from any corrective action. Such action may take the form of individual feedback from a pharmacist to the physician. A concurrent DUR aims both at improving current prescribing patterns and at preventing inappropriate prescribing in the future .
The effectiveness of DUR programs has yet to be established. The few evaluation studies of those programs conducted until now have been criticized for lack of rigor. In general, there are no adequate control groups and prior trends in the quality of prescribing is not taken into account [2, 4–6]. The present study was designed to avoid those limitations by comparing the effectiveness of retrospective and concurrent DUR to improve the quality of physician drug prescriptions in hospital settings.
We conducted an interrupted time-series quasi-experimental study  in three hospitals. The natural history of the prescription was established prior to the intervention in two experimental hospitals. A retrospective DUR was then implemented in one of the hospitals, a concurrent DUR in another, whereas the third hospital served as a control. The three hospitals are independently managed from each other although they are all affiliated with Laval University. They deliver tertiary care to a population of around 750,000 inhabitants in Quebec City area.
Selection of the drug
To remain as close as possible to the routine clinical practice setting, the targeted drug was selected after consulting pharmacists in all three settings to identify a drug that: 1) would be a clinically relevant target for a DUR, 2) had been prescribed in the hospitals for at least two years, 3) had not been subjected to any prior DUR or intervention aiming at modifying physicians prescribing behaviors, and 4) was being prescribed at least ten times per month. Cisapride was the only drug to meet these criteria in all three settings. Cisapride is a gastrointestinal tract promotility agent that was frequently used in Canada before its withdrawal from the market in 2000, after completion of this study.
Criteria for the appropriate prescription of cisapride were made available to us by the Quebec hospitals network on DUR . The criteria had been developed by experts, and approved by the network's scientific committee. We used criteria pertaining to indication for use, combination therapy, dosage, and drug interactions (Table 1). The Pharmacology and Therapeutic Committees in both the concurrent and the retrospective DUR hospitals approved those criteria.
Retrospective DUR hospital
On March 2, 1998, the Pharmacy department in the retrospective DUR hospital distributed a newsletter informing all physicians and pharmacists on the upcoming DUR on cisapride. The newsletter included criteria for the appropriate prescribing of cisapride. Using the explicit criteria, all prescriptions dispensed between March 16 and May 24, 1998 were retrospectively collected and assessed by a pharmacist. Based on this assessment, on June 29, 1998, a report of collated practice patterns of the appropriateness of observed patterns of drug prescriptions was presented in the Pharmacy Department newsletter that was distributed to clinicians in the hospital.
Concurrent DUR hospital
On May 4, 1998, criteria were distributed to all prescribers and pharmacists in the concurrent DUR hospital. Between May 11 and August 14, 1998, at the end of each day, a designated pharmacist had to screen all cisapride prescriptions for appropriateness. When a prescription was deemed inappropriate, the pharmacist had to communicate directly with the physician to discuss the deviation from criteria and propose alternative solutions. Physicians were free to change their prescription. In February 1999, the pharmacy department distributed a newsletter to physicians and pharmacists. The aggregate results on the quality of cisapride prescriptions issued during the 3-month intervention period of May to August 1998 were presented in this newsletter.
We identified patient medical records using the computerized pharmacy database of each hospital. In the control hospital, we reviewed the records of patients who had been prescribed cisapride between November 1994 and September 1999. There was no intervention in the control hospital. In the retrospective DUR hospital, the period of observation was from October 1994 to February 1999 with the first intervention implemented at the 41rst month while the period of observation was from June 1995 to September 1999 in the concurrent DUR hospital, with the first intervention being implemented at the 35th month.
Periods of observation differed slightly among hospitals mainly because pharmacy computer systems had not been implemented at the same time in each hospital. Moreover, cisapride was de-listed in February 1999 from the hospital formulary of the retrospective DUR hospital. A medical archivist abstracted records of all patients who were prescribed cisapride in each hospital during each hospital study period. Information needed to apply DUR criteria (see Table 1) was abstracted. Data included patient age, gastro-intestinal diagnoses, symptoms and procedures with date of occurrence, contra-indicated drugs, cisapride doses, date and time of administration, international normalized ratios, and drugs interacting with cisapride. Data was entered directly on a portable computer using Epi-Info software. The archivist was not aware of the study objectives and of the DUR criteria. Appropriateness of use was later assessed using data collected by the archivist. Explicit criteria were applied by computer.
Appropriateness of use
The unit of analysis is the monthly aggregated prescriptions for all physicians in a given hospital. We divided the study period into equal observation periods of four weeks. For each period, we assessed the proportion of appropriate prescriptions according to each one of the four criteria. For each patient, the appropriateness of the cisapride prescription was assessed both at the first day of treatment (day 1) and three days later (day 4). This second day-4 assessment was performed to allow for the measurement of the effect of the intervention during the course of treatment in the concurrent intervention hospital.
We first estimated the overall average proportion of appropriateness for the entire study period. This allowed a general comparison of the percentage of appropriate prescriptions according to each one of the criteria on day 1 and day 4 of treatment. Since appropriateness of prescriptions was almost perfect as to dosage and drug interactions at both day 1 and day 4, we did not carry out any further analysis on these two criteria and concentrated on the remaining two criteria: indication for use and combination therapy.
To test the effect of each type of DURs on the appropriateness of cisapride prescriptions, we used ARIMA modeling techniques [9–11]. For each hospital, we constructed 4 series of at least 52 observation periods (one series for each of the two criteria applied at day 1 and one series for each of the two criteria applied at day 4).
We estimated the ARIMA (autoregressive, seasonality, and moving average) coefficients for the four series in each of the three study hospitals. After obtaining a fitting model for each of these 12 series, we created indicator series for each experimental hospital. These latter series contained only 0s and 1s, 0s for all time periods prior to the introduction of the DUR in the hospital and 1 afterward. Since two separate activities correspond to the introduction of the DUR, the distribution of criteria and the diffusion of the results, two series were created in each experimental hospital. Four such series were created for the control hospital, one corresponding to each of the experimental series. Two coefficients were then introduced in the equations to model the interventions: t ω0 as an estimate of the average differences in the proportion of appropriate prescriptions (according to criteria) before and after the intervention, and δ1 to estimate whether the effect of the intervention is constant (δ1 = 0), expanding (δ1 > 0) or fading away (δ1 < 0).
In order to illustrate the evolution of the time series in a way that is more illuminating, we built graphical representations of the time series through a smoothing method. In order to attenuate the noise created by the random components in the raw data, we used Tukey's smoothing procedure . This lag 3 smoothing technique consists in replacing the Ith value in the series with the mean of the i-1, i and i+1 values. For each series, we performed 3 mean smooths with a lag of 3. It should be noted that the purpose of the graphic representations is for ease of interpretation only, the ARIMA models were built using raw data, and the parameters of the models were used to draw the conclusions about the effect of the DUR on prescriptions appropriateness.
We used t-tests to compare the ω0 coefficients of the experimental series with those of the control series. A significant t indicates a significant intervention effect over and above the secular trend captured by the variation in the control series. We conducted the analyses using the SAS software package .
For each hospital, the overall average proportion of appropriate prescriptions by criterion of appropriateness is displayed in Table 2. For the entire study period, appropriateness was almost perfect with regards to dosage and drug interactions at both day 1 and day 4 in all three hospitals. For the other two criteria, appropriateness seems to be improving between day 1 and day 4 in all hospitals including the control one. Overall, appropriateness was higher on indication for use than on combination therapy criterion.
Table 3 reports the effect of each DUR intervention on the appropriateness of prescriptions for the indication for use. Time-series results are also presented for the control hospital. Since all omega values but one are positive in both DUR experimental hospitals, it describes a general tendency for improvement whereas this tendency is toward reduced appropriateness in the control hospital in which omega values are all negative. In the concurrent DUR hospital, the improvement induced by the distribution of criteria and the start of pharmacist interventions is statistically significant for prescriptions at day 4, but not at day 1. Figures 1 and 2 depict these time series and the timing of the interventions. It shows that, during the direct interventions period conducted in the concurrent DUR hospital, the appropriateness of prescriptions for the indication for use has decreased in the control hospital. The graphical representations are concordant with the respective negative omega values reported in Table 3. In contrast, during this same period in the concurrent DUR hospital, the appropriateness of prescriptions has remained stable at day 1 and increased at day 4. Again, the graphical representations are concordant with the omega values reported in Table 3. At day 1, the omega value was positive although not statistically significant (P value of 0.11) whereas it was both positive and statistically significant when prescriptions were assessed after the initiation of treatment at day 4 (P value of 0.03). When these changes are compared with those in the control hospital (see Table 3), there is a statistically significant improvement at both day 1 (T = 2.52; P = 0.01) and day 4 (T = 2.62; P = 0.01) in the concurrent DUR hospital only. This improvement is associated with the distribution of criteria to physicians and pharmacists as part of the DUR program.
The effect of the DUR interventions on the appropriateness of combination therapy is presented in Table 4 and depicted in Figures 3 and 4. In all settings the appropriateness has decreased over the study period as illustrated by the fact that 15 of the 16 omega values are negative. Moreover, this decrease seems to be more pronounced in the concurrent DUR hospital in which it is statistically significant after the diffusion of collated results at both day 1 (ω0 = -16.72; P = 0.01) and 4 (ω0 = -14.29; P = 0.04). There is however no statistically significant difference between the control hospital and the two others.
Three important findings emerge from this evaluation. First, conducting a retrospective DUR program had no significant effect on prescription quality. Second, the concurrent DUR program is associated with a significant improvement of the appropriateness of prescriptions for the indication for use. Third, in all hospitals the appropriateness for combination therapy decreased over the study period.
Dissemination of the criteria and of collated results to clinicians were the only interventions in the retrospective DUR hospital. Therefore it may explain the lack of effectiveness of this approach. Although dissemination of criteria may predispose physicians to modify their prescribing behavior, it may not be sufficient. Previous studies have shown that guidelines dissemination alone has no significant effect on prescribing behavior  while studies assessing the impact of DUR intervention letters or reports on prescribing for patients not specifically identified in the letter or report cannot be found in the literature . In theory, such report could reinforce the prescribing behavior change though our results suggest it is not sufficient.
In addition to the dissemination of criteria, the concurrent DUR interventions included direct communication between the pharmacist and the physician during the course of treatment, and the distribution of collated results. As indicated above, disseminating criteria may predispose physicians to behavior change while the other two activities may enable and reinforce the change, respectively. Our results suggest that as a package these three interventions may be sufficient to induce prescribing behavior change in physicians. This effect was however limited to the indication for use criterion.
The appropriateness of prescriptions with regards to combination therapy decreased in all hospitals over the study period. As the decrease in appropriateness was also observed in the control hospital, it suggests a strong secular trend that would have been interpreted wrongly as an adverse effect of the interventions had we conducted this study without an adequate control group. Secular trends in prescribing behavior have been observed in the past .
This overall decrease in appropriateness for the combination therapy criterion was not expected. As reports of adverse events have been published during the study period, it may have influenced physicians and pharmacists to focus more on the indication for use than on combination therapy. In July 1996, in a newsletter sent to physicians and pharmacists Health Canada informed that cisaprise was contraindicated in patients taking drugs that inhibited the cytochrome P450-3A4 enzymes that metabolize cisapride or in those taking drugs that could prolong the QT interval since those patients were at increased risk of developing severe ventricular arythmia or torsades de pointe. The list of all drugs – including cisapride- that could prolong the QT interval or torsades de pointe appeared in another newsletter sent to health professionals in January 1998. These warning newsletters may have had an effect on the prescribing behavior of physicians in our study although previous research has shown such letters alone had no effect . In the USA, change in labeling, black-box warning and press release by the FDA, together with a "Dear Health Care Professional" letter sent by the manufacturer to physicians and pharmacists showed no effect in reducing prescribing of contraindicated drugs to patients on cisapride . Although similar actions are not likely to have impact differently on Canadian physicians and pharmacists, the feedback messages on the combination therapy criterion in our study, if any, may thus have been perceived as clinically unimportant. Indeed, the concomitant prescription of cisapride with a proton pomp inhibitor although inappropriate is not dangerous. On the other hand, physicians seem to have considered very seriously the inappropriateness of prescribing concomitantly cisapride and contraindicated drugs, as illustrated by the very high proportion of prescriptions being appropriate on this criterion.
Our results also show that the appropriateness of prescriptions has improved in all settings and for both criteria between day 1 and day 4. It suggests that either physicians are reviewing their own prescriptions and/or pharmacists, as part of their usual practice, are intervening even in the absence of a formal DUR program . This strong improvement effect illustrates the importance of a control group when one wants to assess the effect interventions have during the course of a treatment .
Our study has strengths and limitations. The greatest strength of this study is the use of a control-group time-series analysis design based on records of drug prescriptions for a period of over four years. We were therefore able to account for prior trends and for regression toward the mean i.e., the tendency for deviant prescriptions to approach the mean on subsequent observation . Moreover, this study was conducted in the context of current clinical practice with no attempt from the investigators to impose the selection of the drug or to enhance compliance of pharmacist in delivering DUR interventions. As a limitation however, there was a lack of randomization. In addition, because no information on prescribing physicians was collected it was not possible to describe physician characteristics in each hospital and test their comparability. Another limitation lies in the fact that the unit of analysis is a hospital level aggregation of prescriptions. Although this insures the reliability of our observations of the proportion of appropriate prescriptions, their number was still relatively modest thus limiting the power to detect an intervention effect. Finally, the concurrent DUR interventions lasted only three months. The effect of this type of DUR could therefore have been greater had we assessed a program implemented on a longer term. It is however current practice in Quebec hospitals to conduct concurrent DUR programs of short duration, in general six weeks .
Care should be taken before generalizing the results of this study to all DUR programs as the effect of those programs depends heavily on the targeted drug, the interventions offered as part of the programs, and on the criterion against which appropriateness is assessed. There is also a need for further research to assess whether the effect of DUR interventions on the drug prescribing process impact on patient outcomes and on the efficiency of health care services.
The decrease in the appropriateness of prescriptions for combination therapy highlights the need to design or integrate more effective interventions. In this regard, as part of a concurrent DUR program, computer-based interventions look promising. Systems designed to warn pharmacists about the inappropriate prescription of a drug have been shown to contribute at improving ambulatory prescribing patterns for the elderly patients . Computer physician order entry displaying drug utilization recommendations could also be integrated in DUR programs . There is also increasing evidence that providing physicians with prescriber-specific and patient-specific profiles positively alter the prescribing behavior of physicians in the ambulatory setting [22–24]. To achieve greater impact, all the preceding interventions could be offered in concert with other well established interventions such as academic detailing . Further research is needed to identify the mix of interventions that would be the most cost-effective.
In conclusion, we have conducted this study in the routine clinical practice setting with no intervention from us in the clinical process. Although cisapride has since been withdrawn from the Canadian market, this study is relevant as it shows that a retrospective DUR the way it is generally conducted in Canadian hospitals may not be sufficient to improve the quality of prescribing. On the other hand, a concurrent DUR with direct feedback to prescribers seems effective to improve the appropriateness with regard to the indication for use. Nevertheless, it may have negative effects on other component of the quality of the prescriptions. Since the effect of DURs varies with both the type of interventions conducted and the criterion applied, there is a need for further research in other settings and with other drugs.
Stolar MH: Drug use review: operational definitions. American Journal of Hospital Pharmacy. 1978, 35: 76-78.
Lipton HL, Bird JA: Drug utilization review in ambulatory settings: state of the science and directions for outcomes research. Medical Care. 1993, 31: 1069-1082.
The ASHP Council on Professional Affairs: ASHP guidelines on the pharmacist's role in drug use evaluation. American Journal of Hospital Pharmacy. 1988, 45: 385-386.
Soumerai SB, Avorn J: Efficacy and cost-containment in hospital pharmacotherapy: state of the art and future directions. Milbank Mem Fund Q Health Soc. 1984, 62: 447-474.
Soumerai SB, McLaughlin TJ, Avorn J: Improving drug prescribing in primary care: a critical analysis of the experimental literature. The Milbank Quarterly. 1989, 67: 268-317.
Hennessy S, Strom BL, Lipton HL, Soumerai SB: Drug utilization review. Pharmacoepidemiology. Edited by: Strom BL. 2000, New York, John Wiley & Sons, Ltd., 505-523. Third
Cook TD: Quasi-experiments: Interrupted time-series designs. Quasi-exprimentation Design and analysis issues for field settings. Edited by: Cook TD and Campbell DT. 1979, Boston, Houghton Mifflir Co, 207-232.
Breton MC, Bourbeau K, Cusson J, Gregoire JP, Guevremont C, Labranche S, Milot A, Robert S, St-Laurent M: The Drug utilization review Network of Quebec. Healthcare Management FORUM. 1996, 9: 44-47.
Box GEP, Jenkins GM: Timeseries analysis: Forecasting and control. 1970, San Francisco, Holden-Day
Box GEP, Tiao GC: Intervention analysis with applications to economic and environmental problems. J Am Stat Assoc. 1975, 70: 70-79.
McCain LJ, McCleary R: The statistical analysis of the simple interrupted time-series quasi-experiment. Quasi-experimentation Design and analysis issues for field settings. Edited by: Cook TD and Campbell DT. 1979, Boston, Houghton Mifflir Co, 233-293.
Tukey JW: Exploratory Data Analysis. 1977, Reading, MA., Addison-Wesley
SAS Institute Inc.: SAS OnlineDoc, Version 8.
Soumerai SB, Majumdar S, Lipton HL: Evaluating and improving physician prescribing. Pharmacoepidemiology. Edited by: Strom BL. 2000, New York, John Wiley & Sons, Ltd., 483-503. Third
Soumerai SB, McLaughlin TJ, Gurwitz JH, Guadagnoli E, Hauptman PJ, Borbas C, Morris N, McLaughlin B, Gao X, Willison DJ, Asinger R, Gobel F: Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial. Journal of the American Medical Association. 1998, 279: 1358-1363.
Ross-Degnan D, Soumerai SB, Fortess EE, Gurwitz JH: Examining product risk in context. Market withdrawal of zomepirac as a case study. Journal of the American Medical Association. 1993, 270: 1937-1942.
Smalley W, Shatin D, Wysowski DK, Gurwitz J, Andrade SE, Goodman M, Chan KA, Platt R, Schech SD, Ray WA: Contraindicated use of cisapride: impact of food and drug administration regulatory action. Journal of the American Medical Association. 2000, 284: 3036-3039.
Majumdar SR, Soumerai SB: Why most interventions to improve physician prescribing do not seem to work. Canadian Medical Association Journal. 2003, 169: 30-31.
Comité scientifique: Rapport d'utilisation des procinétiques dans les désordres de la motilité gastro-intestinale. 1999, Montréal, Réseau de revue d'utilisation des médicaments, 1-5.
Monane M, Matthias DM, Nagle BA, Kelly MA: Improving prescribing patterns for the elderly through an online drug utilization review intervention: a system linking the physician, pharmacist, and computer. Journal of the American Medical Association. 1998, 280: 1249-1252.
Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW: Effects of computerized physician order entry on prescribing practices. Arch Intern Med. 2000, 160: 2741-2747.
Okano GJ, Rascati KL: Effects of Medicaid drug utilization review intervention letters. Clin Ther. 1995, 17: 525-33; discussion 516.
Collins TM, Mott DA, Bigelow WE, Zimmerman DR: A controlled letter intervention to change prescribing behavior: results of a dual-targeted approach. Health Serv Res. 1997, 32: 471-489.
Smith DH, Christensen DB, Stergachis A, Holmes G: A randomized controlled trial of a drug use review intervention for sedative hypnotic medications. Medical Care. 1998, 36: 1013-1021.
Soumerai SB, Avorn J: Principles of educational outreach (academic detailing) to improve clinical decision making. Journal of the American Medical Association. 1990, 263: 549-556.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/6/33/prepub
This study was funded by a grant from the Canadian Institutes for Health Research (HSR-48625).
The authors wish to thank Michel Gaudet, M.Sc., for statistical support, Nancy Aylwin for abstracting hospital records, and Joanne Vidal for editorial assistance.
The author(s) declare that they have no competing interests.
Jean-Pierre Grégoire developed the design, and led the statistical analyses and the writing of the manuscript. Jocelyne Moisan, Louise Potvin, Isabelle Chabot, René Verreault and Alain Milot all contributed to the design, the statistical analysis and the writing of the paper.