- Research article
- Open Access
- Open Peer Review
Application of discrete event simulation in health care: a systematic review
BMC Health Services Researchvolume 18, Article number: 687 (2018)
The objective was to explore the current advances and extent of DES (Discrete Event Simulation) applied to assisting with health decision making, as well as to categorize the wide spectrum of health-related topics where DES was applied.
A systematic review was conducted of the literature published over the last two decades. Original research articles were included and reviewed if they concentrated on the topic of DES technique applied to health care management with model frameworks explicitly demonstrated. No restriction regarding the settings of DES application was applied.
A total of 211 papers met the predefined inclusion criteria. The number of publications included increased significantly especially after 2010.101 papers (48%) stated explicitly disease areas targeted, the most frequently modeled of which are related to circulatory system, nervous system and Neoplasm. The DES applications were distributed unevenly into 4 major classes: health and care systems operation (HCSO) (65%), disease progression modeling (DPM) (28%), screening modeling (SM) (5%) and health behavior modeling (HBM) (2%). More than 68% of HCSO by DES were focused on specific problems in individual units. However, more attempts at modeling highly integrated health service systems as well as some new trends were identified.
DES technique has been an effective tool to approach a wide variety of health care issues. Among all DES applications in health care, health system operations research occupied the most considerable proportion and increased most significantly. Health Economic Evaluation (HEE) was the second most common topic for DES in health care, but with stable rather than increasing numbers of publications.
DES is a form of computer-based modeling methodology that provides an intuitive and flexible method , characterized by the ability to simulate dynamic behaviors of complex systems and interactions between individuals, populations and their environments . The goal of such a model is to comprehensively compare potential practices or strategy options, so as to identify the most efficient and effective ones, especially in situations where it may be infeasible to carry out required surveys or comparative experiments in practice.
In comparison with aggregate models without interaction  like decision trees or Markov models, DES as an operational research technique can be more advantageous to model complex systems at the individual level instead of cohort level. The contexts simulated are usually viewed as queuing networks where individual entities pass through a series of discrete events one by one at discrete intervals, between which they have to wait in queues due to the constrained availability of resources . DES is also able to allow decision makers to conduct “what if” analyses by changing the operational scenarios and rules, to predict the possible impacts resulting from a variety of policy alternatives before truly translated into practice without any alteration in present systems. However, admittedly DES models are also simplified presentations of reality, just as the portraits of other modeling techniques indicate. Typically, patients or medical care providers are referred to as entities, an essential concept of DES, which go through the whole process simulated. Attributes of target entities, which can affect or even determine entities’ responses to events, include age, sex, health status, illness history, duration of disease and other demographics, the values of which are updatable as models are running [1, 5]. Events can be viewed as a wide range of things which may happen during simulation, like the occurrence or recurrence of a disease, admission to health facility, delivery of some medical treatment or transitions between health states. More detailed characterizations of DES are available from the studies by Caro and Karnon et al. [5, 6].
There are two relatively extensive reviews on DES applications in the health care sector [7, 8], which were restricted to specific sub-systems or contexts of health care systems. The earlier one was published in 1999, surveying and classifying the DES literature which focused on single health care units . Around ten years later, the second one sought to analyze DES applications in hospital care . To the best of our knowledge, the overall picture of how DES has been applied to health care is still unclear.
The aim of this review is to provide an up-to-date overview of a considerable body of academic journal literature relevant to the application of DES in health care, thereby further gaining deep insights into the extent of DES used to assist with health decision making and categorizing the wide array of health-related topics where DES was applied.
For this systematic review, two main databases, PubMed and Web of Science, were systematically searched as the main sources for publications of interest. The last search was performed on March 31, 2017, based on broad search terms: ‘discrete event simulation’ or ‘(discrete event) AND (computer simulat* OR model*)’, accompanied by terms like ‘health service*’, ‘Patient*’, ‘healthcare’ or ‘health care’. Papers which brought up discrete event simulation and health-related topics in the title, abstract or keywords and were published over the last 2 decades were searched for without any restrictions on setting and country of origin applied. As a supplement, the reference lists of relevant reviews previously published and the latest studies included were scanned and hand-searching of pertinent open-access journals was also conducted in order to retrieve additional potential peer-reviewed publications.
To identify studies pertaining to our intentions, the inclusion of papers was based on the following criteria:
Studies had to be carried out in health care delivery or public health scenarios, which can be related to any specific context of health service systems.
Discrete event simulation frameworks should be pointed out as the main modeling technique and independently formulated and structured.
How exactly work flows were structured and simulated should be clarified and clearly demonstrated in the form of either flow charts or descriptive texts, according to the recommendations of the ISPOR-SMDM modeling good research practice .
Only peer-reviewed original research was included for further review. Papers published as literature reviews, editorials, conference proceedings or methodological guidelines were excluded.
Only English articles were taken into account.
Selection of publications
After removing duplicates, all studies were screened and reviewed following two main phases. The initial stage of this review was conducted independently by XG and FG following the predefined criteria mentioned above, at title and abstract level, with the aim to identify studies which are appropriate for further full-text review. For inclusion criteria 1, 2 and 4, titles and abstracts screened in the Microsoft Excel table provided adequate information for reviewers to make decisions, but for criterion 3, full texts of records were to be retrieved, when necessary. In case of discrepancies in assessment results between the two reviewers, a third reviewer (WR)’s opinion was taken into consideration.
In view of the emphasis of this review laid on the overview of application advances of DES in health care, therefore only basic information hints of each paper retrieved were deemed necessary to be extracted and screened on a standardized Excel form. All papers were stored and managed via EndNote X7. The following items were extracted and summarized from the literature: Title, author, publication date, journal, medical conditions, perspective for cost measurement, specific settings analyzed, setting classification and application category, on the latter two of which the two precedent similar reviews were focused respectively [7, 8]. In this sense, this work could be perceived as an update for our precedents. Nonetheless, more detailed insights can be gained herein, for example, this review presented the physical contexts simulated more specifically instead of only roughly classifying them as inpatient facilities, outpatient clinics and emergency department. Additionally, a more comprehensive taxonomy of application areas was introduced, which was built on reviews by Mielczarek et al.  and Fone et al. . To indicate the breadth and scope of DES applications in health care, medical conditions surveyed were also indicated and categorized based on ICD-10 codes (International Statistical Classification of Diseases and Related Health Problems 10th Revision) endorsed and recommended by WHO.
A total of 479 citations were yielded in the first step. After removal of 123 duplicates, 356 papers were screened and reviewed at title and abstract level. This first round of reviewing resulted in the exclusion of 136 articles which failed to meet the inclusion criteria: 8% of articles excluded were not related to health care domain, 38% were not original studies, 34% did not perform DES independently, 19% did not or not clearly present DES structures and entity flows and only one paper was not in English.
The remaining 220 articles were further reviewed on the full text level. During this phase, the final 211 articles sufficiently met the eligibility criteria and 9 were excluded mainly due to the fact that they were methodological analyses instead of original research, 2 of which actually focused on other modeling techniques, not obviously related to DES. The review process and results are illustrated as a PRISMA diagram in Fig. 1.
Growing trend of DES applications in health care
The last two decades witnessed a noticeable scale of expansion of publications in this arena as illustrated in Fig. 2, especially after 2010. The number of publications increased from less than 10 papers annually published before 2010 (except for 2008) to almost 35 in 2016, with 73% of the included papers published after 2010. This indirectly reflects a constantly rising popularity of DES modeling in health care management.
Categorization of applications
Based on the nature and inherent properties of the DES models selected, they can be arguably classified into 4 major categories, which were inspired by previous taxonomy [9, 10]: health and care systems operation (HCSO), disease progression modeling (DPM), screening modeling and health behavior modeling (HBM) (Fig. 3), which are explained further in the subsequent sections. Figure 3 illustrates that health and care systems operation accounts for the most considerable proportion (65%) of all modeling studies throughout the whole time span, followed by disease progression modeling (28%), relative to the limited proportion of screening (5%) and health behavior modeling (2%). In addition, the number of health and care systems operation modeling increased by the largest margin compared to other topic areas and disease progression modeling remained at around 5 publications level per year. What is also worth noting is that modeling screening programs and health behavior changes by DES seems to have drawn the attention of health care modelers in recent years.
Research which specified a naturally developing history of a specific disease and an according course of treatment was classified in terms of medical conditions in this section. Out of the 211 papers included, there are a total of 101 papers (48%) which stated explicitly that specific disease areas and corresponding treatment flows were targeted and modeled. The most frequently analyzed medical indications are related to circulatory system, nervous system, Neoplasm and musculoskeletal system diseases, occupying 18%, 15%, 15% and 13% respectively. The detailed distribution of diseases focused on is clearly displayed in Fig. 4.
ICD-10 is divided into 22 chapters from chapter I to chapter XXII, each of which can be further divided into a number of sub-categories, and chapters I-XVII are associated to specific morbid conditions and different body systems. This distribution of diseases presented in Fig. 4 covers more than half (14) of all 22 chapters of ICD-10. Furthermore, it is worth noting that 13 (76%) of all the 17 chapters I-XVII of ICD-10 and nearly all categories related to body systems, except for chapter VIII (Diseases of the ear and mastoid process), are included in Fig. 4 to varying degrees.
Disease progression modeling
In regard to disease progression modeling, DES is commonly employed to transparently conceptualize and construct the course of diseases, health states transition and disease-related events patients will go through under different interventions [11, 12]. This application field is mainly intended to assist in making well-informed medical decisions through comparing different treatment alternatives at medical level in terms of resources consumed, health outcomes or both.
Forty six (77%) of all 60 disease progression modeling compared various medical interventions from a cost-effectiveness perspective. Ten (16%) were undertaken in order to find the most beneficial treatment options, based only on either economic cost or health impact dimension. Most of the remaining 7% papers were predictive models for future health service demands. In total, the most (56) of disease progression models are related to health economic evaluations.
Of the 51 papers which collected and analyzed cost data, 21 (41%) employed the health care payer perspective. 15 (29%) and 14 (28%) were conducted from the perspective of the health care system and society, respectively, with only one analysis taking the point of view of a health care provider.
Health and care systems operation
Health and care systems operation is meant to enable health care managers to better understand the underlying mechanisms of how a system operates, comprehensively investigate complex relationships among different sections within a system with the aid of DES, so as to make optimal operational and administrative decisions [9, 13]. As has been presented in section “Categorization of applications”, DES is most commonly utilized for operations research purposes in health care, with the most significant upward trend in contrast with other application areas.
Distribution of settings for HCSO
It has been repeatedly pointed out that the applications of DES were highly confined to specific single health care units with rare attempts to model broader scenarios [7, 8]. The results of this review are also in support of this conclusion. As is presented in Table 1, more than 68% (93) of HCSO concentrated on single micro-systems. Furthermore, consistent with previous reviews and surveys [8, 14, 15], the major attention of health care modelers was mainly focused on modeling workflows of emergency departments (32) and intensive care units (10) relative to other health care contexts modeled.
Nevertheless, the potential of DES for representing broader continuums of health care delivery system was increasingly taken advantage of. Multi-facility health service providers and even health care systems as a whole were more frequently modeled by DES than reported before, both taking up around 26% (36) of all HCSO papers. The remainder 6% examined coordination effectiveness among departments or micro-systems within a certain health service provider.
Sub-categories of HCSO
The DES approach has been used to address a wide range of operational challenges encountered by health care stakeholders, as is depicted in Fig. 5. This taxonomy was derived from the proposal of Lagergren , which was extended in this review by adding the category of “health economic evaluation”. This classification method distinguishes six major operations research issues consisting of patient scheduling (including appointment and discharge scheduling for both outpatient and inpatient care), resource allocation, capacity planning and management, staff scheduling, system diagnosis and evaluating the effects of operational changes or reconfigurations, the last one of which occupies the most substantial proportion (25%) of all HCSO papers and all others stand at a very similar level of around 12%. 19 (14%) health economic evaluations were identified in HCSO category.
A total of 10 screening modeling papers by DES were identified in this review, most of which were published after 2010.
Among all DES screening models, cancer screening modeling has been most frequently embarked on. It is noteworthy that breast cancer screening dominates this type of modeling, accounting for 50% of cancer models. DES was applied to investigate the costs and health outcomes of different mammographic follow-up schedules [16, 17], alternative breast cancer screening programs [18, 19] as well as the routine performance of a mammography facility under different operational conditions . Health benefits were commonly measured as gains in QALY (quality-adjusted life years) and costs were mainly measured from the perspective of the health care payer and system.
Other screening studies reviewed incorporate two models which analyzed the effects of distinct colorectal cancer screening strategies on patient outcomes and system performance [21, 22], with the remaining evaluating different screening practice options for tuberculosis , Niemann–Pick type C disease  and diabetic retinopathy .
Health behavior modeling
In marked contrast to other industries modeled by DES, the health care domain is deemed more complex to simulate. One important reason, among others, is that entities such as patients or health service providers modeled are individuals characterized by complicated psychological and behavioral attributes. This attribute makes health care more difficult to be represented by computer simulation than to purely model production machines or materials. Incorporating human behavioral factors into modeling practice is a newly emerging concern for health care modelers. Most studies of this type were published after 2011.
One attempt for health behavior modeling was to view a certain individual behavior as the object to be modeled, in others words, to model the trajectories of individual behavioral changes over time under different interventions in order to determine the most influential and efficient mechanisms to encourage desirable health behaviors. This category was highly dominated by smoking-cessation behavior. Smoking behaviors, quit attempts, relapses and sets of events corresponding to smoking-cessation behaviors were simulated by DES structures with the intention to identify the most cost-effective smoking-cessation strategies for diverse populations [26,27,28].
There were other attempts to take human behavioral factors into account while modeling health care delivery, so as to better reflect what really happens in reality. The latest example is the model developed by Brailsford et al. , which represented screening policies for breast cancer via DES with the addition of behavioral factors to each simulated patient’s characteristics. The earliest case was published in 2003 and presented a screening model for diabetic retinopathy focusing on attendance behavior through embedding individual emotion, cognition and social status elements .
This systematic review was undertaken to summarize the advance status and new trends of the application of DES in health care over the last two decades and provide a more quantitative overview of a larger number of DES models compared to precedent reviews.
Rarely applied at policy strategic level
The extent to which DES modeling has been employed to deal with issues at the strategic level is relatively disappointing, which is possibly due in part to the intrinsic attribute of this modeling technique. According to Mielczarek et al. , the DES framework lends itself better to simulate operational details and physical mechanism of a specific process instead of health policy evaluation and planning. Thus, DES is a suitable tool when it comes to concrete operational issues of health care delivery. Regarding public health policy evaluation, in contrast, System Dynamics (SD) is viewed as a more preferred modeling technique.
However, there are still some efforts of applying DES on this highly aggregate level. One of these is the study conducted by Deo et al. , which evaluated the long-run merits and drawbacks of integrating outpatient and HIV services in sub-Saharan Africa. Nguefack et al. built a DES model to estimate the number of new pediatric HIV infections through Mother-to-Child Transmission .
More efforts towards modeling integrated multi-facility providers were identified
The majority of modeling studies included are unit specific, focusing on individual micro-systems such as emergency department and intensive care unit, consistent with previous reviews [7, 8]. Nonetheless, there are an increasing number of studies modeling complex integrated healthcare providers, which were published particularly after 2010. Certainly, these independent multi-facility providers modeled as a whole tended to be simulated at an aggregate and abstract level, because it appeared infeasible to model in great detail—only a sequence of essential activities occurring during patients’ stay from admission to discharge were represented by DES frameworks.
Health care management could benefit a lot from DES used to model integrated providers as a whole rather than just limiting to single units. Given the intricate interdependencies of modern health care services, a more realistic representation of the workings of a real system will be achievable from this integrative perspective. Instead of simplifying the multi-faceted treatment pathways of acute bacterial skin and skin structure infections (ABSSSIs), Revankar et al.  developed a DES framework which captured the complete variations in locations of care from emergency departments, through to outpatient and inpatient units, which properly reflected the real-world complexities related to such treatment administrations.
High-level decision supports can be provided via DES models for hospital managers in terms of the diagnosis of system inefficiencies and evaluation of alternative system configurations [34,35,36,37]. Patient flow through a multidisciplinary sleep center was constructed by Pendharkar et al. with the support of DES . Multiple system constraints resulting in treatment access delays were recognized and alternative reconfigurations aimed at improving access to sleep care were tested.
Patient satisfaction lies in their comprehensive experience throughout a whole integrated system. Consequently, to model an integrated provider as a whole is considered necessary for improving patient overall experience . Bard et al.  applied DES to mimic patient flow through a series of activities as a network in a Family Health Center and gained deep insights into the influential factors jeopardizing patient satisfaction such as excessively long hospitalization and high provider utilization.
Certainly there are a few pitfalls for DES modeling at this higher lever in health care. The most frequently mentioned would be the generalizability issue. These model frameworks for integrated care providers were usually populated by data collected from specific institutions during a certain historical period, which in this sense leads to limitations on the applicability of modeling results into other settings or other periods of time [34, 36]. In addition, lack of necessary data makes it inevitable to make assumptions about the systems modeled or rely on expert estimates as well as data from literature instead of real observations. These alternatives naturally can help simplify the DES modeling practice, but, how and to what extent the validity and accuracy of DES models will be affected should not be disregarded [33, 40, 41].
Modeling of the continuum of entire health services for a certain disease area emerged
Health service pathways are a continuous process throughout the whole health system involving a wide range of care providers, from primary health care clinics (downstream), to highly specialized tertiary units (upstream), caring for different phases of disease progression. Unexpectedly, a few DES models of this continuum of care have been reported since 2005, imitating the integrated networks and interconnectedness among different parts of the whole health care pathways for a certain disease area.
The exploration of DES in this direction started in 2005, which, since then, opened up the possibilities of drawing the attention of health care modelers to a bigger picture of the relationships between diverse components of a health care continuum. These macro modeling endeavors are intended to deal with some critical challenges in which micro-system models can hardly play a role, for example, evaluating the cooperation and transferring efficiency [42, 43] and balancing the distribution of scarce health resources within a health system .
Medical and administrative initiatives may be favorable for one specific section of a health system where they will be conducted, but may conversely cause unintended damages to other parts, or lead bottlenecks to shift within the system [42, 43, 45]. Therefore, applying DES to evaluate the overall long-term performance of policy changes from a scale perspective is considered appealing. With the assistance of DES in modeling the care system for Parkinson’s disease (PD) in the UK, Lebcir et al.  reported the impact of an increased use of PD community services on the demand for hospital-based healthcare and resources.
In addition, complex health care interventions nowadays usually imply integrative programs, implementations of which require the deployment of profound organizational and systematic changes and improvements. Accordingly, analyzing the potential benefits of such sophisticated interventions necessitates the employment of a systematic and holistic horizon [46,47,48,49]. As an example, a recent research presented by Willoughby et al.  mapped a system-wide patient flow with DES through a health service system for speech language pathology and further tested the effectiveness of integrated policy changes which involved increasing group treatments, paraprofessional utilization and so on.
Similarly to the shortcomings stated in section “More efforts towards modeling integrated multi-facility providers were identified”, these system-wide DES simulations also suffer limited generalizability and lack of sufficient data needed. In addition, the construction of macro-models is considered far more time-consuming and data-intensive because of higher complexity and vast numbers of activities involved . Furthermore, synthesizing of data from a range of distinct sources will lead to additional uncertainty into parameters estimated . Another challenge mentioned by Katsaliaki et al.  is the lack of consistent terminology. The same processes or services may be named quite differently across different parts of a health system, which can also give rise to some difficulties in modeling.
Advantages of DES in health economic evaluation have been increasingly admired
According to the results of this review, health economic evaluation is the second most common theme, occupying 36% (75) of all DES applications in health care, following health operations research.
Although Markov models and decisions trees are still the most frequently applied modeling techniques in pharmacoeconomic evaluations, it is not hard to find the arguments for the advantages of DES used in HEE that highlight the valued potential of DES to overcome the inherent limitations of cohort Markov models and decision trees.
One merit of DES is called “individual heterogeneity”, namely, each of the entities modeled is stochastically assigned their own unique properties at the beginning of simulation [26, 52,52,54]. Moreover, these initial characteristics can be updated according to events experienced over time, which will in turn impact the possibilities of the occurrence of following events . Hartz et al.  argued that individual characteristics neglected by most of cohort models might have profound impacts on the long term outcomes predicted. The omission of individuality could be viewed as an obvious deficiency for homogenous cohort models.
Another noteworthy point of DES for HEE is its allowing for “individual interaction”, which is valuable especially for models involving analyzing the effects of treatment delays due to patient competition for constrained resources [56, 57]. Standffield et al.  conducted a cost-utility analysis using DES because of the need to consider the impact of introducing OPSC (Orthopaedic Physiotherapy Screening Clinics and Multidisciplinary Services) on patients queuing time.
Additionally, the advances of the DES-specific software packages make it possible to communicate the frameworks and results of HEE to lay decision makers in an easier and more intuitive way, for example with different levels of animation . The transparent presentation plays an important role in improving the understandability and acceptability of model outputs to health care stakeholders [5, 59, 60].
There are also other reasons for DES being chosen in preference to other alternatives, such as less limitations on the number of events and health states to be represented [61, 62], higher flexibility, less computing intensity , and event-based conceptualization attribute . Taken together, DES models are increasingly considered as a more practical and flexible modeling technique in case of complex and dynamic contexts in health care.
There are several limitations of this review which need to be mentioned. Firstly, only PubMed and Web of Science were searched as main databases. This seems to limit the supply of target publications. However, given the significant number of studies included and no novel application areas of DES in health care was further identified in the later reviewing stage, it is reasonable to believe that the majority of representative literature has been captured.
Secondly, the inclusion criteria could also be seen as a limitation. Only peer-reviewed original DES studies in health care management, which unambiguously presented model structures and mechanisms, were given consideration. This set of criteria excluded a certain proportion of literature, especially papers compiled in proceedings of prestigious conferences. Nevertheless, these criteria were deemed necessary in order to make sure the modeling techniques employed in papers were indeed DES frameworks and also ensure consistency with the suggestions by ISPOR-SMDM . In addition, as discussed before, it was believed that the sufficient representativeness of the papers included was not affected substantially in this review.
Thirdly, this review is limited to a broad overview of applications of DES in health care delivery, but without critically assessing their reporting quality. There is thus a need for more detailed quantitative analyses which were beyond the scope of this study.
This systematic review has presented the versatility and new trends of DES models applied to deal with a broad spectrum of issues facing health care administrators and stakeholders. A survey of 211 studies was conducted and the rapidly growing popularity of DES in health care sector could be reflected by the substantially and constantly increasing number of publications, particularly after 2010. In addition, this extending trend could also been proven by the fact that more than half of ICD-10 chapters have been covered by DES studies. This could probably be viewed as an indication that DES models are becoming more and more embraced in health care management, but this point warrants further evidence of the value and extent of DES implementations in realistic health care decision making, which would be one of the focuses of the following publication.
In this survey, DES models were arguably classified into 4 main categories based on the nature of subjects modeled. This classification may need more discussions because of the intrinsic complexity and overlapping of the applications of DES in health. Undoubtedly, health and care systems operations were the dominant issues approached with the assistance of DES, occupying a massive proportion and expanding at the highest rate. The dominance of this leading application area has also been demonstrated in previous comprehensive reviews on computer simulation in health care, especially, the reviews conducted by Mielczarek et al.  and Fone et al.  which evidently presented quantitative supporting evidence for the extensive use of DES in health system operations.
Although modeling of specific individual units still has a predominant role in all the HCSO areas, more scientific endeavors to simulate more integrated health care settings like hospitals or even the whole health service continuum have been identified than before, which implies the rich potential of DES to provide a broader picture of how health care systems behavior. Both of the precedent DES reviews by Jun et al.  and Günal et al.  also discussed the weaknesses of unit-specific piecemeal modeling as well as the significance of modeling at the macroscopic and integrated level. Additionally, the causes and barriers of the limited number of higher-level DES models were also analyzed and delineated in both.
Health economic evaluations via disease progression modeling have become the second most common theme among all DES applications in health care. This topic area was distinguished as one of major categories in the classification systems proposed by Fone et al.  and Katsaliaki et al. . However, no significantly increasing trend in this sphere has been discovered. The majority of these evaluations could be reasonably categorized as cost-effectiveness analysis, approximately half of which were carried out from the perspective of health care payers.
It is hoped that health policy developers could yield an up-to-date overall picture of the application of DES across various health-related topic areas and get motivated to actively apply DES techniques to tackle the conundrums they are facing. Regarding the quality and value of the research surveyed, a critical appraisal will be demonstrated in the following publication.
Acute Bacterial Skin and Skin Structure Infections
Discrete Event Simulation
Disease Progression Modeling
Health Behavior Modeling
Health and Care Systems Operation
Health Economic Evaluation
International Statistical Classification of Diseases and Related Health Problems 10th Revision
International Society for Pharmacoeconomics and Outcomes Research
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Quality-adjusted Life Years
Society for Medical Decision Making
Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Moller J. Modeling using discrete event simulation: a report of the ISPOR-SMDM modeling good research practices task Force-4. Med Decis Mak. 2012;32(5):701–11.
Pidd M. Computer simulation in management science. 5th ed. New York: John Wiley; 2004.
Brennan A, Chick SE, Davies R. A taxonomy of model structures for economic evaluation of health technologies. Health Econ. 2006;15(12):1295–310.
Brailsford SC, Hilton NA, editors. A comparison of discrete event simulation and system dynamics for modelling healthcare systems. Operational Research Applied to Health Services (ORAHS). Glasgow: Glasgow Caledonian University; 2000.
Caro JJ. Pharmacoeconamic analyses using discrete event simulation. PharmacoEconomics. 2005;23(4):323–32.
Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Moller J, et al. Modeling using discrete event simulation: a report of the ISPOR-SMDM modeling good research practices task Force-4. Value Health. 2012;15(6):821–7.
Jun JB, Jocaboson SH, Swisher JR. Application of discrete-event simulation in health care clinics: a survey. J Oper Res Soc. 1999;25(4):325–35.
Gunal MM, Pidd M. Discrete event simulation for performance modelling in health care: a review of the literature. J Simul. 2010;4(1):42–51.
Mielczarek B, Uzialko-Mydlikowska J. Application of computer simulation modeling in the health care sector: a survey. Simul-T Soc Mod m. 2010;88(2):197–216.
Fone D, Hollinghurst S, Temple M, Round A. Systematic review of the use and value of computer simulation modelling in population health and health care delivery. J Public Health Med. 2003;25(4):325–35.
Caro JJ, Huybrechts KF, Xenakis JG, O'Brien JA, Rajagopalan K, Lee K. Budgetary impact of treating acute bipolar mania in hospitalized patients with quetiapine: an economic analysis of clinical trials. Curr Med Res Opin. 2006;22(11):2233–42.
Mielczarek B. Review of modelling approaches for healthcare simulation. Oper Res Decisions. 2016;26(1):55–72.
Lagergren M. What is the role and contribution of models to management and research in the health services a view from Europe. Eur J Oper Res. 1998;105(2):257–66.
Zhao YC, Peng QJ, Strome T, Weldon E, Zhang M, Chochinov A. Bottleneck detection for improvement of emergency department efficiency. Bus Process Manag J. 2015;21(3):564–85.
Carmen R, Defraeye M, Van Nieuwenhuyse I. A decision support system for capacity planning in emergency departments. Int J Model Simul. 2015;14(2):299–312.
Bessen T, Keefe DM, Karnon J. Does one size fit all? Cost utility analyses of alternative mammographic follow-up schedules, by risk of recurrence. Int J Technol Assess Health Care. 2015;31(5):281–8.
Stout NK, Rosenberg MA, Trentham-Dietz A, Smith MA, Robinson SM, Fryback DG. Retrospective cost-effectiveness analysis of screening mammography. J Natl Cancer Inst. 2006;98(11):774–82.
Lee CI, Cevik M, Alagoz O, Sprague BL, Tosteson AN, Miglioretti DL, et al. Comparative effectiveness of combined digital mammography and tomosynthesis screening for women with dense breasts. Radiology. 2015;274(3):772–80.
Comas M, Arrospide A, Mar J, Sala M, Vilaprinyo E, Hernandez C, et al. Budget impact analysis of switching to digital mammography in a population-based breast cancer screening program: a discrete event simulation model. PLoS One. 2014;9(5):e97459.
Coelli FC, Ferreira RB, Almeida RM, Pereira WC. Computer simulation and discrete-event models in the analysis of a mammography clinic patient flow. Comput Methods Prog Biomed. 2007;87(3):201–7.
Berg B, Denton B, Nelson H, Balasubramanian H, Rahman A, Bailey A, et al. A discrete event simulation model to evaluate operational performance of a colonoscopy suite. Med Decis Mak. 2010;30(3):380–7.
Campbell LA, Blake JT, Kephart G, Grunfeld E, MacIntosh D. Understanding the effects of competition for constrained colonoscopy services with the introduction of population-level colorectal Cancer screening. Med Decis Mak. 2017;37(2):253–63.
Heim JA, Huang H, Zabinsky ZB, Dickerson J, Wellner M, Astion M, et al. Design and implementation of a combined influenza immunization and tuberculosis screening campaign with simulation modelling. J Eval Clin Pract. 2015;21(4):727–34.
van Karnebeek CDM, Mohammadi T, Tsao N, Sinclair G, Sirrs S, Stockler S, et al. Health economic evaluation of plasma oxysterol screening in the diagnosis of Niemann-pick type C disease among intellectually disabled using discrete event simulation. Mol Genet Metab. 2015;114(2):226–32.
Davies R, Roderick P, Canning C, Brailsford S. The evaluation of screening policies for diabetic retinopathy using simulation. Diabet Med. 2002;19(9):762–70.
Igarashi A, Goto R, Suwa K, Yoshikawa R, Ward AJ, Moller J. Cost-effectiveness analysis of smoking cessation interventions in Japan using a discrete-event simulation. Appl Health Econ Health Policy. 2016;14(1):77–87.
Getsios D, Marton JP, Revankar N, Ward AJ, Willke RJ, Rublee D, et al. Smoking cessation treatment and outcomes patterns simulation: a new framework for evaluating the potential health and economic impact of smoking cessation interventions. PharmacoEconomics. 2013;31(9):767–80.
Xenakis JG, Kinter ET, Ishak KJ, Ward AJ, Marton JP, Willke RJ, et al. A discrete-event simulation of smoking-cessation strategies based on varenicline pivotal trial data. PharmacoEconomics. 2011;29(6):497–510.
Brailsford SC, Harper PR, Sykes J. Incorporating human behaviour in simulation models of screening for breast cancer. Eur J Oper Res. 2012;219(3):491–507.
Brailsford S, Schmidt B. Towards incorporating human behaviour in models of health care systems: an approach using discrete event simulation. Eur J Oper Res. 2003;150(1):19–31.
Deo S, Topp SM, Garcia A, Soldner M, Yagci Sokat K, Chipukuma J, et al. Modeling the impact of integrating HIV and outpatient health services on patient waiting times in an urban health clinic in Zambia. PLoS One. 2012;7(4):e35479.
Nguefack HLN, Gwet H, Desmonde S, Oukem-Boyer OOM, Nkenfou C, Tejiokem M, et al. Estimating mother-to-child HIV transmission rates in Cameroon in 2011: a computer simulation approach. BMC Infect Dis. 2016;16(11):1–11.
Revankar N, Ward AJ, Pelligra CG, Kongnakorn T, Fan W, LaPensee KT. Modeling economic implications of alternative treatment strategies for acute bacterial skin and skin structure infections. J Med Econ. 2014;17(10):730–40.
Huynh N, Snyder R, Vidal JM, Sharif O, Cai B, Parsons B, et al. Assessment of the nurse medication administration workflow process. J Healthc Eng. 2016;2016:1–14.
Jahn B, Pfeiffer KP, Theurl E, Tarride JE, Goeree R. Capacity constraints and cost-effectiveness: a discrete event simulation for drug-eluting stents. Med Decis Mak. 2010;30(1):16–28.
Pendharkar SR, Bischak DP, Rogers P, Flemons W, Noseworthy TW. Using patient flow simulation to improve access at a multidisciplinary sleep Centre. J Sleep Res. 2015;24(3):320–7.
Fialho AS, Oliveira MD, Sa AB. Using discrete event simulation to compare the performance of family health unit and primary health care Centre organizational models in Portugal. BMC Health Serv Res. 2011;11:274.
Tako AA, Kotiadis K, Vasilakis C, Miras A, le Roux CW. Improving patient waiting times: a simulation study of an obesity care service. Bmj Quality & Safety. 2014;23(5):373–81.
Bard JF, Shu Z, Morrice DJ, Wang DE, Poursani R, Leykum L. Improving patient flow at a family health clinic. Health Care Manag Sci. 2016;19(2):170–91.
Asamoah DA, Sharda R, Rude HN, Doran D. RFID-based information visibility for hospital operations: exploring its positive effects using discrete event simulation. Health Care Manag Sci. 2018;21(3):305–16.
Crawford EA, Parikh PJ, Kong N, Thakar CV. Analyzing discharge strategies during acute care: a discrete-event simulation study. Med Decis Mak. 2014;34(2):231–41.
Lebcir R, Demir E, Ahmad R, Vasilakis C, Southern D. A discrete event simulation model to evaluate the use of community services in the treatment of patients with Parkinson’s disease in the United Kingdom. BMC Health Serv Res. 2017;17(1):50.
Santos A, Gurling J, Dvorak MF, Noonan VK, Fehlings MG, Burns AS, et al. Modeling the patient journey from injury to community reintegration for persons with acute traumatic spinal cord injury in a Canadian Centre. PLoS One. 2013;8(8):1–10.
Matta A, Li J, Sahin E, Lanzarone E, Fowler J. Editors. Proceedings of the international conference on health care systems engineering. Int Conf health care Syst Eng 2013; Milan, Italy. Bern: Springer; 2014.
Chemweno P, Brackenier L, Thijs V, Pintelon L, Van Horenbeek A, Michiels D. Optimising the complete care pathway for cerebrovascular accident patients. Comput Ind Eng. 2016;93:236–51.
Soto-Gordoa M, Arrospide A, Merino Hernandez M, Mora Amengual J, Fullaondo Zabala A, Larranaga I, et al. Incorporating budget impact analysis in the implementation of complex interventions: a case of an integrated intervention for multimorbid patients within the CareWell study. Value Health. 2017;20(1):100–6.
Monks T, Worthington D, Allen M, Pitt M, Stein K, James MA. A modelling tool for capacity planning in acute and community stroke services. BMC Health Serv Res. 2016;16(1):530.
Willoughby KA, Chan BTB, Marques S. Using simulation to test ideas for improving speech language pathology services. Eur J Oper Res. 2016;252(2):657–64.
Bowers J, Mould G, Marshall C. Location of services and the impact on healthcare quality: insights from a simulation of a musculoskeletal physiotherapy service. J Oper Res Soc. 2015;66(7):1212–21.
Pilgrim H, Tappenden P, Chilcott J, Bending M, Trueman P, Shorthouse A, et al. The costs and benefits of bowel cancer service developments using discrete event simulation. J Oper Res Soc. 2009;60(10):1305–14.
Katsaliaki K, Brailsford S, Browning D, Knight P. Mapping care pathways for the elderly. J Health Organ Manag. 2005;19(1):57–72.
Hartz S, Getsios D, Tao S, Blume S, Maclaine G. Evaluating the cost effectiveness of donepezil in the treatment of Alzheimer's disease in Germany using discrete event simulation. BMC Neurol. 2012;12:2.
Erenay FS, Alagoz O, Banerjee R, Said A, Cima RR. Cost-effectiveness of alternative colonoscopy surveillance strategies to mitigate metachronous colorectal cancer incidence. Cancer. 2016;122(16):2560–70.
Fines P, Garner R, Bancej C, Bernier J, Manuel DG. Development and implementation of microsimulation models of neurological conditions. Health Rep. 2016;27(3):3–9.
Simpson KN, Pei PP, Moller J, Baran RW, Dietz B, Woodward W, et al. Lopinavir/ritonavir versus darunavir plus ritonavir for HIV infection: a cost-effectiveness analysis for the United States. PharmacoEconomics. 2013;31(5):427–44.
Standfield L, Comans T, Raymer M, O'Leary S, Moretto N, Scuffham P. The efficiency of increasing the capacity of physiotherapy screening clinics or traditional medical services to address unmet demand in Orthopaedic outpatients: a practical application of discrete event simulation with dynamic queuing. Appl Health Econ Health Policy. 2016;14(4):479–91.
Cooper K, Brailsford SC, Davies R. Choice of modelling technique for evaluating health care interventions. J Oper Res Soc. 2007;58(2):168–76.
Caro JJ, Möller J, Karnon J, Stahl J, Ishak J. Special Topics. Discrete Event Simulation for Health Technology Assessment. 1st ed. Boca Raton: CRC Press; 2016. p. 273–310.
Southard PB, Chandra C, Kumar S. RFID in healthcare: a six sigma DMAIC and simulation case study. Int J Health Care Qual Assur. 2012;25(4):291–321.
Langley I, Doulla B, Lin HH, Millington K, Squire B. Modelling the impacts of new diagnostic tools for tuberculosis in developing countries to enhance policy decisions. Health Care Manag Sci. 2012;15(3):239–53.
Lee D, Wilson K, Akehurst R, Cowie MR, Zannad F, Krum H, et al. Cost-effectiveness of eplerenone in patients with systolic heart failure and mild symptoms. Heart. 2014;100(21):1681–7.
Higashi H, Barendregt JJ. Cost-effectiveness of total hip and knee replacements for the Australian population with osteoarthritis: discrete-event simulation model. PLoS One. 2011;6(9):e25403.
Vataire AL, Aballea S, Antonanzas F, Roijen LH, Lam RW, McCrone P, et al. Core discrete event simulation model for the evaluation of health care technologies in major depressive disorder. Value Health. 2014;17(2):183–95.
Katsaliaki K, Mustafee N. Applications of simulation within the healthcare context. J Oper Res Soc. 2011;62(8):1431–51.
The author is grateful to Wolf H. Rogowski (WR) for the valuable feedback on the draft as well as the suggestions provided during the reviewing process. The written comments of Fabia Mareike Gansen (FG) as well as the assistance provided in the initial phase of literature selection are highly appreciated.
The author is supported by China Scholarship Council (CSC). The Financial source was not involved in study design, review process, interpretation of results and writing of this manuscript.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the author on reasonable request.
Ethics approval and consent to participate
Consent for publication
The author declares that he/she has no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.