- Research article
- Open Access
- Open Peer Review
The quality, safety and governance of telephone triage and advice services – an overview of evidence from systematic reviews
BMC Health Services Researchvolume 17, Article number: 614 (2017)
Telephone triage and advice services (TTAS) are increasingly being implemented around the world. These services allow people to speak to a nurse or general practitioner over the telephone and receive assessment and healthcare advice. There is an existing body of research on the topic of TTAS, however the diffuseness of the evidence base makes it difficult to identify key lessons that are consistent across the literature. Systematic reviews represent the highest level of evidence synthesis. We aimed to undertake an overview of such reviews to determine the scope, consistency and generalisability of findings in relation to the governance, safety and quality of TTAS.
We searched PubMed, MEDLINE, EMBASE, CINAHL, Web of Science and the Cochrane Library for English language systematic reviews focused on key governance, quality and safety findings related to telephone based triage and advice services, published since 1990. The search was undertaken by three researchers who reached consensus on all included systematic reviews. An appraisal of the methodological quality of the systematic reviews was independently undertaken by two researchers using A Measurement Tool to Assess Systematic Reviews.
Ten systematic reviews from a potential 291 results were selected for inclusion. TTAS was examined either alone, or as part of a primary care service model or intervention designed to improve primary care. Evidence of TTAS performance was reported across nine key indicators – access, appropriateness, compliance, patient satisfaction, cost, safety, health service utilisation, physician workload and clinical outcomes. Patient satisfaction with TTAS was generally high and there is some consistency of evidence of the ability of TTAS to reduce clinical workload. Measures of the safety of TTAS tended to show that there is no major difference between TTAS and traditional care.
Taken as a whole, current evidence does not provide definitive answers to questions about the quality of care provided, access and equity of the service, its costs and outcomes. The available evidence also suggests that there are many interactional factors (e.g., relationship with other health service providers) which can impact on measures of performance, and also affect the external validity of the research findings.
The provision of after-hours primary medical care services has been changing in many countries in response to increasing demand and pressures on existing services [1,2,3]. Telephone triage and advice services (TTAS) are one approach to expanding the provision of out-of-hours medical care. TTAS involves people with a health problem receiving assessment and advice over the telephone . TTAS advice may include a recommendation to visit an Emergency Department (ED), make an appointment with a general practitioner (GP) or other healthcare provider, or the administration of home or self-care [5, 6]. The most common TTAS delivery models are either nurse- or physician-led, very often involving the use of call centre technology, incorporating clinical decision support systems to aid in the evaluation of patients’ health conditions .
One of the major drivers for TTAS has been the need to reduce the burden on GPs and EDs. Some estimates suggest that more than half of out-of-hours calls can be handled by telephone advice alone . With this in mind it would seem reasonable to assume that access to health information and advice could relieve pressures on GP services and EDs particularly through reductions in “inappropriate” or “avoidable” attendances [9,10,11]. For many countries TTAS has become an important part of the organisation and delivery of out-of-hours care . A number of healthcare call centres have been established in the United Kingdom (UK) , Australia, Sweden, Denmark, the Netherlands, Canada, along with the United States of America (USA), which does not have government-sponsored national telephone triage access but does have some localised systems [6, 11, 14].
With the expansion of TTAS over the last decade there has been a corresponding increase in its popularity and utilisation across many jurisdictions [11, 15]. This growth has in turn stimulated interest in the potential to incorporate WebRTC (real time communications) technologies , to provide accessible and easy-to-use video capabilities to expand the type of advice and care provided. These capabilities may have the potential to address the rising demands of over stretched health services , and enable access for people living in remote areas. TTAS involve many medical, technological and social/organisational challenges [17, 18] around how the service is run and monitored , along with issues related to the service’s accessibility, quality and appropriateness [20, 21]. The challenges and opportunities afforded by major telehealth care innovations underscore the need for a comprehensive assessment of available evidence related to their quality, safety and governance.
The existing evidence base on the topic examined aspects of TTAS from different perspectives (e.g., clinical, management and patients) using a wide range of measures that are not consistently applied or assessed. This makes it difficult for policymakers to use existing evidence to inform policy decisions. Systematic reviews represent the highest level of evidence synthesis. We aimed to undertake an overview of such reviews to determine the scope, consistency and generalisability of findings in relation to the governance, safety and quality of TTAS .
Our overview included any systematic review focused on telephone based triage and advice services, available in English; and published since 1990. We decided to commence our search at 1990 as the types of telehealth services under consideration commenced operation around this time .
We excluded reviews which were related only to the use of telephone advice for a specific population or demographic (e.g. ethnic minority), condition (e.g., depression) medical specialty (e.g., asthma); and ongoing or chronic conditions (e.g., diabetes); technical assessments not related to patient or healthcare outcomes; and general health education. Table 1 lists the inclusion and exclusion criteria used in this overview.
To maximise our search criteria we used the PICO (Population, Intervention, Comparison, Outcome) search strategy form. The patient or problem targeted were presenting patients, requiring triage or consultation. The intervention was telephone based triage, advice or consultation by telephone. Most of the studies included in the reviews compared telephone advice with face-to-face consultations, but some also compared telephone advice given by different healthcare professionals. Key indicators included those performance measures related to the safety, quality or governance of TTAS and/or other relevant patient/health-related outcomes.
We sourced our results from the following databases: PubMed, MEDLINE, EMBASE, CINAHL, Web of Science and the Cochrane Library. The search terms used included ‘tele’ terms, with ‘tele’ truncated to elicit all related terms (telehealth, telemedicine, telephone, etc.); terms relating to the type of care: triage, advice, consultation; and other related terms such as after-hours, out-of-hours and primary care. The full search strategy is outlined in Additional file 1.
The title and abstract of the reviews identified through our searches was scrutinised, and potentially relevant papers were downloaded for further consideration. Following the search, the full text of these reviews was examined and those which fit our criteria were added to our results. We also scrutinised the reference lists of all relevant papers to identify reviews for inclusion or exclusion.
To ensure reliability, the search strategy was devised by three researchers (AG, LL and RL), the search was completed by one team member (RL) and then replicated by another (LL), and all three researchers discussed the identified texts to arrive at a consensus on the reviews for inclusion.
Data collection and analysis (selection, data extraction and management)
The final group of texts was read in full and the data extracted. We abstracted descriptive details of the included systematic reviews: the authors, title, year published and country. Data were collected about the studies reviewed in each text: the number and type of reviewed studies, the date range of the studies and the study populations (Table 1). Information was compiled about the indicators or performance measures used, the purpose of the review, the design, search methods, the performance of a grey literature search, and results (see Additional file 2 for a detailed summary table). An appraisal of the methodological quality of the systematic reviews was independently undertaken by two researchers (AG, JL) using A Measurement Tool to Assess Systematic Reviews (AMSTAR) . Discrepancies in quality rating between researchers were reconciled through discussion.
A total of 291 papers were identified and subjected to title/abstract review. Of these, 237 were excluded and 20 papers were downloaded and subjected to full text review, along with a further 26 papers identified through hand searching of reference lists. Ultimately, ten systematic reviews were selected for inclusion. Our search flow diagram is depicted in Fig. 1.
Description of included systematic reviews
The ten selected systematic reviews examined TTAS either alone, or in the context of primary care service models or interventions to improve primary care. The systematic reviews were all published within the period, 2003–2013. Four were published between 2003 and 2006, and six between 2011 and 2013. The reviews came from researchers in the UK, Australia, Canada, the Netherlands and Portugal. Each of these countries had a TTAS service. The UK National Health Service (NHS) established their national TTAS, NHS Direct in 1998, which was replaced with NHS 111 in 2014 . Australia established the healthdirect helpline service in 2006 . Portugal has Health Line 24 , and both the Netherlands and Canada have various services. Table 2 details the characteristics and aims of the included systematic reviews.
Study designs within systematic reviews
Nine of the ten systematic reviews allowed us to identify the studies they included, while one provided an outdated link to their reference list, which proved inaccessible. Our attempts to contact the authors to obtain this information were unsuccessful . From the remaining nine reviews we identified 127 individual papers, 42 (33%) of which were included in more than one review. Thirty-one studies were included in two of the reviews, nine studies featured in three reviews, one study was included in four and another one in five of the reviews.
The studies ranged in date from 1978 to 2011 and originated from the UK (53), the USA (36), Australia (12), Canada (6), The Netherlands and Denmark (both 4), New Zealand and France (both 3), Switzerland (1), Germany (1), Sweden (1), Italy (1), India (1), and Israel (1). There were nine Randomised Controlled Trials (RCTs). The vast majority of studies were observational, including before and after and interrupted time series designs, as well as several surveys. The reviews also included some case studies, evaluations and reports.
To assess the quality of the systematic reviews we: i) conducted a quality appraisal of the systematic reviews against the AMSTAR checklist; ii) evaluated the methodological quality of the studies reported in the review; and iii) considered the comments made in the review regarding the overall quality of the evidence presented.
Out of a maximum score of 11, quality ratings of the systematic reviews against the AMSTAR checklist ranged from 2 to 8. Five reviews achieved a rating greater than or equal to 5 [6, 14, 15, 27, 28]. Three reviews provided no evidence of having assessed the quality of included studies. Of the other seven reviews, a range of methods was used for quality assessment. One used a version of the checklist developed by the Cochrane Effective Practice and Organisation of Care Review Group . Another used a modified checklist for assessing economic evaluations . Two reviews used a hierarchy of evidence method [15, 30] and two reviews used a revised version of the Critical Appraisal Skills Programme [6, 11, 31]. One review used a modified version of the Scottish Intercollegiate Guidelines Network (SIGN) checklist . Eight of the reviews commented on the limitations of the evidence, citing poor study designs, poorly reported studies and low checklist scores. Overall, some of the evidence appears reliable, but much of it is not strong .
Six of the systematic reviews did not include a search of the grey literature [6, 7, 12, 14, 28, 30], while four did [11, 15, 27, 33]. This overview conducted a broad Google search to identify other existing reviews but did not elicit any new results. All searches conducted in this overview were restricted to English language systematic reviews only.
Quality, safety and governance dimensions
We identified the following nine quality, safety and governance dimensions from the available evidence: access, appropriateness, patient compliance, patient satisfaction, cost, safety, health service utilisation, clinical workload and clinical outcomes. Most of the systematic reviews examined between one to five measures, one review investigated eight, but no review considered all nine. We proceeded to group the review results together based on these indicators and then considered the ways in which they investigated and compared the results.
Expanding healthcare access to marginalised communities or populations is an often-cited driver of the introduction or expansion of TTAS services . However, the impact of TTAS on improving access was not apparent from the two systematic reviews that examined this issue. Carrasqueiro et al. considered health care access, but did not state how many studies they found that explored this issue. The authors reported that in some instances healthcare access for some people with severe symptoms was expedited , they nevertheless concluded that there was no consistent improvement in access based on TTAS availability. Chapman et al. reported on one study which found that the NHS Direct service was easily accessible, but callers appeared to be those already utilising other healthcare services .
In their review, Blank et al.  noted the lack of consistency in the definition of “appropriateness” of telephone triage decisions by the studies included in their rapid evidence scan. The five systematic reviews which considered evidence about the appropriateness of TTAS used one of two definitions, examining either the number of calls which could be managed with telephone advice alone, or the appropriateness of the advice given.
Two reviews looked at the percentage of calls able to be handled with telephone advice alone. Both the Cochrane review by Bunn et al.  and the review by Fry  concluded that approximately 50% of calls handled by doctors or nurses could be handled with telephone advice alone. Bunn et al. also compared the call handling abilities of doctors and nurses and concluded that doctors handled 62% of calls with telephone advice alone, while nurses managed 59% with telephone advice alone .
The review by Blank et al. examined studies in which the advice given over the phone was compared with “appropriate” advice – either the “gold standard,” professional advice, subsequent treatment or diagnosis, patient validation or measure of adverse events. This review found a 44–98% rate of accuracy/appropriateness with a median of 75% . The review also compared doctors’ and nurses’ handling of calls, but found that results were not consistent. This review and the review by Fry did however note there may be issues with under-referral and under-estimation of urgency [11, 14].
The review by Leibowitz et al. reported on an RCT which compared the number of deaths in seven days between those whose calls were handled by doctors or nurses. This RCT found no difference between the two groups. This paper also noted that studies using simulated patients reported variability and inadequacies in the advice given over the phone. Alternatively, studies which used real patients reported that the majority received appropriate advice .
The review by Carrasqueiro et al. examined studies in which audits of medical records were conducted to determine the adequacy of the advice given over the phone. The authors concluded that they could not demonstrate high rates of advice appropriateness or adequacy, but did not specify exactly how this conclusion was drawn . The impact of other provider characteristics (e.g., age, length of experience) on the appropriateness of the advice given was not discussed by any systematic review.
When advice is offered to patients via a new medium, such as the telephone, it is imperative to assess patients’ willingness to adhere to the advice. Services that provide advice with which patients are unwilling to comply are not likely to prove worthwhile. The three systematic reviews which investigated compliance all compared compliance rates by the advice given, which they grouped into three categories: emergency or urgent care, office/physician care, and home or self-care. Blank et al. reported that the median rate for compliance with advice to seek physician care was 66%, urgent care 75%, and self-care 77% .
The Purc-Stephenson and Thrasher review also found that callers were more likely to comply with advice to use emergency services or self-care than office care, but did not provide figures . This review also commented that patient perceptions and expectations, along with the quality of the provider communication influenced caller compliance . The third review by Carrasqueiro et al.  failed to provide compliance rates but did report that compliance varied significantly depending on the advice given, and that the variance was due to patient characteristics – original intention, complaint, age, income. This review, as well as the review by Blank et al. , also noted that rates of patient compliance were slightly higher when patient self-reporting data were examined, compared with provider data.
Two of the systematic reviews provided overall compliance rates. Blank et al. estimated an overall compliance rate of between 56 and 98%, with a median of 77%  while Purc-Stephenson and Thrasher reported an overall compliance rate of 62% . Both of these reviews included some self-reported data, which may be potentially misleading.
Five systematic reviews examined patient satisfaction of TTAS with most reporting satisfactory levels. Ismail et al. reported that their analyses found generally positive patient satisfaction rates, although one of the examined studies found low rates . In their Cochrane review, Bunn et al. found patient satisfaction rates to be no different from other forms of care, and sometimes higher . Fry considered patient satisfaction of the (no longer existing) UK NHS Direct service and found several studies reporting good levels of patient satisfaction .
Carrasqueiro et al. likewise found that most studies presented positive levels of patient satisfaction, but they, along with Leibowitz et al. observed that satisfaction rates decreased when patients’ initial expectations were not met [7, 30]. Patients were dissatisfied when telephone-based care represented a barrier to traditional forms of care, for example when patients rang physicians requesting a home-visit but were offered a telephone consultation instead [7, 30].
The findings from the four systematic reviews which examined the costs of TTAS, point to the potential for cost-effectiveness, but also suggest a lack of thorough research. Brebner et al. stated simply that all the studies which examined the telephone services found them to be cost-effective and did not elaborate further . Carrasqueiro et al. found no thorough cost analysis studies . Bunn et al. concluded that there was little cost difference between telephone advice and traditional care, but noted a study in which doctors made telephone advice calls to patients instead of granting same day appointments. In this case, the doctors’ phone bills increased 26% . Finally, Ismail et al. concluded that while some cost reductions were possible, analyses suggested that the likelihood of savings across urgent care were low .
Two systematic reviews considered the issue of TTAS safety, primarily by investigating the frequency of adverse events, errors and hospitalisation rates. Carrasqueiro et al. reviewed studies which used patient surveys and medical records and concluded that the evidence suggests that TTAS was reasonably safe . Huibers et al.  reported on 13 observational studies which showed that on average triage was safe in 97% of patients. However, 10 studies using high-risk simulated patients showed that on average 46% were safe .
Health service utilisation
Four systematic reviews investigated changes in health service utilisation brought about by the availability of TTAS. Carrasqueiro et al. reported that no clear pattern of change was evident . The review by Ismail et al. reported conflicting results which produced no clear pattern, but acknowledged that reductions in downstream workloads were a possibility .
The systematic review by Chapman et al. included one study finding no change in service utilisation, while another study observed a decrease in use of GP services and an increase in out-of-hours consultations and ED attendances . Two reviews reported a reduction in demand for GP consultations, particularly after-hours [11, 27]. One of these found no change in hospital use , while the other reported that ED activity could be reduced by up to two thirds .
Bunn et al.  reported that when telephone consultation was compared with traditional care, two of four studies reported no difference in the number of ED visits, while two studies reported increases in ED visits, one of which was a statistically significant increase, the other was not. Regarding hospital admissions, telephone consultation by doctors did not lead to any change in admission rates, while a phone service run by clinic clerks did lead to a reduction in admissions at 12 months. Bunn et al. also reviewed three studies which compared one type of health care professional with another . Two RCTs compared nurse telephone advice with advice from a doctor in an out-of-hours deputising service [34, 35], and one Controlled Clinical Trial (CCT) compared telephone advice by a health assistant with telephone advice from a nurse or doctor . All three observed a non-statistically significant rise in visits to the ED in the intervention groups. The two RCTs also examined hospital admission rates and found no changes.
Changes in health service utilisation could also produce workload changes for health professionals. GPs may be especially susceptible to such changes. Four reviews considered how TTAS may affect GPs workloads. All four systematic reviews found evidence that telephone advice services had the potential to reduce GPs workloads [11, 15, 28, 30].
Two systematic reviews investigated clinical outcomes and clinical effectiveness. The first concluded that ED tele-consultation was clinically effective . Carrasqueiro et al. reviewed studies which utilised patient survey data to consider clinical outcomes. They determined that there were no long term studies which could lead to a conclusion one way or the other .
This systematic overview collated the available evidence related to the governance, safety and quality of TTAS across nine key performance indicators – access, appropriateness, compliance, patient satisfaction, cost, safety, health service utilisation, physician workload and clinical outcomes. Some important indicators, such as the acceptability of TTAS were not specifically reported in the reviews we examined, but they may be considered in evaluations of performance conducted by the services themselves, or their funding bodies. These evaluations could potentially provide another source of information about these services.
The indicators of appropriateness and patient satisfaction received the most attention in the reviews. Appropriateness was evaluated in different ways ranging from the adequacy of triage advice, to the ability of doctors and nurses to handle enquiries over the telephone [11, 27]. The majority of reviews that investigated patient satisfaction with TTAS reported that it was generally comparable to, or occasionally even greater than satisfaction with traditional care [11, 27, 28].
The systematic reviews which examined safety compared the frequency of adverse events, errors and hospitalisation rates, and reported that triage was safe in most cases [7, 12]. However, findings from simulated high risk patients showed that on average 50% of patients received advice that was considered unsafe, presenting a higher risk of adverse events . Safety can be very difficult to measure, as those who are already considered unwell or at risk, may be more prone to adverse events, through no fault of the service. Care should be taken about mistakenly using hospitalisation as a measure of safety. If TTAS provides advice that the caller should attend the hospital, this should be seen as advantageous for the patient’s safety, rather than an adverse event related to the use of the TTAS. A clearer definition of adverse events or safety issues related to the service would be of benefit to further research in this area.
The most reliable findings came from studies about clinician workload and patient compliance. The reviews which evaluated clinician workload reported that there was some potential for telephone triage and advice to reduce clinician workload, particularly for GPs. The reviews which investigated patient compliance mostly concluded that callers were more likely to comply with advice to seek emergency care or provide self-care, than advice to visit a GP or other health service [6, 7, 14].
The indicators which revealed the most variable and weakest evidence related to the utilisation of TTAS, service access, and costs. Reviews about access and the cost of TTAS showed no consistent evidence of improvement or gain [7, 15, 27, 28, 33]. Neither of the two reviews which considered clinical outcomes provided strong findings.
Taken as a whole, the available evidence does not provide definitive answers to questions about the quality of care provided, access and equity of the service, its costs and outcomes. The available evidence suggests that there are many interactional factors which can impact on measures of performance , and also affect the external validity or generalisability of the research findings . For instance, TTAS is one (albeit very important) organisational model for dealing with out-of-hours care. Other models include individual general practices, primary care centres or even GP cooperatives . These models are not necessarily exclusive to one distinct area or location, quite often they co-exist across municipalities, primary health care networks, regions or nations . Decisions about the operation of TTAS are also influenced by a large number of demographic (e.g., rural and remote), cultural (e.g., language), finance (e.g., health care costs) and governance (e.g., policy priorities) factors . Moreover, many of the quality and safety dimensions of TTAS are shaped by factors related to the integration and accessibility of care (e.g., the organisation of primary care) or technical infrastructure (e.g., patient access to information), specific to a community . This suggests that TTAS safety and quality dimensions are intrinsically linked to properties of a larger and broader system .
We have in this overview bought together the available evidence and provided a list of measures by which to evaluate TTAS. There is not a simple yes or no answer to questions about the overall effectiveness and benefit of TTAS. Much of the evidence is contingent on the different circumstances and context of different systems. There remain many gaps in the evidence base. This overview provides a framework which can inform future research directions about TTAS and provide guidance relating to their implementation and expansion.
The diffusion of new technologies (e.g., telehealth care) has provided additional scope to expand and diversify health care services and provide opportunities for greater consumer engagement . In the face of these developments, it is important that researchers continue to seek answers to the many unresolved questions about TTAS , albeit with a renewed and greater diversification of qualitative and quantitative research along with strategies that engage with patients as a means of enhancing the patient-centredness of TTAS . Such research endeavours can inform decision making about the governance, quality and safety of TTAS into the future.
Accident and Emergency
Critical Appraisal Skills Programme
Controlled Clinical Trial
Effective Practice and Organisation of Care
Population, Intervention, Comparison, Outcome
Randomised Controlled Trial
Scottish Intercollegiate Guidelines Network
Telephone Triage and Advice Services
Web Real Time Communication
Jessopp L, Beck I, Hollins L, Shipman C, Reynolds M, Dale J. Changing the pattern out of hours: a survey of general practice cooperatives. Br Med J. 1997;314(7075):199.
Hallam L. Out of hours primary care. Br Med J. 1997;314(7075):157.
Salisbury C. The demand for out-of-hours care from GPs: a review. Fam Pract. 2000;17(4):340–7.
Lattimer V, George S. Nurse telephone triage in out-of-hours primary care. Prim Care Manag. 1996;6(5):3–6.
Rolland E, Moore KM, Robinson VA, McGuinness D. Using Ontario's Telehealth health telephone helpline as an early-warning system: a study protocol. BMC Health Serv Res. 2006;6(1):10.
Purc-Stephenson RJ, Thrasher C. Patient compliance with telephone triage recommendations: a meta-analytic review. Patient Educ Couns. 2012;87(2):135–42.
Carrasqueiro S, Oliveira M, Encarnação P. Evaluation of telephone triage and advice services: a systematic review on methods, metrics and results. Stud Health Technol Inform. 2011;169:407–11.
Christensen MB, Olesen F. Out of hours service in Denmark: evaluation five years after reform. Br Med J. 1998;316(7143):1502.
Driscoll P, Vincent CA, Wilkinson M. The use of the accident and emergency department. Arch Emerg Med. 1987;4(2):77–82.
Martin A, Martin C, Martin PB, Martin PA, Green G. Eldridge S: 'Inappropriate' attendance at an accident and emergency department by adults registered in local general practices: how is it related to their use of primary care? J Health Serv Res Policy. 2002;7(3):160–5.
Fry MM. A systematic review of the impact of afterhours care models on emergency departments, ambulance and general practice services. Australas Emerg Nurs. 2011;14(4):217–25.
Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: a systematic review. Scand J Prim Health Care. 2011;29(4):198–209.
McLean S, Sheikh A, Cresswell K, Nurmatov U, Mukherjee M, Hemmi A, Pagliari C. The impact of telehealthcare on the quality and safety of care: a systematic overview. PLoS One. 2013;8(8):e71238.
Blank L, Coster J, O’Cathain A, Knowles E, Tosh J, Turner J, Nicholl J. The appropriateness of, and compliance with, telephone triage decisions: a systematic review and narrative synthesis. J Adv Nurs. 2012;68(12):2610–21.
Chapman JL, Zechel A, Carter YH, Abbott S. Systematic review of recent innovations in service provision to improve access to primary care. Br J Gen Pract. 2004;54(502):374–81.
Jang-Jaccard J, Nepal S, Celler B, Yan B. WebRTC-based video conferencing service for telehealth. Computing. 2016;98(1-2):169-93.
Clarke A. Evidence-based evaluation in different professional domains: similarities, differences and challenges. In: Shaw IF, Greene JC, Mark MM, editors. Handbook of evaluation: policies, programs and practices. London: Sage; 2006. p. 559–81.
Ekeland AG, Bowes A, Flottorp S. Methodologies for assessing telemedicine: a systematic review of reviews. Int J Med Inf. 2012;81(1):1–11.
Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inf. 2010;79(11):736–71.
Salk ED, Schriger DL, Hubbell KA, Schwartz BL. Effect of visual cues, vital signs, and protocols on triage: a prospective randomized crossover trial. Ann Emerg Med. 1998;32(6):655–64.
McKinstry B, Hammersley V, Burton C, Pinnock H, Elton R, Dowell J, Sawdon N, Heaney D, Elwyn G, Sheikh A. The quality, safety and content of telephone and face-to-face consultations: a comparative study. Qual Saf Health Care. 2010;19(4):298–303.
National Health Performance Authority. The Framework's conceptual underpinning. http://www.aihw.gov.au/health-performance/performance-and-accountability-framework/4-conceptual-underpinning/. Accessed 28 Aug 2017.
Bruyère Research Institute. AMSTAR – assessing the methodological quality of systematic reviews. https://amstar.ca/. Accessed: 19 June 2017.
National Health Service. NHS choices. http://www.nhs.uk/NHSEngland/AboutNHSservices/Emergencyandurgentcareservices/Pages/NHS-111.aspx. Accessed 23 December 2015.
Ng JY, Fatovich DM, Turner VF, Wurmel JA, Skevington SA, Phillips MR. Appropriateness of healthdirect referrals to the emergency department compared with self-referrals and GP referrals. Med J Aust. 2012;197:498–502.
Ministry of Health (Portugal). Saúde 24. http://www.saude24.pt/PresentationLayer/home_00.aspx. Accessed 23 December 2015.
Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Database Syst Rev. 2004;4:Cd004180.
Ismail SA, Gibbons DC, Gnani S. Reducing inappropriate accident and emergency department attendances: a systematic review of primary care service interventions. Br J Gen Pract. 2013;63(617):e813–20.
Cochrane Effective Practice and Organisation of Care. Effective practice and organisation of care (EPOC) Reources for review authors. http://epoc.cochrane.org/epoc-specific-resources-review-authors. Accessed 19 April 2016.
Leibowitz R, Day S, Dunt D. A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Fam Pract. 2003;20(3):311–7.
Critical Appraisal Skills Programme (CASP). Making sense of evidence; CASP checklists. http://www.casp-uk.net/#!casp-tools-checklists/c18f8. Accessed 19 Apr 2016.
Scottish Intercollegiate Guidelines Network: Critical appraisal: notes and checklists. http://www.sign.ac.uk/checklists-and-notes.html. Accessed 28 Aug 2017.
Brebner JA, Brebner EM, Ruddick-Bracken H. Accident and emergency teleconsultation for primary care--a systematic review of technical feasibility, clinical effectiveness, cost effectiveness and level of local management. J Telemed Telecare. 2006;12(Suppl 1):5–8.
Lattimer V, George S, Thompson F, Thomas E, Smith H, Moore M, Bond H, Mullee M, Turnbull J, Glasper A. Safety and effectiveness of nurse telephone consultation in out of hours primary care: randomised controlled trial. BMJ. 1998;317(7165):1054–9.
Thompson F, George S, Lattimer V, Smith H, Moore M, Bond H, Turnbull J, Mullee M, Thomas E, Glasper A. Overnight calls in primary care: randomised controlled trial of management using nurse telephone consultation. BMJ. 1999;319(7222):1408.
Strasser PH, Levy JC, Lamb GA, Rosekrans J. Controlled clinical trial of pediatric telephone protocols. Pediatrics. 1979;64(5):553.
Black A, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B, Procter R, Majeed A, Sheikh A. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8(1):e1000387.
Huibers L, Giesen P, Wensing M, Grol R. Out-of-hours care in western countries: assessment of different organizational models. BMC Health Serv Res. 2009;9(1):105.
Mossialos E, Wenzl M, Osborn R, Anderson C. International profiles of health care systems, 2014. The Commonwealth Fund: New York; 2015.
Georgiou A, Marks A, Braithwaite J, Westbrook JI. Gaps, disconnections, and discontinuities—the role of information exchange in the delivery of quality long-term care. Gerontologist. 2013;53(5):770–9.
Georgiou A, Westbrook JI, Braithwaite J. An empirically-derived approach for investigating health information technology: the elementally entangled Organisational communication (EEOC) framework. BMC Med Inform Decis Mak. 2012;12(1):68.
Hordern A, Georgiou A, Whetton S, Prgomet M. Consumer eHealth - an overview of the research evidence and the implications for future policy. HIMJ. 2011;40(1):6–14.
McKinstry B, Sheikh A. Unresolved questions in telephone consulting. J R Soc Med. 2006;99(1):2–3.
The National Patient Safety Foundation’s Lucien Leape Institute. Safety is personal; partnering with patients and families for the safest care. http://patientcarelink.org/improving-patient-care/patient-family-engagement/. Accessed 28 Aug 2017.
This review was supported by Healthdirect Australia.
Availability of data and materials
The datasets supporting the conclusions of this article are included within the article and its additional files.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.