- Research article
- Open Access
- Open Peer Review
Healthcare ecosystems research in mental health: a scoping review of methods to describe the context of local care delivery
BMC Health Services Researchvolume 19, Article number: 173 (2019)
Evidence from the context of local health ecosystems is highly relevant for research and policymaking to understand geographical variations in outcomes of health care delivery. In mental health systems, the analysis of context presents particular challenges related to their complexity and to methodological difficulties. Method guidelines and standard recommendations for conducting context analysis of local mental health care are urgently needed. This scoping study reviews current methods of context analysis in mental health systems to establish the parameters of research activity examining availability and capacity of care at the local level, and to identify any gaps in the literature.
A scoping review based on a systematic search of key databases was conducted for the period 2005–2016. A systems dynamics/complexity approach was adopted, using a modified version of Tansella and Thornicroft’s matrix model of mental health care as the conceptual framework for our analysis.
The lack of a specific terminology in the area meant that from 10,911 titles identified at the initial search, only 46 papers met inclusion criteria. Of these, 21 had serious methodological limitations. Fifteen papers did not use any kind of formal framework, and five of those did not describe their method. Units of analysis varied widely and across different levels of the system. Six instruments to describe service availability and capacity were identified, of which three had been psychometrically validated. A limitation was the exclusion of grey literature from the review. However, the imprecise nature of the terminology, and high number of initial results, makes the inclusion of grey literature not feasible.
We identified that, in spite of its relevance, context studies in mental health services is a very limited research area. Few validated instruments are available. Methodological limitations in many papers mean that the particular challenges of mental health systems research such as system complexity, data availability and terminological variability are generally poorly addressed, presenting a barrier to valid system comparison. The modified Thornicroft and Tansella matrix and related ecological production of care model provide the main model for research within the area of health care ecosystems.
The role of context is critical in health services research. Geographical variations in the fate of healthcare interventions have been documented widely. The significance of local context in such variations is recognised, with the more complex the intervention, the greater the relevance of local factors to its outcome . In health care, “context” could be defined as all sources of evidence of the local system: geographic, social and demographic factors, other environmental factors, service availability and scope, capacity, use, costs and the historical development of the health care system. Evidence from the context of local health systems is thus highly relevant for research and policymakers. The analysis of context of care of “healthcare ecosystem research” is an emerging discipline that should play a critical role in implementation sciences  and in the analysis of complex interventions [1, 3]. However, a broader approach than the traditional unidimensional model of evidence is required . “Contextual evidence” has recently been identified as a major source of knowledge in health systems research together with experimental, observational, expert and experiential knowledge . In spite of its relevance, the need for context analysis in health services and delivery research has not been sufficiently recognised [1, 2, 4].
Evidence about local conditions is important at all stages in the policy process from assessing resource availability and setting policy priorities to examining the impact of policy decisions . The World Health Organization (WHO) has urged exploration of the care context in mental health systems . The WHO Mental Health Gap Action Program (mhGAP) has called for a comprehensive and systematic description of mental health services, including what those services are doing . A knowledge of care delivery at the service delivery level is critical to evidence informed policy , and in the implementation of models of care such as integrated care  and the balanced care model . However, this research faces challenges related to the complexity of mental health systems, and to methodological issues. Mental health care systems are particularly complex due to the number of sectors, levels, and types of service through which care is delivered, the variability of the service delivery over time and the high ambiguity, partly due to the lack of a stable terminology [1, 7]. Descriptions of local service delivery which do not take this complexity into account risk providing policymakers with an inaccurate or limited assessment of the local pattern of service availability, affecting their ability to plan appropriately.
A review of methods used to describe the context of local mental health care is urgently needed. This study sought to take a broad view of available methods of context analysis in systems of mental health care delivery at the service delivery level, identifying and mapping their main components and characteristics. This would identify gaps, provide insight into conceptualisation of the context of mental health systems and inform future context analysis in mental health services research. This is consistent with the call by the WHO to specifically reference service location, availability and function .
Rationale for conducting a scoping review
Scoping reviews “examine the extent, range and nature of research activity in a particular field, without necessarily delving into the literature in depth or attempting to assess its quality” . They are used to “identify parameters and gaps in a body of literature” rather than “generat (ing) a conclusion related to the focussed question”, with “inclusion/exclusion … developed post-hoc”, and a broad research question rather than a “focussed research question with narrow parameters” . A scoping review was considered appropriate for this study due to the broad scope of the research area, the diversity of study designs already known to the authors, and the absence of a definitive terminology.
General scoping review process
We have used the five stage model for scoping reviews developed by Arksey and O’Malley , and extended by Levac . The five stages of this approach are: (i) identifying the research question; (ii) identifying relevant studies; (iii) selecting studies; (iv) charting the data; and (v) collating, summarising, and reporting the results. We have also used the guidance for scoping reviews developed by members of the Joanna Briggs Institute .
Identifying the research question
The main research question of this scoping review was:
“What are the main gaps in the available literature relevant for context analysis of mental health systems?”
“What are the available methods for standard description of mental health service delivery which could be applicable for international context analysis of mental health systems?”
“What are the key domains or components of methods for context analysis in mental health systems research?”
An additional objective of this scoping review was to identify a workable set of search terms that optimise the literature review in this new research area.
In order to answer these questions we have adopted a systems dynamics/complexity approach , and a modified version of Tansella and Thornicroft’s matrix model of mental health care (TT-Matrix)  (Table 1) as the conceptual framework for our scoping analysis. Tansella and Thornicroft developed this framework to facilitate the “bridging of information between different levels of analysis” , and to address issues related to system complexity encountered in mental health systems research: for example, the conflating of proxies of inputs or processes such as the number of psychiatric beds used, with outcome; and a failure to take account of evidence obtainable at different levels of the system through a reliance on experimental evidence gained at the individual or micro level . The matrix concept has continued to be developed in mental health services research to provide a basis for mental health performance measurement [16, 17]. The modified version of the TT-Matrix (mTT-Matrix) provides 12 quadrants of indicators of health care according to the Donabedian process of care (input, throughput and output ); and the levels of care: 1) macro (country or region); 2) meso (local-catchment areas); 3) micro (facilities, services, care teams); and 4) nano (individual agents such as consumers, carers and professionals). We are looking specifically at the care service delivery system at the meso level (quadrant 2A), and the aggregation of information from the micro level to the meso level (quadrant 3A), and from meso level to macro level (quadrant 1A).
Identifying relevant studies
A systematic search was carried out, using the above research questions: the period of reference was 2005–2016. Databases used were the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science (WoS) and Medline databases. LSC and MF selected the search terms. Broad terminology was required due to the low specificity of the applicable terminology. The search was carried out with the assistance of an academic librarian. The search terms used for the first search using CINAHL, WoS and Medline were (“mental health care” OR “mental health care delivery” OR “mental health service*” OR “mental health system*” OR “psychiatric service” OR “psychiatric care”) AND (classification OR description OR availability OR “meso-level analysis” OR “meso level analysis” OR “geographical mapping” OR mapping OR “healthcare instrument” OR “health care instrument” OR “healthcare tool” OR “health care tool” OR “local care”).
Some key articles known to the authors were noted to be missing, so an additional search was carried out using key words from those articles; these were mental health AND (“cross-country comparison*” OR “cross country comparison*” OR “international comparison*” OR “cross-cultural comparison*” OR “cross cultural comparison” OR “health system* research”). A search of the British Library on Demand database was also made using all the above key words. Further titles, by an author with an interest in the area, known to one of the authors (LSC) were added.
MF conducted the database search based on the search terms, and conducted a review of titles. Abstracts of potentially relevant papers were identified, and duplicates were deleted. Studies were initially included if they described or conceptualised the context of mental health care; mapping of mental health services; service availability, capacity or accessibility in geographic areas, or instruments assessing service availability, capacity or accessibility. Initial exclusion criteria were papers only reporting on service utilisation, interventions, financing and costs, and governance, due to their being not specifically related to availability. Also excluded as being too limited in scope were studies related to specific groups, such as child and adolescent mental health, mental health of culturally and linguistically diverse (CALD) populations, forensic mental health, or veterans’ mental health. Conference abstracts and non- scientific literature were excluded as their inclusion would have created an unfeasibly large database. Eligible study designs were broad, and included qualitative analysis gathered by experts, studies using a mixed approach, modelling studies, secondary analysis from databases, surveys and comparative studies. At this point we decided to include studies where the comparison was within countries and not just international or cross country, in case these methods could potentially also be used in cross country comparison.
The identified abstracts were reviewed by MF and CG, who discussed differences, and, where they could not be resolved, a further discussion was held with LSC. Study selection was an iterative process. In meetings with MF, CG and LSC, and due to increasing familiarity with the scope of papers, the search was refined, with additional exclusion criteria applied: papers reporting only on workforce or placement or bed capacity, or those including data exclusive to only one domain of care (residential, outpatient care, or day services), (unless describing all services in that domain) were excluded, again due to being too limited in scope. Micro level studies were also excluded as not relevant to the level of the system under study. It was noted that papers could be separated into conceptual, analytical and descriptive categories. At this point, a preliminary framework for data extraction was identified and piloted with five papers, based on the emerging picture of study characteristics.
The remaining full texts were read by MF and CG. Papers were excluded at this stage again for limited or incompatible interpretations of the concept of service availability, including service utilisation, service capacity only; or for providing no data on availability. A further number of grey literature articles were excluded. Conceptual papers were also excluded at this point as being outside the scope of the question, which related specifically to methods used. References of included papers were hand searched for further articles by MF and CG and cross checked in the same manner.
MF and CG then met again with LSC, and discussed the different categories of data to be extracted from the included papers.
Charting the data
A data extraction tool was discussed, based on the characteristics of the included papers. It was piloted with five papers by LSC, MF and CG. MF and CG then each used the tool on all the included papers, following which they reviewed each others’ decisions. Differences were discussed, and any that could not be resolved were discussed with LSC for a final decision.
The data extraction tool categorised papers into descriptive and analytic studies. It then focussed on the key characteristics of the studies, and finally on the methods used in the mental health system descriptions. Extracted study characteristics were those describing the type or scope of services and included target population (specific target population such as people with lived experience of mental illness, carers, specific diagnostic group such as depression, eating disorders, etc.) and whether this was formally defined; socio-economic context if described; the sectors described (health, social, education, employment, housing, other); service types (hospitals, clinics and so forth); care branches (domains of service delivery); workforce capacity (types of professionals); placement capacity (beds or places where described), and geographic accessibility (distance to services for service users). Variables relating to the methods used included the framework (if the study used a standardised framework); study geographical boundary and whether this was formally defined; level of analysis (macro, meso, or micro as above); classification or taxonomy if included in framework; study design; and presence and type of comparison.
Collating, summarising and reporting the results
We first performed a numerical analysis of the characteristics of the papers to provide an overall picture of the geographic and demographic characteristics of the studies, and basic methodology (whether or not a standardised framework was utilised). As the methods used to describe mental health care delivery included several instruments, we then created a table of key analytical characteristics of each instrument. All in all, six instruments were used in the studies (see Table 3). While the scientific literature included many papers using data obtained through these instruments, in several papers only data from selected sections of the particular instrument were used, or in the case of WHO Assessment Instrument for Mental Health Systems (WHO-AIMS) and the Mental Health Country Profile (MHCP), only selected extracts from the whole country report were included in the study. Therefore, where possible, the full characteristics of these tools have been gathered from the original report documentation to enable a full description of the instrument or framework. In the case of the Adult Service Mapping Exercise (ASME), we were not able to locate the core instrument online. Following this, we analysed the key conceptual approaches taken by the identified methods of context analysis of mental health care delivery.
10,911 titles were identified in the initial search. After removal of duplicates, 6149 papers remained. After review of titles, 444 abstracts remained, following review of which 271 were excluded. Ninety-five were not relevant to the topic; 57 were not mental health related; 94 papers were excluded due to interpretations of the concept that were either limited (one type or branch of care only), or incompatible with the study concept (for example studies of service or resource utilisation, system governance, interventions, or care needs); 10 previously unidentified articles of grey literature were excluded at this point, as were 14 papers relating to areas of care outside the inclusion criteria, such as child and adolescent mental health, or CALD mental health. A previously unidentified duplicate was removed. Members of the team introduced three more papers for consideration based on knowledge of the scope of the study. The remaining 176 full text articles were reviewed independently by CG and MF: 130 papers were excluded either because they were not relevant to the topic; had a too limited scope (i.e. they related only to service capacity or to one type of service such as hospital acute care), or were commentaries and conceptual papers. All in all, 46 studies were eligible for inclusion in our scoping review (Fig. 1).
A shared meaning of key concepts in the assessment of mental health care delivery was lacking. For example, in full text papers reviewed, a number of papers were excluded where the concept of service availability had been variously interpreted as service utilisation, service workforce and service capacity. Thirty three papers related to service availability were excluded because they provided no data, 17 papers were excluded because they provided data only on workforce capacity, and seven papers were excluded because availability was conceptualized as either service utilisation or as availability of interventions.
Characteristics of included studies
Of the 46 eligible studies, 36 (78.3%) [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54] were descriptive, and 10 (21.7%) were analytical [55,56,57,58,59,60,61,62,63,64]. Thirty six papers (80.4%) presented service availability data from a single country, of which 19 [20, 28, 32,33,34, 37, 40,41,42, 50, 51, 53,54,55,56,57, 59, 61, 64] took a regional or local approach, while 17 [19, 21,22,23,24,25,26,27, 30, 36, 38, 39, 43,44,45,46, 49] looked at availability from a national level. Ten papers presented service data from more than one country, of which seven [29, 35, 47, 48, 52, 58, 60] took a regional or local approach, and three [31, 62, 63] were at the national level. Overall, excluding two papers which included over 40 Lower Income Countries and Lower-Middle Income Countries (LIC/LMIC), not all of which were identified, 22 papers (48%) used data from Europe, most notably Spain and Italy, nine papers (20%) were from Africa, seven (15%) from Asia, four (9%) from the Middle East, two (4%) from the Americas (one from USA and one from Chile), and one (2%) from Australasia. Of the LIC/LMIC countries studied, eight were from Africa, and three were from Asia. However, in 25 studies (54.3%) the precise boundaries of the study area were not formally defined.
Twenty eight studies (60%) provided socio-demographic context [21, 24, 25, 27, 29, 31, 34, 37,38,39,40, 42,43,44,45, 47,48,49,50, 52, 53, 55,56,57, 59,60,61,62]. Two papers [34, 53] which presented data from atlases of mental health care included comprehensive local area data. Of the 16 studies which linked one or more socio-demographic indicators with mental health, only four provided supporting evidence with validated indicators using a standardised instrument (e.g. European Social Demographic Schedule -ESDS) [34, 48, 59, 60]. These four papers all used the European Service Mapping Schedule (ESMS) for service availability data. Papers based on WHO-AIMS and MHCP instruments also included legislative and policy context at a national level.
Where target populations were formally defined, 11 studies included children and/or adolescents [19, 21, 25, 30, 31, 36, 40, 50, 51, 53, 63]; three studies included people with alcohol and other drug dependence (AOD) [36, 61, 63]; two studies included people with intellectual disability (ID) [21, 36]; three were specific to serious mental illness or psychosis [57, 62, 63]; two included people over 65 years [21, 51]; and one study each included the following subpopulations: maternal/perinatal mental health ; people requiring long term rehabilitation ; survivors of suicide attempts ;and socially marginalized groups . A further 21 studies did not specify a particular mental health population.
The main characteristics of included studies are detailed in Table 2.
We then analysed the methods used in included studies (Table 3). Six instruments providing data on service availability were identified in the included studies, and these were used in a total of 31 papers. Three of these were psychometrically validated instruments: ESMS/DESDE (Description and Evaluation of Services and Directories for Long Term Care-an evolution of the ESMS and thus described together) (used in 12 papers: [20, 28, 34, 48, 52, 53, 56, 58,59,60,61, 64]); WHO-AIMS: (used in 11 papers [21, 22, 24, 25, 27, 31, 33, 50, 51, 62, 63]); and MHCP (used in three papers [43,44,45]). ESMS/DESDE and WHO-AIMS are based on taxonomies of care (ESMS/DESDE on a hierarchical tree taxonomy), and DESDE has undergone formal ontological analysis . The MHCP is structured into four domains relevant to policy, including context, resources, provision and outcomes. However, while the MHCP provided a taxonomy for mental health systems generally, it should be noted that the domains for health service delivery did not include any classification of service types. Two other instruments- those of the Best Practice In Promoting Mental Health In Socially Marginalized People In Europe study (PROMO) in 14 European capital cities  and the Programme for Improving Mental Health Care in five LMICs study (PRIME)  were designed specifically for those studies, and were included in one paper each. The ASME, used in three papers [23, 54, 55], was designed specifically for the English context. WHO-AIMS, MHCP, and the instruments from the PRIME and PROMO studies are instruments designed specifically for mental health services, while ESMS/DESDE and ASME have a broader health service application. ESMS/DESDE was developed for all long term care services. Fifteen studies did not use a structured framework [19, 26, 30, 32, 35,36,37,38,39,40,41,42, 46, 49, 57], of which five did not provide any method [37, 40,41,42, 46]. Four of these [37, 40, 41, 42] formed part of a group of seven papers in a special supplement related to a conference on mental health care in capital cities: however three of this seven papers were excluded from this study as they did not include any data on service availability.
In the case of ESMS/DESDE papers, the unit of analysis was care teams provided by individual services, aggregated at local level (2A in the mTT matrix), while in WHO-AIMS, ASME and MHCP papers, services data was aggregated at national level (1A in the mTT matrix). Of the 23 papers not using taxonomy based instruments (i.e all those papers not using ESMS/DESDE or WHO-AIMS), eight, including all three papers using the MHCP, counted services provided at a higher organisational level of care, such as psychiatric hospitals in a local area, along with individual services, such as day centres or mental health departments in larger organisations, thus conflating these different levels of care [30, 36, 37, 39, 40, 43,44,45]. In a further seven papers [23, 29, 35, 41, 42, 55, 57], including two of the three papers using the ASME, individual services were conflated in the same way with individual care teams (section 4A of the mTT matrix) such as crisis resolution teams, or assertive outreach teams.
Of the 15 papers which did not use a specific instrument to frame their analysis of service availability data, three [30, 36, 39] used internationally based frameworks, five [19, 26, 32, 49, 57] used a framework relevant specifically to the region in which the study took place, four [37, 40,41,42] categorised their data around service types but did not justify their categorisation or their choice of units of analysis, and three [35, 38, 46] did not specify any framework for their data on service availability. Of those studies using international frameworks, two [30, 36] were based on the Mental and Social Health Atlas of Saudi Arabia, which used the framework provided by the WHO Mental Health Atlas, while the third drew broadly on the WHO Mental Health Atlas, as well as recommendations from the 2001 WHO World Health Report to structure their findings . Three studies described service availability according to the specific structure of the national system under study [19, 26, 49], while one described service availability based on a regionally prescribed framework of services required for the prevention of recurrent suicidal behaviour .
Terminology used to identify units of analysis varied widely, but only ESMS/DESDE and WHO-AIMS provide glossaries of terms used. MHCP studies included detailed qualitative data at the local level in order to ameliorate the effect of terminological variability on data interpretation. Terms used in papers for residential care included “psychiatric hospitals”, “supportive homes”, “crisis homes”, “safe homes”, “social rehabilitation centres”, “group homes”, “short and long term residential units”, “community based psychiatric inpatient units, respite, and community residential facilities” and those for non-residential care including “day hospitals” “psychiatric clinics”, “outpatient clinics”, “day centres”, “mental health dispensaries”, “mental health departments in social diseases prevention centres”, “day treatment facilities”, “fixed clinics”, “outpatient department”, “community mental health centres”, “sheltered workshops”, “day activity services”; “crisis resolution teams”, “assertive outreach teams”, “early intervention in psychosis team”, “home care nursing services”, and “mobile crisis teams”.
Data was obtained from sources at different levels of the health system. Studies using the ESMS and DESDE and the PROMO instrument take a bottom up approach, gathering data from providers at individual service level. WHO-AIMS takes a top down approach, the papers using this instrument collecting national data at a high level from sources such as heads of departments, universities, and professional boards. Where the instrument was used at a regional level, data was collected from similar sources at that level. In these studies however, the data is still interpreted through a national prism. Papers using the MHCP instrument and that of the PRIME study used both national and local sources, both methods combining national level data with qualitative data from the local level gathered from sources including professionals, clients, families and other stakeholders. The PRIME study is undertaken at district level, but uses a top-down approach, with data from administrative databases, key officials and service heads. Data for the ASME was gathered at a national level from Local Implementation Teams, although one paper  first identified relevant Trusts providing rehabilitation services using the ASME, and then went to the individual units to obtain data. In the 15 papers using other, non- framework based methods, existing administrative databases or literature were sourced, with four also using surveys sent to senior health or government officials [36, 38, 39, 57].
Seven studies included the health sector only [30, 32, 38, 39, 46, 62, 63]. Eighteen studies included the health and social sectors [19, 20, 22, 23, 26, 29, 31, 33, 45, 49, 50, 52, 54,55,56, 59, 61, 64]. This included papers using MHCP and ASME. At least one other sector, such as employment, education, justice, or housing was included in almost half of included studies (21 papers) [21, 24, 25, 27, 28, 34,35,36,37, 40,41,42,43,44, 47, 48, 51, 53, 57, 58, 60]. This included papers using ESMS/DESDE, WHO-AIMS, and those from the PRIME and PROMO study. The instrument of the PROMO study included several sectors, but for a limited target population (marginalised populations).
Of the 36 studies undertaken within a single country, seven [28, 51, 53,54,55, 59, 61] included comparison at regional or local level, and four included a comparison over time [19, 30, 32, 38]. All of the ten cross country studies included comparison of service availability: seven at regional or local level [29, 35, 47, 48, 52, 58, 60], and three at national level [31, 62, 63].
Forty-one papers (89%) identified themselves, or were assessed by us, as being situational and/or gap analyses. The remaining five papers comprised the following: efficiency analyses [58, 64] territorial planning , ecological analysis  and standard description for comparison . Thirty-two studies (70%) included recommendations for policy makers related to service provision based on the findings. Visual tools were used in 12 papers (25%), four of which incorporated graphics issued by Geographical Information Systems. In three of these the visual tool presented data on service availability.
The methodological characteristics of included papers are summarised in Table 3.
In those papers using instruments to provide data on service availability, this was WHO-AIMS in 11 papers (24%) [21, 22, 24, 25, 27, 31, 33, 50, 51, 62, 63], ESMS/DESDE in 12 papers (26%) [20, 28, 34, 48, 52, 53, 56, 58,59,60,61, 64], MHCP in three papers [43,44,45] (7%), ASME in three papers (7%) [23, 54, 55] and the PRIME  and PROMO  project instruments in one paper each (2%).
To our knowledge this is the first scoping review on methods for context analysis of system provision and healthcare ecosystems research in mental health. Scoping reviews are appropriate in new areas of research, where they can “identify gaps in the research knowledge base, clarify key concepts, and report on the types of evidence that inform practice in the field” . They “examine the extent, range and nature of research activity” . Research questions are thus “less likely to address very specific research questions” but become more focussed in an iterative approach, due to the requirement that they identify all relevant literature regardless of design . They are broad in nature to provide breadth of coverage: comprehensiveness and breadth are important in this search . Thus, scoping studies may often produce very high numbers of initial results [10, 66, 67]. The lack of a clearly defined terminology, reflected in the wide range of search terms which needed to be included, reinforces the need for an approach taking a broad view of the literature. For these reasons, a scoping review was considered to be a more appropriate review method than a systematic review, which would require a focussed question with clearly defined outcomes.
Implications for research
The WHO has called for description of systems of mental health care delivery and the gap analysis , but few standardised and validated methods are available to do so. Despite the complexity of mental health systems, many studies lack key methodological components such as a standardised framework, explanation of terminology, or explanation for choice of units of analysis: of the 46 papers included, 21 had serious methodological limitations, limiting their validity in international comparisons. The final number of included studies relative to the high number of initial results in the literature search indicates both a limited amount of research, and a lack of targeted and standardised research terminology in the area. The limited number of studies providing an explanation of the concepts or terms used presents difficulties when comparing systems, particularly across regions or countries, where the variation between systems may be greatest. The exclusion of full text papers due to limited interpretation of the concept of availability, or a conflation of availability with utilisation, demonstrates the lack of conceptual clarity in research in this area.
Comparisons between systems of care enable the sharing of knowledge, assist in problem solving and inform best practice. However, the replicability and comparability of several studies was undermined by a lack of clarity around terminology and scope, by the absence of structural organisation such as a taxonomy, and by inaccessibility or poor accessibility of some core instruments. A standardised framework was used in only half of those studies providing comparisons, and target populations were often either not specified (21 papers) or were very broad. The dearth of studies providing an explanation of the concepts or terms used was particularly relevant in comparisons across regions or countries, where the variation between systems may be greatest. Variation in terminology also creates a commensurability risk if units of analysis are not clearly defined and located within the overall system. The need for internationally agreed glossaries of terms has been underscored recently . While the use of international frameworks enables international comparison, where the frameworks for data analysis are specific to a specific country or region, this is not the case. Lack of an analytical framework, or of a justification of the choice of units of analysis, limits the relevance of findings.
A systems thinking approach in health services research has been widely advocated . Methods such as those of the ASME or MHCP which included only health or one other sector, may fail to identify information from other parts of the system key to an accurate analysis. Whole system analyses such as the Atlases of health described in two papers, taking into account the wider ecosystem in which healthcare operates, will be increasingly relevant to the emerging discipline of health ecosystems research.
Socio-demographic indicators varied, and were frequently not linked to evidence supporting their use in relation to need for mental health services. The level of availability of socio-demographic data was consistent with the level of availability of service delivery data presented in each article: i.e. national level socio-demographic characteristics where service delivery at a national level was reported. However, difficulty in obtaining relevant socio-demographic data was described in several papers, particularly those reporting studies carried out in LIC/ LMICs, or at lower than national level. This, and the identification of only one standardised instrument to collect such data suggests the need for a more systematic approach to the provision of socio-demographic context for assessment of service availability to be made within the context of local need.
Data aggregated at national level is not necessarily representative of the pattern of care across smaller areas, and may result in ecological fallacy. Additionally, administrative databases may be unreliable data sources, particularly in less resourced countries . A bottom-up approach, gathering data at the local or regional level, can provide a more accurate and detailed picture of health care availability in small areas. However, local data can also be unreliable, difficult to obtain, and may not be collected routinely at local level. One paper identified in the search  related to the Emerging Mental Health Systems in LMICs’ (EMERALD) project. While it did not include data on availability, and was thus excluded from the study, it focused on capacity building for mental health research in these countries, and is thus critical in this area, particularly in the context of the relatively low number of studies identified from LIC/LMIC. Of the identified instruments using local sources of evidence, only two (ESMS/DESDE and MHCP) were standardised and psychometrically validated, and only one of these (ESMS/DESDE) gathered data on availability at this level, enabling its use in comparative studies.
Implications for policy
Policy makers require evidence from the local context as well as global evidence at all stages of the policy making process to inform policy options . Data on service availability and capacity using a whole system approach can help identify gaps or duplications in care delivery, enable comparison of best practice with other areas, and assist in the prediction and monitoring of the effect of interventions. However, the research to policy gap is well documented. Guidance for health systems which is “transparent, systematic and adapted to the local contexts…(and) …use (s) validated approaches.., in user-friendly formats” can bridge this gap . Studies which use validated instruments and a bottom up approach, collaborating with local services and policy makers to identify local need, collect data and validate information gathered, are most likely to satisfy these criteria . Interpretive aids such as visual tools and glossaries, which increase accessibility to complex data could also improve dissemination and policy uptake.
Limitations of the study
1/ LSC participated in the development of one of the tools which introduced potential bias. However, this was limited by the selection process being undertaken by MF and CGE, who were not involved with the development of the system, and at this time had no experience in the use of it.
2/ Grey literature was not included in this review. However, as stated, a very high number of results were returned by the search due to factors such as imprecise terminology in the area. Had grey literature also been included, the number of results could have threatened the feasibility of the review. In some cases, copyright restrictions or lack of availability of the core instrument meant we could not access the core instrument.
Recommendation for future studies
The development of validated guidelines for the analysis of context of local service delivery is needed to increase the reliability of context studies and their relevance to policymakers through a more standardised approach. These should use a whole systems approach and provide standards for the description and grouping of target populations for international comparisons. They should also include interpretive aids such as glossaries to standardise terminology and key conceptual terms, as well as visual representations of complex data.
Further research is needed in LIC/LMIC to redress the current balance favouring Upper Income Countries in research. Developing capacity in LIC/LMIC through projects such as the EMERALD project, as well as standardised frameworks to enable comparison, is needed to enable this.
Future studies should ensure their core instrument is accessible for replicability. They should also systematically assess socio-economic context and formally define target population. There is a need for more analytical studies as opposed to purely descriptive papers.
This scoping review has identified that context studies in mental health services is an area of limited research. Instruments with which to assess service availability are scarce, with some of those identified not easily accessible or unable to be generalised. Fifteen papers, or around one third of included studies did not use any kind of formal framework, and five of those made no description of method. Most studies presented a limited view of the system under study, even when using data collected by instruments designed to take a wider systems view. Four of the six instruments identified (ESMS/DESDE, WHO-AIMS, and the instruments of the PRIME and PROMO studies) took a whole system approach, but two of these (WHO-AIMS, PRIME) were from a top down perspective, and thus constrained by the limitations to local relevance of aggregated data. One instrument (ESMS/DESDE) is readily accessible and validated, and takes both a local approach and a whole systems perspective, and was used in 12 papers. In general, the challenges of commensurability, of terminological variability, and of data availability and validity which face this area of research are poorly addressed, with few standardised frameworks available and only three of these (ESMS/DESDE, WHO-AIMS, MHCP) having undergone psychometric testing. This presents a barrier to valid system comparison, particularly across regions or countries, where regional and historical variations in service provision increase terminological variability. On the other hand, we have identified the relevance to this area of research of use of a standardised instrument, formal geographic boundaries, a glossary of terms, formal target populations and a whole systems approach.
Alcohol and Other Drugs
Adult Service Mapping Exercise
Culturally and Linguistically Diverse
Cumulative Index to Nursing and Allied Health Literature
Description and Evaluation of Services and Directories for Long Term Care
Emerging Mental Health Systems in Low and Middle Income Countries
European Social Demographic Schedule
European Service Mapping Schedule
Low and Middle Income Countries
Mental Health Country Profile
WHO Mental Health Gap Action Program
Modified Tansella and Thornicroft matrix
Programme for Improving Mental Health CarE
Best Practice In Promoting Mental Health In Socially Marginalized People In Europe
Tansella and Thornicroft’s matrix model of mental health care
World Health Organisation
World Health Organisation Assessment Instrument for Mental Health Systems
Web of Science
Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016 May;4(16):1–136.
Lee BY, Wong KF, Bartsch SM, Yilmaz SL, Avery TR, Brown ST, et al. The Regional Healthcare Ecosystem Analyst (RHEA): a simulation modeling tool to assist infectious disease control in a health system. J Am Med Inform Assoc. 2013 Jun 1;20(e1):e139–46.
Y Chung, L Salvador-Carulla, J Salinas-Perez, J Uriarte-Uriarte, A Iruin-Sanz, CR García-Alonso. Use of the Self-Organising Map Network (SOMNet) as a Decision Support System for Regional Mental Health Planning. Health Res Policy Syst. 2018;In press.
Salvador-Carulla L, Lukersmith S, Sullivan W. From the EBM pyramid to the Greek temple: a new conceptual approach to Guidelines as implementation tools in mental health. Epidemiol Psychiatr Sci. 2017;26(2):105–14.
Lewin S, Oxman AD, Lavis JN, Fretheim A, Marti SG, Munabi-Babigumira S. SUPPORT Tools for evidence-informed Policymaking in health 11: Finding and using evidence about local conditions. Health Res Policy Syst Lond. 2009;7(Suppl 1):S11.
World Health Organization, Mental Health Gap Action Programme, World Health Organization. MhGAP: Mental Health Gap Action Programme: scaling up care for mental, neurological, and substance use disorders [Internet]. 2008 [cited 2017 Nov 21]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK310851/
Bate P, Robert G, Fulop N, Øvretviet J, Dixon-Woods M. Health Foundation (Great Britain). Perspectives on context: a collection of essays considering the role of context in successful quality improvement : original research. 2014.
Goodwin N. Understanding integrated care: a complex process, a fundamental principle. Int J Integr Care [Internet]. 2013 22 [cited 2018 Feb 26];13(1). Available from: http://www.ijic.org/articles/abstract/10.5334/ijic.1144/
THORNICROFT G, TANSELLA M. Balancing community-basedand hospital-based mental health care. World Psychiatry. 2002 Jun;1(2):84–90.
Brien SE, Lorenzetti DL, Lewis S, Kennedy J, Ghali WA. Overview of a formal scoping review on health system report cards. Implement Sci Lond. 2010;5:2.
Arksey H, O’malley L. Scoping Studies: Towards a Methodological Framework. Int J Soc Res Methodol. 2005;8(1):19–32.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci Lond. 2010;5:69.
Peters MDJ, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141.
Frenk J. The Global Health System: Strengthening National Health Systems as the Next Step for Global Progress: e1000089. PLoS Med San Franc. 2010 Jan;7(1):e1000089.
Tansella M, Thornicroft G. A conceptual framework for mental health services: the matrix model. Psychol Med. 1998;28(3):503–8.
Vinci ALT, Rijo RPCL, Marques JM de A, Alves D. Proposal of an Evaluation Model for Mental Health Care Networks Using Information Technologies for its Management. Procedia Comput Sci 2016;100:826–831.
Buckingham & Associates Pty Ltd. National Mental Health Benchmarking Project Manual Part 2: extracted from Key Performance Indicators for New Zealand’s Mental Health Services - Background paper [Internet]. New Zealand Ministry of Health; 2004. Available from: https://www.amhocn.org/sites/default/files/publication_files/benchmarking_manual_part_2.pdf
Donabedian A. The Quality of Care: How Can It Be Assessed? JAMA. 1988;260(12):1743.
Kahng SK, Kim H. A Developmental Overview of Mental Health System in Korea. Soc Work Public Health. 2010;25(2):158–75.
Dernovsek MZ, Sprah L. A preliminary report on the implementation of the European Service and Mapping Schedule (ESMS) in Slovenia to map preventive and promotion activities in mental health. Psychiatr Danub. 2008;20(2):141–7.
Ofori-Atta A, Read UM, Lund C. A situation analysis of mental health services and legislation in Ghana: challenges for transformation. Afr J Psychiatry. 2010;13(2):99–108.
Li Y, Jia F, Hou C, Ryan B, Blashki G, Everall I, et al. A survey of the mental health workforce in Guangdong: implications for policy and workforce planning. Australas Psychiatry. 2015;23(6):675–8.
Glover G. Adult mental health care in England. Eur Arch Psychiatry Clin Neurosci. 2007;257(2):71–82.
Roberts M, Mogan C, Asare JB. An overview of Ghana’s mental health system: results from an assessment using the World Health Organization’s Assessment Instrument for Mental Health Systems (WHO-AIMS). Int J Ment Health Syst. 2014;8:16.
Saymah D, Tait L, Michail M. An overview of the mental health system in Gaza: an assessment using the World Health Organization’s Assessment Instrument for Mental Health Systems (WHO-AIMS). Int J Ment Health Syst. 2015;9:4.
Lora A. An overview of the mental health system in Italy. Ann DellIstituto Super Sanità. 2009;45:5–16.
Kigozi F, Ssebunnya J, Kizza D, Cooper S, Ndyanabangi S. An overview of Uganda’s mental health care system: results from an assessment using the world health organization’s assessment instrument for mental health systems (WHO-AIMS). Int J Ment Health Syst. 2010;4:1.
Dernovsek MZ, Sprah L. Assessment of mental health services in Slovenia with The European Service Mapping Schedule. Psychiatr Danub. 2008;20(3):439–42.
Hanlon C, Luitel NP, Kathree T, Murhar V, Shrivasta S, Medhin G, et al. Challenges and Opportunities for Implementing Integrated Mental Health Care: A District Level Situation Analysis from Five Low- and Middle-Income Countries: e88437. PLoS One San Franc [Internet]. 2014;9(2). Available from: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0088437. [cited 2017 Nov 26].
Koenig HG, Al Zaben F, Sehlo MG, Khalifa DA, Al Ahwal MS. Current State of Psychiatry in Saudi Arabia. Int J Psychiatry Med. 2013;46(3):223–42.
Saxena S, Lora A, Morris J, Berrino A, Esparza P, Barrett T, et al. Focus on Global Mental Health: Mental Health Services in 42 Low- and Middle-Income Countries: A WHO-AIMS Cross-National Analysis. Psychiatr Serv Arlingt. 2011;62(2):123–5.
Neri G, Guzzetta F, Pazzi L, Bignami R, Picardi A, de Girolamo G. How Does the Residential Care System Change? A Longitudinal Survey in a Large Region of Italy. Community Ment Health J. 2011;47(2):201–8.
Sikwese A, Mwape L, Mwanza J, Kapungwe A, Kakuma R, Imasiku M, et al. Human resource challenges facing Zambia’s mental health care system and possible solutions: Results from a combined quantitative and qualitative study. Int Rev Psychiatry. 2010;22(6):550–7.
Fernandez A, Gillespie JA, Smith-Merry J, Feng X, Astell-Burt T, Maas C, et al. Integrated mental health atlas of the Western Sydney Local Health District: gaps and recommendations. Aust Health Rev Collingwood. 2017;41(1):38–44.
ITO H, SETOYA Y, SUZUKI Y. Lessons learned in developing community mental health care in East and South East Asia. World Psychiatry. 2012;11(3):186–90.
Al-Habeeb A, Helmi B, Qureshi N. Mental and Social Health Atlas: An Update, Ministry of Health, Saudi Arabia, 2015. Int Neuropsychiatr Dis J. 2016;6(3):1–20.
Tosevski DL, Pejuskovic B. Mental health care in Belgrade — Challenges and solutions. Eur Psychiatry. 2005;20(Supplement 2):S266–9.
Li K, Sun X, Zhang Y, Shi G, Kolstad A. Mental health care in China: review on the delivery and policy issues in 1949–2009 and the outlook for the next decade. Acta Neuropsychiatr. 2014;26(3):134–45.
Chahine LM, Chemali ZN. Mental health care in Lebanon: policy, plans and programmes. East Mediterr Health J. 2009;15(6):1596-612.
Ferre Navarete F, Palanca I. Mental health care in Madrid. Eur Psychiatry. 2005;20:S279–84.
Raboch J, Wenigova B. Mental health care in Prague. Eur Psychiatry. 2005;20(Supplement 2):S289–93.
Gaddini A, Ascoli M, Biscaglia L. Mental health care in Rome. Eur Psychiatry. 2005;20(Supplement 2):S294–7.
Jenkins R, Heshmat A, Loza N, Siekkonen I, Sorour E. Mental health policy and development in Egypt - integrating mental health into health sector reforms 2001-9. Int J Ment Health Syst. 2010;4:17.
Kiima D, Jenkins R. Mental health policy in Kenya -an integrated approach to scaling up equitable care for poor populations. Int J Ment Health Syst. 2010;4:19.
Niemi M, Thanh HT, Tuan T, Falkenberg T. Mental health priorities in Vietnam: a mixed-methods analysis. BMC Health Serv Res Lond. 2010;10:257.
Liu J, H M, YL H, B X, YF X, HY T, et al. Mental health system in China: history, recent service reform and future challenges. World Psychiatry. 2011;10(3):210–6.
Priebe S, Matanov A, Barros H, Canavan R, Gabor E, Greacen T, et al. Mental health-care provision for marginalized groups across Europe: findings from the PROMO study. Eur J Pub Health. 2013;23(1):97–103.
Salvador-Carulla L, Saldivia S, Martinez-Leal R, Vicente B, Garcia-Alonso C, Grandon P, et al. Meso-level Comparison of Mental Health Service Availability and Use in Chile and Spain. Psychiatr Serv Arlingt. 2008;59(4):421–8.
Picardi A, Lega I, Candini VP, Dagani JP, Iozzino LP, de Girolamo G. Monitoring and Evaluating the Italian Mental Health System: The “Progetto Residenze” Study and Beyond. [Review]. J Nerv. 2014;202(6):451–9.
Petersen I, Bhana A, Campbell-Hall V, Mjadu S, Lund C, Kleintjies S, et al. Planning for district mental health services in South Africa: a situational analysis of a rural district site. Health Policy Plan. 2009;24(2):140–50.
Lund C, Kleintjes S, Kakuma R, Flisher AJ. Consortium the MhRP. Public sector mental health systems in South Africa: inter-provincial comparisons and policy implications. Soc Psychiatry Psychiatr Epidemiol. 2010;45(3):393–404.
Rezvyy G, Øiesvold T, Parniakov A, Ponomarev O, Lazurko O, Olstad R. The Barents project in psychiatry: a systematic comparative mental health services study between Northern Norway and Archangelsk County. Soc Psychiatry Psychiatr Epidemiol. 2007;42(2):131–9.
Fernandez A, Salinas-Perez JA, Gutierrez-Colosia MR, Prat-Pubill B, Serrano-Blanco A, Molina C, et al. Use of an integrated Atlas of Mental Health Care for evidence informed policy in Catalonia (Spain). Epidemiol Psychiatr Sci. 2015;24(6):512–24.
Killaspy H, Harden C, Holloway F, King M. What do mental health rehabilitation services do and what are they for? A national survey in England. J Ment Health. 2005;14(2):157–65.
Beecham J. Access to mental health supports in England: Crisis resolution teams and day services. Int J Law Psychiatry. 2005;28(5):574–87.
Pirkola S, Sund R, Sailas E, Wahlbeck K. Community mental-health services and suicide rate in Finland: a nationwide small-area analysis. Lancet. 2009;373(9658):147–53.
Cooper SL, Lezotte D, Jacobellis J, DiGuiseppi C. Does Availability of Mental Health Resources Prevent Recurrent Suicidal Behavior? An Ecological Analysis. Suicide Life - Threat Behav Wash. 2006;36(4):409–17.
Torres-Jiménez M, García-Alonso CR, Salvador-Carulla L, Fernández-Rodríguez V. Evaluation of system efficiency using the Monte Carlo DEA: The case of small health areas. Eur J Oper Res. 2015;242(2):525–35.
Tibaldi G, Munizza C, Pasian S, Johnson S, Salvador-Carulla L, Zucchi S, et al. Indicators predicting use of mental health services in Piedmont. Italy J Ment Health Policy Econ. 2005;8(2):95–106.
Salvador-Carulla L, Tibaldi G, Johnson S, Scala E, Romero C, Munizza C. Patterns of mental health service utilisation in Italy and Spain. Soc Psychiatry Psychiatr Epidemiol. 2005;40(2):149–59.
Ala-Nikkola T, Pirkola S, Kaila M, Saarni SI, Joffe G, Kontio R, et al. Regional Correlates of Psychiatric Inpatient Treatment. Int J Environ Res Public Health [Internet]. 2016 Dec [cited 2017 Nov 21];13(12). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5201345/
Lora A, Kohn R, Levav I, McBain R, Morris J, Saxena S. Service availability and utilization and treatment gap for schizophrenic disorders: a survey in 50 low- and middle-income countries. Bull World Health Organ. 2012;90(1):47–54B.
Bruckner TA, Scheffler RM, Shen G, Yoon J, Chisholm D, Morris J, et al. The mental health workforce gap in low- and middle-income countries: a needs-based approach. Bull World Health Organ. 2011;89(3):184–94.
Salvador-Carulla L, Garcia-Alonso C, Gonzalez-Caballero J, Garrido-Cumbrera M. Use of an Operational Model of Community Care to Assess Technical Efficiency and Benchmarking of Small Mental Health Areas in Spain. J Ment Health Policy Econ. 2007;10:87–100.
Salvador-Carulla L, Alvarez-Galvez J, Romero C, Gutiérrez-Colosía MR, Weber G, McDaid D, et al. Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: the eDESDE-LTC study. BMC Health Serv Res Lond. 2013;13:218.
Gagliardi AR, Whitney B, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci Lond [Internet]. 2016;11. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0399-1. [cited 2017 Nov 21].
Chambers D, Wilson P, Thompson C, Harden M. Social Network Analysis in Healthcare Settings: A Systematic Scoping Review. PLoS One San Franc. 2012;7(8):e41911.
Montagni I, Salvador-Carulla L, Mcdaid D, Straßmayr C, Endel F, Näätänen P, et al. The REFINEMENT Glossary of Terms: An International Terminology for Mental Health Systems Assessment. Adm Policy Ment Health Ment Health Serv Res. 2018;45(2):342–51.
Huckel Schneider C, Sturmberg J, Gillespie J, Wilson A, Lukersmith S, Salvador-Carulla L. Applying complex adaptive system thinking to Australian health care: Expert commentary. Sydney: Menzies Centre for Health Policy, University of Sydney; 2016.
Thornicroft G, Cooper S, Van Bortel T, Kakuma R, Lund C. Capacity Building in Global Mental Health Research: Harv Rev Psychiatry. 2012;20(1):13–24.
Semrau M, Evans-Lacko S, Alem A, Ayuso-Mateos JL, Chisholm D, Gureje O, et al. Strengthening mental health systems in low- and middle-income countries: the Emerald programme. BMC Med. 2015;13:79.
Bosch-Capblanch X, Lavis JN, Lewin S, Atun R, Røttingen J-A, Dröschel D, et al. Guidance for Evidence-Informed Policies about Health Systems: Rationale for and Challenges of Guidance Development. PLoS Med [Internet]. 2012 Mar 6 [cited 2017 Nov 26];9(3). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3295823/
This scoping/systematic review is part of a general strategy to improve the knowledge base on mental health systems research promoted by PSICOST organisation in collaboration with Universidad Loyola Andalucia and the Australian National University together with a series of other international health systems research organisations and public policy agencies. Future steps include: developing guidelines for context analysis in mental health care; knowledge transfer in context analysis and re-framing context analysis in healthcare ecosystems research.
We would like to acknowledge the valuable contributions of Elaine Tam to the initial literature search and of Dr. Karine Chevreul for her comments on the paper.
MF is the recipient of an Australian Government Research Training Program Scholarship for the purpose of her PhD, of which this paper forms a component.
Availability of data and materials
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.