- Research article
- Open Access
- Open Peer Review
A cognitive perspective on health systems integration: results of a Canadian Delphi study
© Evans et al.; licensee BioMed Central Ltd. 2014
- Received: 6 January 2014
- Accepted: 30 April 2014
- Published: 19 May 2014
Ongoing challenges to healthcare integration point toward the need to move beyond structural and process issues. While we know what needs to be done to achieve integrated care, there is little that informs us as to how. We need to understand how diverse organizations and professionals develop shared knowledge and beliefs – that is, we need to generate knowledge about normative integration. We present a cognitive perspective on integration, based on shared mental model theory, that may enhance our understanding and ability to measure and influence normative integration. The aim of this paper is to validate and improve the Mental Models of Integrated Care (MMIC) Framework, which outlines important knowledge and beliefs whose convergence or divergence across stakeholder groups may influence inter-professional and inter-organizational relations.
We used a two-stage web-based modified Delphi process to test the MMIC Framework against expert opinion using a random sample of participants from Canada’s National Symposium on Integrated Care. Respondents were asked to rate the framework’s clarity, comprehensiveness, usefulness, and importance using seven-point ordinal scales. Spaces for open comments were provided. Descriptive statistics were used to describe the structured responses, while open comments were coded and categorized using thematic analysis. The Kruskall-Wallis test was used to examine cross-group agreement by level of integration experience, current workplace, and current role.
In the first round, 90 individuals responded (52% response rate), representing a wide range of professional roles and organization types from across the continuum of care. In the second round, 68 individuals responded (75.6% response rate). The quantitative and qualitative feedback from experts was used to revise the framework. The re-named “Integration Mindsets Framework” consists of a Strategy Mental Model and a Relationships Mental Model, comprising a total of nineteen content areas.
The Integration Mindsets Framework draws the attention of researchers and practitioners to how various stakeholders think about and conceptualize integration. A cognitive approach to understanding and measuring normative integration complements dominant cultural approaches and allows for more fine-grained analyses. The framework can be used by managers and leaders to facilitate the interpretation, planning, implementation, management and evaluation of integration initiatives.
- Health systems integration
- Integrated care
- Shared mental models
- Group cognition
- Organizational culture
- Change management
Innovative healthcare delivery models often incorporate the concepts of integration and integrated care. By bringing together multiple professionals, services, and organizations, integration efforts aim to replace fragmented care with care that is coordinated and patient-centered . Despite the promise of improved efficiency, quality of care, and patient satisfaction, healthcare systems capable of delivering integrated care have not developed widely [2–4].
Over the past two decades, most scholars have focused on the structural and process challenges involved in integrating care. The resulting body of knowledge on integration barriers and enablers has informed many positive system changes. Progress has also been made in understanding, effecting, and evaluating various types of integration, including organizational, functional, service, and clinical integration . However, reports continue to highlight the seemingly intractable problems inherent in fostering collaboration and cooperation across professional and organizational boundaries [6–9]. These problems point toward the need to move beyond the integration of organizational structures and processes to understand the social cognitions that are implicated in integration efforts. Normative integration, defined as “an ethos of shared values and commitment to coordinating work [which] enables trust and collaboration in delivering healthcare”, may be the key to addressing ongoing challenges to integrated care delivery ; but we still know little about how to achieve and measure normative integration.
One way to explore normative integration is by examining organizational and professional cultures, defined as the beliefs and behaviour patterns dominant among a group of people . For example, at the outset of the implementation of an integration initiative one may gauge the extent to which there are discrepancies among cultures that will need to be addressed. In turn, post-implementation cultural harmonization may be used as evidence of normative integration. Although culture is rarely directly measured in studies of integration, cultural differences are often offered as explanations for failed or suboptimal integration initiatives [8, 11, 12]. Despite this popular attribution, there is little evidence on the effectiveness of strategies to change culture, and debates persist regarding the feasibility and desirability of merging various cultures in healthcare [13, 14].
An alternative perspective is to view integration through the lens of individual and shared mental models. Mental models are psychological representations, consisting of knowledge and beliefs, which enable interpretation and action in a specific domain . Mental models are developed over time through experience, direct communication and interaction with others, and vicarious learning . When multiple individuals develop a common psychological structure for understanding their environment, this is referred to as a Shared Mental Model (SMM) [15, 17]. Mental models inform perception and behavior, and shape culture in a co-evolutionary process [10, 18, 19]. Like cultural congruence, mental model similarity may also be used as an indicator of readiness to integrate, or post-implementation similarity as a sign of successful integration. While both constructs, culture and SMMs, help to describe and explain normative integration, a mental models perspective may allow for more fine-grained analyses of inter-organizational and inter-professional relations . Mental models may also be more amenable to change than cultures and SMMs may be a means for bridging diverse cultures [19, 21].
Drawing from an interdisciplinary literature review and SMM theory, Evans and Baker  developed a framework of mental model content, antecedents, and outcomes specific to integration initiatives in the healthcare sector. Their framework outlines important knowledge and beliefs whose convergence or divergence across stakeholder groups may influence inter-professional and inter-organizational relations. This study builds on that work, and tests and improves the Mental Models of Integrated Care (MMIC) Framework through an iterative process using expert opinion. An expert-validated framework will support the application of a cognitive perspective in future research and practice on healthcare integration.
In this study, we draw from Shared Mental Models (SMMs) theory, which posits that team effectiveness is maximized when members of a team have a shared understanding of their tasks and roles . SMMs fall into three broad categories: task-related (goals and performance requirements), team-related (interpersonal interaction requirements and skills of team members), and beliefs (preferences or expectations) . SMMs in these content areas improve team and organizational performance by facilitating coordinated action and adaptation under changing conditions [15, 22–24]. SMMs allow individuals to develop common views of what is happening, what is likely to happen next, and why it is happening, and thus guides their behaviours in ways that are consistent and coordinated with each other in the completion of interdependent tasks . Although diversity in perspectives has been linked to high decision quality and improved team performance, this is likely due to the positive effects of dissent at the strategy formulation or decision-making stage; at the point of action or implementation, however, SMMs positively impact performance . That being said, identical knowledge and beliefs are not necessary; convergence around a broad frame of mental models provides the common meaning needed for action, regardless of differing views on specific issues . Thus, we define “shared” as similar and overlapping, not identical, mental models among a group of individuals.
Research in strategic management, stakeholder management, change management, systems change, and team, organizational and network performance also emphasize the importance of developing common aims and shared understandings e.g., [27–29]. Although SMMs are group-level phenomena, the extent to which individual perspectives and behaviours are consistent (or not) can impact performance beyond the group level [28, 30, 31]. For example, in a study examining healthcare personnel’s mental models of the organizational implementation of clinical practice guidelines, personnel in high-performing facilities exhibited SMMs, while those in lower performing facilities did not . SMMs help to shape broader constructs such as inter-organizational macro-cultures, industry mindsets, and institutional logics, which also suggests that SMMs can manifest at the systems level [32–34]. Hence, SMMs may facilitate the implementation of complex system-level change involving the inputs of a multitude of diverse actors. Systems integration, which aims to link various sectors, organizations and professionals across the continuum of care, is a prime example of such a change. For a more thorough justification of the application of SMMs theory to healthcare integration and a discussion of strengths and limitations, please refer to Evans and Baker .
Applied to integration, the three mental model categories are integration-task, system-role, and integration-belief . An integration-task mental model encompasses the purpose and approach to integration and includes the following contents: services to be integrated; external customers; goals; long-term vision; and processes. A system-role mental model refers to one’s understanding of the system and its components, and includes the following contents: knowledge, skills and abilities of participating professionals and organizations; role clarity; role interdependence; role contribution; and interaction patterns. Finally, an integration-belief mental model includes relevant preferences and expectations as well as meanings, assumptions, and interpretations of key issues. This initial framework was derived from a review of previous theory and research . In this paper, we examine the extent to which the Mental Models of Integrated Care (MMIC) Framework, and its dimensions and contents, are relevant, acceptable and useful to those with expertise and experience in integration efforts, and we revise the framework based on this assessment.
We evaluated and refined the proposed MMIC Framework using expert opinion in a two-round web-based modified Delphi process. The Delphi method is a consensus-building technique that solicits the opinions of content experts in a given field through the use of a series of questionnaires combined with the provision of feedback . Using the participant list (n = 344) from the Health Council of Canada’s National Symposium on Integrated Care, we selected and invited a random sample of 172 individuals to participate in our study (50% of the population). The Symposium, held on 10 October 2012, brought together policymakers, planners, managers, care providers, educators, researchers and patient advocates from across Canada to share and promote the spread of innovative practices in integrated care.
Following the Symposium, experts were invited via email to participate in the study. They were asked to commit to completing two questionnaires (~30 minutes each) and to have access to email or the Web for receiving and returning questionnaires. Completing the questionnaires implied consent to participate. Respondents were assured confidentiality, but not anonymity. Coffee cards ($10 each) were offered as an incentive for participating in both rounds of the study. For each round, two reminders were sent to non-respondents via email. The study received ethics approval from the Office of Research Ethics at the University of Toronto (protocol #28076).
The questionnaire was developed and designed using Dillman’s  criteria for question and questionnaire design with Survey Monkey as the platform. Prior to administration, the questionnaire was pre-tested with eight volunteers (two care providers, one health services researcher, and five managers from a range of healthcare organizations) for clarity, and to anticipate the average completion time. All volunteers had knowledge of and/or experience in integration initiatives. As a result of the pre-test, modifications were made to the questionnaire’s length, instructions, and lay-out. The pre-test also highlighted differences in how “integration” and “integrated care” were defined and perceived. All the volunteers preferred “mental models of integration” over “mental models of integrated care” because the latter focuses exclusively on direct patient care to the perceived exclusion of other forms of integration, such as organizational and functional integration. We changed the terminology accordingly. We also made minor modifications to two of three mental model categories based on the pre-test: “integration-task mental model” and “integration-role mental model” were replaced with “strategy mental model” and “roles mental model” respectively.
The first round of the web-based questionnaire took place between 22 October and 12 November 2012 and consisted of 46 items (Additional file 1). The questionnaire provided a two-page overview of the proposed framework, followed by questions which asked respondents to rate the clarity, comprehensiveness, usefulness, and importance of the concepts in the framework, and the framework itself, using seven-point ordinal scales. Respondent demographics were also collected and spaces for comments were provided throughout the questionnaire.
The second questionnaire, sent only to those who responded to the first round, took place between 4 December and 20 December 2012 and consisted of 33 items (Additional file 2). The questionnaire did not include questions on respondent demographics and questions on importance were not repeated due to high ratings from round one. The general format and remaining questions were the same as in the first questionnaire with three additions: (a) a summary of the feedback from round one, including response rate, participant demographics, and major findings, (b) an enhanced introductory explanation of the framework, its background, and its potential applications, and (c) quantitative and qualitative feedback from the first questionnaire for each question. The feedback included an outline and explanation of the modifications made and not made to the framework based on the first round results. The second questionnaire was pre-tested for clarity and ease of completion prior to administration with a sub-set of four volunteers from the original group of eight from round one. No changes were made to the questionnaire as a result of the pre-test.
The results for both questionnaires were tabulated using quantitative and qualitative analyses. Descriptive statistics were used to describe the structured responses as a whole, while the open comments were coded and categorized using thematic analysis to identify common themes and key issues. The Kruskall-Wallis test was used to examine cross-group agreement by level of integration experience, current workplace, and current role. In the questionnaire, respondents had the option of selecting multiple roles and workplaces; less than 5% (n = 4) and 15% (n = 12) chose to do so respectively. For the purpose of analysis, however, each respondent was assigned only one role and one workplace based on the information they provided in the questionnaire (i.e., primary affiliation).
Round I questionnaire
Personal characteristics of respondents
Round I (n = 90)
Round II (n = 68)
76.7% (73% ON)
83.8% (79% ON)
Home & Community Care
University or Research Institute
Level of Integration3
No Direct Experience
Results from thematic analysis of open comments
Provided a three-page overview to explain:
We retained the term “mental model” for two reasons: (a) no other term conveys the intended meaning and (b) the term maintains the link between this work and the literature on mental models.
• Purpose of the framework
• The value of a cognitive perspective on integration
• Structure of the framework, particularly the section on beliefs/perceptions
• The broad meaning of the term “shared”
• What the framework aims, and does not aim, to do
• Practical applications of the framework
• How the framework may be used, including examples
In the original framework, the beliefs/perceptions mental model consisted of the same contents as in the knowledge category, but with a focus on “what should be” or “what is perceived or expected”. For example, “clients” can be viewed in terms of which populations are being targeted for integrated care (knowledge of the integration strategy) versus which populations should be targeted (personal belief/perception). Repetition of the content areas created confusion and the contents failed to capture important beliefs/perceptions. Respondents also noted that their way of thinking about integration was complex and dynamic, incorporating knowledge, beliefs and perceptions; they therefore did not view beliefs/perceptions as a separate mental model.
• Terms and definitions in some parts of the framework, including the terms “mental model” and “shared”
Replaced the term “Mental Models of Integration” with “Integration Mindset”
• Patient/caregiver perspective and role
Removed “belief/perceptions” as a separate type of mental model and added it as a component to the two remaining mental model types
Include additional content, particularly in the area of beliefs/perceptions
Developed additional content to include in the framework, primarily on beliefs/perceptions:
• Strategy: “evaluation” added as a knowledge component and four beliefs/perceptions added
• Roles: four beliefs/perceptions added
Edited several terms and definitions in the framework for clarity and comprehensiveness. For example:
Additional framework contents identified through respondent comments and literature.
• Strategy: “services to be integrated” changed to “targets”; “external customers changed to “clients”; and “processes” changed to “methods”
• Strategy: the definition for knowledge of clients was changed from “patients/caregivers who will benefit from integration” to “characteristics and needs of the populations, patients and/or caregivers who will benefit from integration, and the nature of that benefit”
• Roles: “knowledge and skills” changed to “competencies”; “role recognition” changed to “accountabilities”; and “interaction mechanisms” changed to “communication”. Minor modifications were made to the definitions for these terms as well (i.e. “role recognition: the purpose and responsibilities of each professional, organization and client” was replaced with “accountabilities: the activities and results that team members are individually or jointly responsible for”).
Reduce the number of content areas to improve clarity and reduce overlap
Replaced the term “Roles Mental Model” with “Relationships Mental Model”
Reducing the number of content areas may be premature without further research.
Applied minor edits to some terms and definitions. For example:
Further clarify some terms and definitions in the framework
• Strategy: “aptitude for change” changed to “readiness for change”
Several respondents noted that the term “role” focused attention on individual professionals and emphasized boundaries.
Further explain why factors such as culture and leadership are not included in the framework
• Relationships: we originally described the content for “each participating professional” and “each unit”, the latter defined as a program, department or organization. To improve clarity we now describe each content area using inter-professional and inter-organizational “team members” as the referent point.
The next stage of this research aims to develop a measurement tool for capturing and comparing Integration Mindsets as well as practitioner tools.
Develop an assessment tool or checklist to facilitate application
• Relationships: “recognition of shared responsibility: willingness to go beyond what one is obliged to do to support or contribute to the integration process” was replaced with “recognition of shared responsibility: a willingness to share the burden of work and act as a team to contribute to the integration process and/or to the delivery of integrated care”
Updated the three-page overview with an explanation of the range of contextual factors that influence (but are not inherently a part of) Integration Mindsets
Sample respondent comments used to identify beliefs/perceptions content
Sample respondent comments
Strategy Mental Model
Consequences of integrating
• “To be successful each participant must see benefit for him/her as well as the collective”
• “Without willingness to accept and acknowledge the value of integration, it will be difficult to make progress”
Appropriateness of selected strategy
• “Need to include the concept of agreement with the goals, long-term vision and methods”
• “Although there may be shared knowledge unless there is acceptance and commitment there will be challenges”
Integrity of decision-making processes
• “Decisions [must] have clear rationale that can be publicly defended”
• “Important to have all views represented at the decision-making table”
Readiness for change
• “Add a concept about willingness to put energy into finding out, testing hypotheses, innovation”
• “Include individual’s belief about their perceived freedom or ability to make change or execute integration activities”
Relationships Mental Model
Appropriateness of role structure
• “It goes beyond knowledge to understanding, appreciation and agreement on the roles”
• “Knowledge without buy-in is not sufficient”
Identification with the integration initiative
• “Each participant must have enthusiasm and investment in the initiative and must think beyond their current boundaries”
• “Crucial to know why they are there: voluntarily or a directed (forced) integration?”
Recognition of shared responsibility
• “If the involved parties don’t recognize the need for specific organization engagement, the overall integration activity may not gain traction”
• “More emphasis on interdependence, being more collaborative, recognizing that each brings expertise”
Importance of client involvement
• “The key issue is the integration of patient/family into interprofessional teamwork and this is dependent on changing current attitudes about practice and patient involvement”
• “Most important players in planning and implementing integration are the persons and their families”
Round II questionnaire
In the second round, 68 individuals responded (75.6% response rate). Over 95 comments were provided. In general, the ratings and comments were less critical than those provided in round one. Less than 3% of respondents gave a low rating (between 1 and 3 on a 7-point scale) for any of the survey items. The overwhelming majority of responses (85-96%) fell between 5 and 7 on a 7-point scale compared to only 63-82% in the first round, representing an average increase of 16%. The ratings increased substantially for the practical applications of the framework as a whole and for the Strategy Mental Model contents (Figures 1 and 2 respectively). For the Roles Mental Model (later renamed the Relationships Mental Model), the ratings only increased for comprehensiveness while views on clarity remained the same and views on usefulness decreased (Figure 1). This result suggests that there may be a trade-off between comprehensiveness and complexity, and the perceived usefulness of the contents.
The main themes elicited from the review of open comments and the changes made in response to them are summarized in Table 2. The quantitative and qualitative analysis identified the need to further clarify some terms and definitions as well as to reiterate the scope of the framework. Although ratings for comprehensiveness increased by about 20% in round two for both the Strategy and Roles Mental Model, upon analysis of respondent comments, it was apparent that most respondents were judging comprehensiveness based on the framework’s inclusion of influencing factors. Several respondents suggested that leadership, organizational culture, and policy be included in the framework. Although these are important factors that influence the implementation and success of integration initiatives, they describe the context for integration and not knowledge and beliefs regarding integration strategies.
Cross-group agreement by level of integration experience, role, and workplace
Analysis of cross-group agreement revealed some statistically significant group-level differences in responses, primarily in ratings of usefulness of the concepts and framework (Additional file 3). Among the three respondent characteristics tested – level of integration experience, current role, and current workplace – only level of integration experience had a consistent relationship with group-level responses across both rounds of the study. In the first round, respondents with micro-level integration experience provided higher ratings than those with meso- and macro- level experience in select areas, primarily related to usefulness. In the second round, however, a significant difference between groups was detected for only one survey item which examined the usefulness of the framework for managing integration. Those with meso-level integration experience provided a lower mean rating (5.6 on a 7-point scale) for this question than those with micro- and macro-level experience (6.1 on a 7-point scale).
In regards to current role, respondents in a management, administration or consultant role initially provided higher ratings in comparison with other groups, particularly researchers, for the usefulness of the framework for interpreting, planning, implementing, managing, and evaluating integration. However, in the second round there were no significant differences identified in ratings across the four role types, which may suggest an increase in cross-group agreement. While ratings did not vary by respondents’ workplace in round one, they did in round two in relation to two survey items. For these two survey items, which examine usefulness of the “Roles Mental Model” concept and usefulness of the framework for interpreting integration experiences, ratings were generally lower among those working for a coordinating or advisory body, or for the government, than those working in other settings. This may be because individuals working in these settings are farther removed from direct patient care. For example, most of the coordinating and advisory bodies represented in the sample are focused on health system performance measurement and reporting.
In summary, the results suggest that respondents typically agreed on the clarity, comprehensiveness, and importance of the proposed concepts and framework; the key point of contention was in regards to how useful the concepts and framework are in practice and in what ways. Such differences are expected given the diversity of integration foci and methods, and the variety of professional groups and organizations involved in integration. Respondents who were “closer” to patients, such as managers and care providers working in healthcare settings, generally provided higher ratings than researchers, policymakers, or those working for advisory bodies. However, there were fewer differences detected in round two compared with round one, which may indicate increased cross-group agreement.
Overview of the integration mindsets framework
Integration mindsets framework
Mental Model Type
Knowledge (information & awareness)
Beliefs/Perceptions (opinions or internal feelings)
Strategy Mental Model
A conceptualization of what is being integrated and how, why and for whom it is being integrated
• Consequences of integrating
• External clients
• Appropriateness of selected strategy
• Integrity of decision-making processes
• Long-term vision
• Readiness for change
Relationships Mental Model
A conceptualization of the organizations, groups, and individuals (including one’s self) involved in integration and how they are connected
• Appropriateness of role structure
• Identification with the integration initiative
• Recognition of shared responsibility
• Importance of client involvement
Definitions of concepts in the integration mindsets framework
Strategy Mental Model – Knowledge Content
functions, services, organizations and/or systems identified for integration
characteristics and needs of the populations, patients and/or caregivers who will benefit from integration, and the nature of that benefit
primary aims of integration, which may be related to costs, efficiency, productivity, quality of care, patient safety and/or patient outcomes
how the services, programs or functions, organizations and/or systems will “look” or operate when fully integrated
approaches and enablers for achieving integration – which may be clinical, technological, patient or caregiver-centered, administrative, financial, organizational, governance and/or policy-related – and timeline for implementation
key performance dimensions and indicators for assessment of the integration initiative
Strategy Mental Model – Beliefs\Perceptions Content
Consequences of integrating
the expected outcomes (positive and/or negative) of integration for one’s self, for other participating individuals and organizations, for external clients, and for the healthcare system
Appropriateness of selected strategy
the extent of agreement with the selected targets, clients, goals, long-term vision, methods and evaluation approach for an integration initiative
Integrity of decision-making processes
the extent to which decisions regarding integration are made in a manner that is equitable and transparent
Readiness for change
the ability and willingness to implement the desired integration initiative
Relationships Mental Model – Knowledge Content
the knowledge and skill sets of each team member1
how each team member contributes to patient health and well-being
the activities and results that team members are individually or jointly responsible for
how and to what extent the work of each team member depends on or is influenced by another
sources of information and how information flows between team members, including frequency and methods for contact
Relationships Mental Model – Beliefs\Perceptions Content
Appropriateness of role structure
the extent of agreement with the content and distribution of roles, including relative accountabilities and communication methods
Identification with the integration initiative
the extent of self-association with the integration initiative (i.e. the team, partnership, network, etc.) in addition to one’s professional group and organization
Recognition of shared responsibility
a willingness to share the burden of work and act as a team to contribute to the integration process and/or to the delivery of integrated care
Importance of client involvement
the extent to which the involvement of patients and their caregivers is considered necessary and beneficial to integration efforts
The Integration Mindsets Framework aims to identify the cognitive contributors to integration success or failure. The framework outlines important knowledge and beliefs whose convergence or divergence across stakeholder groups may influence inter-professional and inter-organizational relations. Although SMMs are only one of many factors that influence integration, they may help to explain variations in performance, and may be leveraged to accelerate change in tandem with modifications to policy and legislation, organizational structures and context, and administrative and care processes.
A cognitive perspective on integration enhances our understanding of normative integration and facilitates its measurement, in part by complementing a cultural lens. Frameworks and methods used to measure “integrated system culture” provide limited, high-level insights  that are often not actionable, which may explain why integration evaluation methods rarely measure the cultural aspects of integrating . The Integration Mindsets Framework allows us to explore and potentially measure the micro-foundations of culture with a focus on knowledge and beliefs specific to integration as opposed to general cultural attributes. Knowledge is of particular importance for two reasons: (1) the absence of knowledge in frameworks of organizational culture, and (2) knowledge of a change (i.e., what the change is, how it will impact the organization, how it will be implemented, etc.) has been linked to less resistance to change . Questionnaires or interview guides may be developed to collect data on each of the content areas listed in Table 4 from various stakeholders involved in an integration activity, including managers and clinicians from organizations across the continuum of care such as hospitals, long-term care homes and community-based agencies. This type of data will complement the high-level information captured by cultural tools on an organization’s orientation in regards to people, innovation, control and outcomes , and provide a more complete understanding of normative integration in a given context and at a given point in time.
The two-round Delphi process allowed the original framework by Evans and Baker  to be tested against expert opinion. The study design created an opportunity for structured dialogue on integration among a varied group of healthcare stakeholders, and between investigators and participants. The respondents represented diverse policy contexts, professional roles, and organization types, thereby supporting the validity and generalizability of the concept of “Integration Mindsets” and the accompanying framework. Respondents asked questions, offered suggestions, and shared examples from their professional experiences; this rich feedback helped to clarify and further develop the concepts, structure and practical value of the framework while retaining and validating the underlying theoretical foundations. After the second round, there was strong interest among participants in the development of assessment or discussion tools for use in the field as well as vignettes or cases demonstrating use of the framework.
Many of the limitations associated with the Delphi method relate to small sample size and the identification, selection and commitment of the expert panel members . In this study, we used a random sample of experts who attended a national symposium on integrated care, thereby reducing selection bias. Our sample was relatively large (n = 90) and response rates relatively high (52% and 76% respectively) in comparison to most Delphi and survey studies respectively [35, 36]. Although respondents were overwhelmingly female (74%), from the province of Ontario (73%), and in a management position (57%), these distributions reflect the broader population of symposium attendees. While the diversity of the respondents and organizations represented support generalizability of the framework, further research is required to determine generalizability beyond the Canadian context. In addition, patient and caregiver perspectives were not captured because the framework and method of study would require modification to appropriately reflect and elicit their views; however, the framework does conceptualize patients and caregivers as valued members of the healthcare team, and patient advocates did participate in the study (n = 6). Finally, although the Kruskall-Wallis test is robust in detecting differences across groups with small sample sizes (n ≥ 5), the sub-group sample sizes by respondent level of integration, workplace and role were unequal with relatively large ranges (e.g., fifty-three managers versus eleven researchers). The results must therefore be interpreted with caution.
Given the cultural and cognitive challenges to integration [6–9, 40, 41], there is an increasing need for research and practical tools that can enhance our understanding and ability to measure and influence normative integration. The Integration Mindsets Framework offers a starting point for doing so by drawing our attention to how various stakeholders involved in a specific integration initiative think about and conceptualize integration. The cognitive and cross-boundary (i.e., inter-organizational and inter-professional) focus of the framework supports recommendations from the change management literature to engage individuals and groups at all levels in leading and participating in change efforts, to pay attention to change recipients’ emotional, cognitive, and behavioral reactions to change, and to address divergent visions and goals or mis-alignments in the knowledge and beliefs needed for the change [42–44].
We can use the Integration Mindsets Framework to explore: How do mindsets evolve (or not) with implementation? How do views differ among leaders and staff, providers and managers, and providers and patients/caregivers? How similar are mindsets among different organizations involved in integration? These questions may be explored through discussions among team members or partnering organizations using the framework as a guide, or through formal measurement once a measurement tool has been developed for capturing and comparing integration mindsets. Below we offer more specific examples of potential applications of the framework.
Interpretation: Current or past integration efforts may be (re-)interpreted using the framework. For example, despite careful redesign of structures and processes, and a favourable environment, some integration efforts still fail to meet objectives. Differences in integration mindsets may help partly explain such cases.
Planning: The framework can be used to direct and focus early discussions and planning efforts among team members or partnering organizations, and to assess system or organizational readiness for integration.
Implementation: The framework draws our attention to important knowledge content, some of which may be co-created or clarified, recorded, and formally agreed to during the integration implementation stage.
Management: Awareness of the extent to which integration mindsets are shared and where similarities and differences lie can help guide change management interventions. A lack of cross-understanding (possessing an accurate understanding of the mental models of others) negatively impacts collective learning and performance . Managers and leaders can use information on integration mindsets to identify pockets of resistance to change or to leverage the support of champions, to re-frame the initiative, as well as to develop interventions which align with the identified problem [46, 47]. With regards to the latter, for example, a lack of shared knowledge may be addressed through training and education, whereas a lack of shared beliefs may require more extensive and potentially long-term dialogue and negotiation in addition to changes to structures and incentives .
Evaluation: The extent to which integration mindsets are shared may help assess the success and sustainability of an integration activity. As relationships develop and new work practices become embedded over time, actors retrospectively make sense of what they did together and knowledge and beliefs/perceptions become more congruent [48, 49]. “Shared integration mindsets” may be used as one indicator, among many, of a successful and sustainable integration activity.
In addition to improving the Integration Mindsets Framework in preparation for empirical research and practical use, this study also establishes a common language for discourse on cognition in health systems integration and encourages interdisciplinary and cross-level theorizing. Further research is needed to determine the contribution of the Integration Mindsets Framework to research and practice. In particular, future studies should examine the framework’s relevance in various countries and health systems, and develop and test measurement tools for capturing and comparing “integration mindsets” with attention to psychometric properties. An international Delphi panel is an appropriate next step to examine the framework’s generalizability beyond the Canadian context and to further validate the framework prior to its application. Incorporating patient views is also fundamental to the validation process. Complex patients accustomed to interacting with multiple care providers may be briefed on the structures and processes associated with a particular integrated care model, and asked what they think their providers need to agree on – and what shared knowledge and beliefs they require – to work together effectively under that model. This open-ended discussion may be followed by a more focused discussion of the contents of the framework and the extent to which the contents resonate with patients. Future research will also help clarify the nature of the framework’s contents and their relative importance to collaboration. For example, with regards to the “evaluation” component of the Strategy Mental Model, is having a shared awareness of what the metrics are important in and of itself? Or is having a shared belief in the validity of the metrics and individual or collective ability to influence the metrics important? In addition, how might the influence of convergent and divergent integration mindsets differ based on the type and level of integration, the degree of similarity or dissimilarity, the number and diversity of actors involved, and the context? In summary, a better understanding of the evolution and interplay of meanings, interpretations and knowledge about integration across inter-professional and inter-organizational boundaries may help to accelerate progress towards integrated care.
This research was funded by the Vanier Canada Graduate Scholarship Program and the Health System Performance Research Network. An earlier version of this paper was presented at the 29th European Group for Organizational Studies (EGOS) Colloquium in Montreal, Quebec on July 4, 2013.
- Singer S, Burgers J, Friedberg M, Rosenthal M, Leape L, Schneider E: Defining and measuring integrated patient care: promoting the next frontier in health care delivery. Med Care Res Rev. 2011, 68 (1): 112-127. 10.1177/1077558710371485.View ArticlePubMedGoogle Scholar
- Bevan G, Janus K: Why hasn’t integrated health care developed widely in the United States and not at all in England?. J Health Polit Policy Law. 2011, 36 (1): 141-164. 10.1215/03616878-1191135.View ArticlePubMedGoogle Scholar
- Hudson B: Ten years of jointly commissioning health and social care in England. Int J Integr Care. 2011, 11: URN:NBN:NL:UI:10-1-101296Google Scholar
- Jiwani I, Fleury M: Divergent models of integration: the Canadian way. Int J Integr Care. 2011, 11: URN:NBN:NL:UI:10-1-101421Google Scholar
- Curry N, Ham C: Clinical and service integration: the route to improved outcomes. 2010, London: The King’s FundGoogle Scholar
- Kreindler S, Larson B, Wu F, Carluzzo K, Bbemudu J, Struthers A, Van Citters A, Shortell S, Nelson E, Fisher E: Interpretations of integration in early accountable care organizations. Milbank Q. 2012, 90 (3): 457-483. 10.1111/j.1468-0009.2012.00671.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Ling T, Brereton L, Conklin A, Newbould J, Roland M: Barriers and facilitators to integrating care: experiences from the English Integrated Care Pilots. Int J Integr Care. 2012, 12: URN:NBN:NL:UI:10-1-113730Google Scholar
- Pate J, Fischbacher M, Mackinnon J: Health improvement: countervailing pillars of partnership and profession. J Health Organ Manag. 2010, 24 (2): 200-217.View ArticlePubMedGoogle Scholar
- Williams P, Sullivan H: Faces of integration. Int J Integr Care. 2009, 9: [http://www.ijic.org/index.php/ijic/article/view/509/1016]Google Scholar
- Cameron K, Ettington D: The conceptual foundations of organizational culture. Higher education handbook of theory and research. Edited by: Smart J. 1988, New York: Agathon Press, 356-396.Google Scholar
- Friedman L, Goes J: Why integrated health networks have failed. Front Health Serv Manage. 2001, 17 (4): 3-54.PubMedGoogle Scholar
- Suter E, Oelke N, Adair C, Armitage G: Ten key principles for successful health systems integration. Healthc Q. 2009, 13 (Sp): 16-23. 10.12927/hcq.2009.21092.View ArticlePubMedPubMed CentralGoogle Scholar
- Davies H, Nutley S, Mannion R: Organisational culture and quality of health care. Qual Health Care. 2000, 9 (2): 111-119. 10.1136/qhc.9.2.111.View ArticlePubMedPubMed CentralGoogle Scholar
- Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma M, Eccles M: The effectiveness of strategies to change organisational culture to improve healthcare performance: a systematic review. Implementation Sci. 2011, 6 (1): 33-40. 10.1186/1748-5908-6-33.View ArticleGoogle Scholar
- Mohammed S, Ferzandi L, Hamilton K: Metaphor no more: a 15-year review of the team mental model construct. J Air Waste Manage Assoc. 2010, 36 (4): 876-910.Google Scholar
- Fiske S, Taylor S: Social cognition. 1991, New York, NY: McGraw-HillGoogle Scholar
- Cannon-Bowers J, Salas E, Converse S: Shared mental models in expert team decision-making. Current Issues in Individual and Group Decision Making. Edited by: Castellan NJJr. 1993, Hillsdale, NJ: Erlbaum, 221-246.Google Scholar
- Defillippi R, Ornstein S: Psychological perspectives underlying theories of organizational learning. The Blackwell handbook of organizational learning and knowledge management. Edited by: Easterby-Smith M, Lyles M. 2003, Cambridge, MA: Blackwell Publishing Ltd, 19-37.Google Scholar
- Korte R, Chermack T: Changing organizational culture with scenario planning. Futures. 2007, 39 (6): 645-656. 10.1016/j.futures.2006.11.001.View ArticleGoogle Scholar
- Evans JM, Baker GR: Shared mental models of integrated care: aligning multiple stakeholder perspectives. J Health Organ Manag. 2012, 26 (6): 713-736. 10.1108/14777261211276989.View ArticlePubMedGoogle Scholar
- Lau C, Kilbourne L, Woodman R: A shared schema approach to understanding organizational cultural change. Res Organ Change Dev. 2003, 14: 225-256.View ArticleGoogle Scholar
- DeChurch L, Mesmer-Magnus J: The cognitive underpinnings of effective teamwork: a meta-analysis. J Appl Psychol. 2010, 95 (1): 32-53.View ArticlePubMedGoogle Scholar
- DeChurch L, Mesmer-Magnus J: Measuring shared team mental models: a meta-analysis. Group Dyn Theory Res Pract. 2010, 14 (1): 1-14.View ArticleGoogle Scholar
- Okhuysen G, Bechky B: Coordination in organizations: an integrative perspective. Acad Manag Ann. 2009, 3 (1): 463-502. 10.1080/19416520903047533.View ArticleGoogle Scholar
- Kellermanns F, Walter J, Floyd S, Lechner C, Shaw J: To agree or not to agree? A meta-analytic review of strategic consensus and organizational performance. J Bus Res. 2011, 64 (2): 126-133. 10.1016/j.jbusres.2010.02.004.View ArticleGoogle Scholar
- Fiol C: Consensus, diversity, and learning in organizations. Organ Sci. 1994, 5 (3): 403-420. 10.1287/orsc.5.3.403.View ArticleGoogle Scholar
- Best A, Holmes B: Systems thinking, knowledge and action: towards better models and methods. Evid Policy. 2010, 6 (2): 145-149. 10.1332/174426410X502284.View ArticleGoogle Scholar
- Kellermanns F, Walter J, Lechner C, Floyd S: The lack of consensus about strategic consensus: advancing theory and research. J Air Waste Manage Assoc. 2005, 31 (5): 719-737.Google Scholar
- Paroutis S, Pettigrew A: Strategizing in the multi-business firm: strategy teams at multiple levels and over time. Hum Relations. 2007, 60 (1): 99-135. 10.1177/0018726707075285.View ArticleGoogle Scholar
- Hysong S, Best R, Pugh J, Moore F: Not of one mind: mental models of clinical practice guidelines in the veterans health administration. Health Serv Res. 2005, 40 (3): 829-848. 10.1111/j.1475-6773.2005.00387.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Vlaar P, Van de Bosch F, Volberda H: Coping with problems of understanding in interorganizational relationships: using formalization as a means to make sense. Organ Stud. 2006, 27 (11): 1617-1638. 10.1177/0170840606068338.View ArticleGoogle Scholar
- Abrahamson E, Fombrum C: Macrocultures: Determinants and consequences. Acad Manag Rev. 1994, 19 (4): 728-755.Google Scholar
- Phillips M: Industry mind-sets: exploring cultures of two macro-organizational settings. Organ Sci. 1994, 5 (3): 384-402. 10.1287/orsc.5.3.384.View ArticleGoogle Scholar
- Thornton P, Ocasio W: Institutional logics. The SAGE Handbook of Organizational Institutionalism. Edited by: Greenwood R, Oliver C, Sahlin K, Suddaby R. 2008, London: Sage, 99-129.View ArticleGoogle Scholar
- Mullen P: Delphi: myths and reality. J Health Organ Manag. 2003, 17 (1): 37-52. 10.1108/14777260310469319.View ArticlePubMedGoogle Scholar
- Dillman D: Mail and internet surveys: the tailored design method. 2000, New York: John Wiley and Sons, IncGoogle Scholar
- Gillies R, Shortell S, Anderson D, Mitchell J, Morgan K: Conceptualizing and measuring integration: findings from the health systems integration study. Hosp Health Serv Adm. 1993, 38 (4): 467-489.PubMedGoogle Scholar
- Strandberg-Larsen M, Krasnik A: Measurement of integrated health care delivery: a systematic review of methods and future research directions. Int J Integr Care. 2009, 9: [http://www.ijic.org/index.php/ijic/article/view/305/608]Google Scholar
- Washington M, Hacker M: Why change fails: knowledge counts. Leadersh Organ Dev J. 2005, 26 (5): 400-411. 10.1108/01437730510607880.View ArticleGoogle Scholar
- Evans JM, Baker GR, Berta W, Barnsley J: The evolution of integrated health care strategies. Annual Review of Health Care Management: Revisiting the Evolution of Health Systems Organization (Advances in Health Care Management, Volume 15). Edited by: Goes J, Savage G, Friedman L. 2013, Bingley, UK: Emerald Group Publishing Limited, 125-162.Google Scholar
- Hartgerink JM, Cramm JM, Van Wijngaarden JDH, Bakker TJEM, Mackenbach JP, Nieboer AP: A framework for understanding outcomes of integrated care programs for the hospitalised elderly. Int J Integr Care. 2013, 13: [http://www.ijic.org/index.php/ijic/article/view/1063/2287]Google Scholar
- Best A, Greenhalgh T, Lewis S, Saul J, Carroll S, Bitz J: Large-system transformation in health care: a realist review. Milbank Q. 2012, 90 (3): 421-456. 10.1111/j.1468-0009.2012.00670.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Holt D, Vardaman J: Towards a comprehensive understanding of readiness for change: The case for an expanded conceptualization. J Change Manag. 2013, 13 (1): 9-18. 10.1080/14697017.2013.768426.View ArticleGoogle Scholar
- Oreg S, Vakola M, Armenakis A: Change recipients’ reactions to organizational change: A 60-year review of quantitative studies. J Appl Behav Sci. 2011, 47 (4): 461-524. 10.1177/0021886310396550.View ArticleGoogle Scholar
- Huber G, Lewis K: Cross-understanding: Implications for group cognition and performance. Acad Manag Rev. 2010, 35 (1): 6-26.View ArticleGoogle Scholar
- Martin G, Weaver S, Currie G, Finn R, McDonald R: Innovation sustainability in challenging health-care contexts: Embedding clinically led change in routine practice. Health Serv Manag Res. 2012, 25 (4): 190-199. 10.1177/0951484812474246.View ArticleGoogle Scholar
- Waring J, Bishop S: Lean healthcare: Rhetoric, ritual and resistance. Soc Sci Med. 2010, 71 (7): 1332-1340. 10.1016/j.socscimed.2010.06.028.View ArticlePubMedGoogle Scholar
- Weick K: The Social Psychology of Organizing. 1979, New York, NY: McGraw Hill, 2Google Scholar
- Weick K, Sutcliffe K, Obstfeld D: Organizing and the process of sensemaking. Organ Sci. 2005, 16 (4): 409-421. 10.1287/orsc.1050.0133.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/14/222/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.