- Research article
- Open Access
- Open Peer Review
Impact of interprofessional education about psychological and medical comorbidities on practitioners’ knowledge and collaborative practice: mixed method evaluation of a national program
BMC Health Services Researchvolume 16, Article number: 465 (2016)
Many patients with chronic physical illnesses have co-morbid psychological illnesses, which may respond to interprofessional collaborative care. Continuing education programs frequently focus on skills and knowledge relevant for individual illnesses, and unidisciplinary care. This study evaluates the impact of “Mind the Gap”, an Australian interprofessional continuing education program about management of dual illnesses, on practitioners’ knowledge, use of psychological strategies and collaborative practice.
A 6-h module addressing knowledge and skills needed for patients with physical and psychological co-morbid illnesses was delivered to 837 practitioners from mixed health professional backgrounds, through locally-facilitated workshops at 45 Australian sites. We conducted a mixed-methods evaluation, incorporating observation, surveys and network analysis using data collected, before, immediately after, and three months after training.
Six hundred forty-five participants enrolled in the evaluation (58 % GPs, 17 % nurses, 15 % mental health professionals, response rate 76 %). Participants’ knowledge and confidence to manage patients with psychological and physical illnesses improved immediately. Among the subset surveyed at three months (response rate 24 %), referral networks had increased across seven disciplines, improvements in confidence and knowledge were sustained, and doctors, but no other disciplines, reported an increase in use of motivational interviewing (85.9 % to 96.8 %) and mindfulness (58.6 % to 74 %).
Interprofessional workshops had an immediate impact on the stated knowledge and confidence of participants to manage patients with physical and psychological comorbidities, which appears to have been sustained. For some attendees, there was a sustained improvement in the size of their referral networks and their use of some psychological strategies.
Chronic physical illnesses frequently co-exist with psychological illnesses . Chronic illnesses and psychological illnesses can each escalate progression of the other, and each impacts on patients’ self-management capacities. The standard medications for many psychiatric illnesses have metabolic side-effects, just as some medications for chronic illnesses can have psychological side-effects. The general practice consultation may be the best opportunity for many patients with co-morbid psychological and physical illness to be identified and commence treatment . However, incident depression in patients with chronic disease [3, 4], and chronic disease in those with psychological illness are often under-detected in primary care [5, 6].
One of the challenges for clinicians working with co-morbid psychological and physical illnesses is that the evidence base for best practice [7, 8], and continuing education programs, tend to focus on individual illnesses in isolation. In practice, clinicians cannot abstract decisions about treatment of one illness from consideration of the patient’s other illnesses.
Patients with co-morbidities often experience uncoordinated care from a porous network of clinicians and services [9, 10]. Collaborative care models can improve clinical outcomes for patients with a range of chronic diseases and depression [11–13]. A number of Medicare-subsidised items exist in Australia that could support more interdisciplinary care for patients with co-morbid psychological and physical illnesses. Patients with chronic disease can access chronic disease management and team care items  under Medicare, as well as subsidised referrals to some allied health practitioners. Patients with psychological illnesses can be referred for a set number of Medicare-subsidised consultations from mental health professionals in the private sector, under the Better Access initiative . All these policy initiatives rely on health professionals having a suite of management approaches for both physical and psychological illnesses, and the ability to collaborate with one another on a joint plan.
More than two-thirds of treating psychologists in Australia work in private practice . Their connections with GPs have historically been patchy, since patients can self-refer to both disciplines, and there has been no obligation for the professionals to communicate with each other. Even though there has been good take-up of the Better Access Medicare items, these appear to be mainly accessed by people under the age of 45 years, rather than the older age groups more likely to suffer from co-morbid physical illnesses . The most recent review of the Better Access initiative called for more pathways between GPs and psychologists . Interprofessional education (IPE) offers the potential for disciples to learn perspectives shared by one another, and to become better collaborators. Although IPE has proven benefits in developing knowledge and skills in individual practitioners, there are few studies exploring organisational changes or changes in collaborative practice with other health professionals . How to effectively develop and deliver educational programs addressing co-morbidity also receives little attention in the evaluation literature .
This paper aims to contribute to knowledge in both of these areas. We report immediate knowledge, and longer term skills and collaborative practice outcomes of “Mind the Gap”, a national interprofessional educational program for allied health professionals, GPs and mental health professionals, addressing the care of patients with co-morbid psychological and chronic physical illnesses.
Mind the Gap was an advanced learning module which aimed to develop participants’ skills and knowledge to work singly or interprofessionally with patients with co-morbid psychological and physical illnesses. Aims, content and educational strategies are summarised in Table 1. The module was delivered in one six-hour workshop, or two three-hour workshops, facilitated by a local clinician with expertise in psychological care. All facilitators received a facilitator’s guide and a presentation, with speaking notes. The program was delivered through the Medicare Locals, primary care support organisations, which at the time of the study had 61 regional offices across Australia. As this program was funded by the Department of Veterans’ Affairs, one of the referral pathways covered was to the Veterans and Veterans Families Counselling Service (VVCS), an Australian Government funded service providing counselling and support for war and defence service-related mental health conditions .
The theory of instructional design underpinning the Mind the Gap learning module reflects Reigeluth’s Elaboration Theory : that is, that effective teaching about complex topics requires the information to be reconfigured into smaller units of information which are scaffolded in order to accommodate learning and memory limitations. Recognition and management of co-morbid psychological and medical conditions is a complex topic. Most clinicians are likely to have some prior knowledge or pre-conceptions about this topic, which may be called upon to act as scaffolding for deeper learning. The instructional design therefore began with familiar information, drawing on the statistics of co-morbidity, encouraging clinicians to contextualize this in their own clinical experiences. The less familiar material – counselling techniques, referral processes – were then sequenced and arranged around more familiar concepts, such as one’s own referral networks.
The evaluation aimed to (1) assess the effectiveness of information delivery and workshop facilitation; (2) measure change in knowledge and attitudes among participants; and (3) assess the impact of the learning module on provider behaviour, including application of newly acquired skills in clinical practice and on interprofessional collaboration. This was a systems based evaluation examining inputs, processes, outputs and outcomes . Process indicators collected were: satisfaction with the delivery and design of the program. Output indicators were: immediate changes in knowledge, attitudes to interprofessional practice, and confidence to undertake psychological strategies with patients with physical and psychological comorbidities. Outcome indicators (sought at three months after the intervention) prioritised in this evaluation were: sustained application of skill-sets for patients with co-morbid psychological and chronic physical illnesses; changes in organisational practices; and increase in interprofessional collaboration (assessed through social network analysis). Figure 1 presents the logic model of the educational program, used to frame the evaluation.
Participants completed two questionnaires at baseline and three months after the workshop: a 27 item questionnaire addressing confidence, knowledge and skills used in managing patients with co-morbid illness, and attitudes to other professionals (Additional file 1: Figure S1); and a 15-item social network questionnaire exploring the size and structure of the individual’s networks with other health professionals (Additional file 2: Figure S2).
The 27-item questionnaire comprised the Interprofessional Socializing and Valuing Scale (ISVS) , measured on a seven-point Likert scale, and a number of items about participants’ knowledge, experience and confidence managing patients with comorbid illnesses, measured on a five-point Likert scale.
The 15-item social network questionnaire explored ego (personal) networks, the networks centred on individual practitioners . Participants were asked to rate relationships with a total of 15 health professionals identified as being of particular relevance to management of co-morbid chronic disease and psychological illness; they could also nominate up to 6 additional professionals. Additional file 3: Table S1 shows the range of disciplines identified by participants in this study. The survey collected data on three different professional relationships within each personal network: the sharing of information, referral patterns, and working together in other ways. Data were paired using an anonymised unique identifier.
Participants were also asked to complete a 10-item questionnaire immediately after the workshop, addressing confidence or knowledge, and their plans (if any) to change using a commitment to change (CTC)  approach (Additional file 4: Figure S3). CTC statements from participants focused on intended changes as a result of training (maximum of three). The 3 month follow-up survey (Additional file 5: Figure S4) also included some open questions on whether the changes had been achieved and were likely to be continued, and challenges in instituting the change. The degree of difficulty in instituting changes was rated on a five-point Likert Scale.
Four workshops were observed by three evaluators, using a structured observation sheet (total observation, 23 h).
Distribution of surveys
Pre and post questionnaires were distributed to participants at workshops by facilitators. Participating Medicare Locals facilitated distribution and follow-up of questionnaires three months after each workshop. Pre and post data were provided for 41 workshops, and follow up data were provided by the Medicare Locals for participants from 30 workshops. Figure 2 presents the response rates at each sampling interval.
Pre and post workshop data were analysed using SPSS (v20.0, IBM). As a quality assurance measure, data entries from surveys collected from 9 % of the workshops were double-checked by a second evaluator (concordance 96 %). Changes in knowledge, awareness, confidence and attitudes were assessed by treating Likert scales as continuous data, once it had been confirmed that the means were normally distributed. Matched pairs analysis using t-tests was conducted for before-after, and before-follow up groups. Participants did not all use their unique identifying codes on all three surveys, so comparisons were not made across all three groups. Use of different psychological strategies at follow-up was stratified by discipline group, to accommodate a slight over-representation of doctors in the three-month follow-up data. Commitment to change data were compared to pre-workshop and 3 month follow up data using matched samples. Network analysis was performed using UCINET  software, comparing network characteristics before and after the intervention. Social network derived data were also entered into an SPSS-X compatible dataset and analysed using a paired samples analysis where indicated.
Forty-five workshops were held across seven states and territories, with between 10 and 54 attendees (mean, 19) per workshop. Of the 837 workshop participants, 645 enrolled in the evaluation (54 % GPs, 16 % nurses, 14 % mental health professionals, response rate 77 %). Other allied health practitioners who attended were occupational therapists, pharmacists, physiotherapists, podiatrists and social workers. At three-month follow-up, doctors were over-represented (64 %), and mental health workers and nurses relatively under-represented (11 % and 6 % respectively).
Workshops observed in several states were consistent in their delivery of the program as designed, with the same materials being used in all of them. The workshops were generally well received. Of 590 who provided feedback, 75 % felt that the five learning objectives had been fully met.
Improvement in knowledge and confidence
Immediately after the workshop, improvements were reported across all six knowledge and confidence items. The subsample who completed the three month evaluation reported higher mean scores for each item than the overall mean at baseline (Table 2). All these differences are statistically significant, reflecting the large sample size, with changes in knowledge being greater than changes in confidence.
Increase in use of psychological strategies
Mental health professionals and social workers reported nearly 100 % use of the strategies taught at this workshop at baseline, and at three-month follow-up. When analysed by professional group, the group that reported the most significant change in uptake of strategies was GPs, who had significant increases in reported use of mindfulness and motivational interviewing (Table 3).
Changes in clinical practice
Five hundred ten participants provided responses to this question at baseline. Narrative analysis indicated that the following intentions to change were described:
Actively seeking and assessing patients for physical and psychological co-morbidities.
Improved care planning and shared management, including better collaboration with local services
Involving patients and carers more in disease management planning
One hundred fifty-three participants provided usable responses to the follow-up questionnaire on changes to practice. Respondents indicated that changes instituted were the same as planned changes in 76 % (322) of 421 changes. Matched data between post and follow-up workshop questionnaires were available for 130 respondents from 30 workshop sites, reflecting on 369 changes, or which 77 % were achieved. For this group, changes proposed post-workshop and those instituted and reported at follow up were broadly consistent although often varied in detail. Actual changes in practice tended to be more specific and incremental, for example “try to undertake more chronic disease management” became “I now schedule more time with complex patients” or “collaborating better with local GPs” became “including more detail in letters to doctors”. Respondents who had instituted changes suggested that practice changes had been easy or very easy to make in 61 % of cases. In 22 % (62 changes) they were considered difficult to implement. There were no significant differences between participant discipline or type of change across those changes rated as hard and those rated as easy, suggesting that local or individual factors determined the ease of instituting change.
Change in attitude to interprofessional working
At baseline, respondents’ scores indicated they had been in agreement with all statements. In the matched pairs analysis, there was improvement in the three month follow-up subsample (Table 4).
Change in collaborative practice
Five hundred and forty-one participants (65 %) responded to the pre‐workshop social network questionnaire and follow up data were available from 166 respondents from 27 (60 %) workshop locations. Matched data were available for 116 respondents. There were no significant changes in the size of respondents’ professional networks either overall or for each type of relation (information exchange, referral or collaboration) (Table 5). There were also no significant differences in total number of network ties, directionality of information and referral flow, or frequency of interaction within networks, between baseline and follow up.
There was an increase in total number of network connections between respondents and four specific professional disciplines. Significant increases occurred in network ties to exercise physiologists, psychologists, psychiatrists and the Veterans and Veterans Families Counselling Service (VVCS)  (Table 6). Network ties to other counsellors and cardiologists significantly decreased over the same period. Respondents also described a significant increase in mean frequency of interactions with psychologists (+0.074, p = 0.016), the only group for which this was the case.
Outgoing referrals were the most common type of network tie overall, with incoming referrals the least common. This remained constant at baseline and at follow-up (Additional file 6: Table S2). By individual discipline, respondents consistently indicated the strongest type of tie with practice nurses was collaboration, and with pharmacists, seeking information.
The interprofessional workshops had an immediate impact on the stated knowledge and confidence of participants to manage patients with physical and psychological comorbidities. For participants for whom follow-up data was available, there was a sustained improvement in stated knowledge and confidence and in their use of some psychological interventions.
The collaborative practice outcomes were less marked. There was a sustained improvement in attitudes to inter-professional recorded at three months. The increase in actual networking was most marked between doctors and psychologists, perhaps reflecting the fact that these were the two largest participant groups. In this regard, “Mind the Gap” met its objectives of improving attitudes and pathways between some primary care givers of patients with co-morbid medical and psychological illnesses.
There were no overall increases in the size of professional networks for information sharing, referral or other forms of collaboration. Apparent decreases in the use of non-specified counsellors in favour of increased links with specific psychological services such as psychiatrists, psychologists and the Veterans and Veterans Families Counselling Service (VVCS) suggests a shift in practice towards the overt and deliberate prioritisation of mental health supports for this patient group, and a greater awareness of the role and utility of the VVCS, in keeping with the awareness-raising of this service incorporated into the program.
The workshops were ambitious in scope, and occasionally fell short in execution. We attended some workshops where almost all the attendees were doctors, with a few nurses, and others where psychologists were the largest single group. The content of the workshop sometimes struggled to be meaningful for specific health disciplines, rather than for a generic ‘health worker’ audience. Some psychologists expressed concern that the rapid overview of complex psychological interventions might engender in other participants over-confidence about their ability to deliver complex psychological interventions.
Our evaluation found an increase in the use of mindfulness and motivational interviewing, but not other strategies, among participants who were not mental health professionals. Neither of these strategies is new in the general practice therapeutic landscape. Motivational interviewing, in particular, forms the basis of standard general practice guidelines such as those for smoking cessation . In the absence of a control group (unwieldy in the evaluation design for such a large, multi-centred program), it is possible that the increase in stated use may be due to relabelling. Some clinicians may have simply relabelled strategies they already used after recognising their own practices described at the workshop as a named psychological strategy. If this were the case, the pre-workshop low rates of usage may have been an underestimate.
An important limitation to this study is the low numbers at follow-up. Typically, longitudinal surveys used in health education and evaluation have significant attrition in response rates to repeated surveys [31, 32] and our evaluation is no different. The response rate at three months had decreased from 73 % (post-workshop survey) to 47 % of the relevant population. An additional methodological issue may have been respondent burden, which seemed to particularly impact on the social network survey, with 16 % of those who completed the pre-workshop survey and 15 % of those who completed the three month survey failing to complete the social network survey. Social network surveys are recognised as imposing significant respondent burden . In our study the minimum number of items in the social network survey was 75 (15 names, 5 types of relationships), to a maximum of 90. We organised the survey by alters (asking all five questions about each alter, or health discipline) rather than by questions (asking one question at a time for all alters), due to the high number of alters we identified. For written surveys, response rates are higher if the surveys are organised by question rather than alter, probably because the appearance of respondent burden is reduced . We also found that the unique identifier – a combination of middle name and year of birth was irregularly used, limiting our capacity to undertake paired analyses.
Although social network analysis is likely to be a valuable evaluation tool for interprofessional education, our study results suggest that the total number of names generated should be limited and should be arranged by question rather than alter. This holds true for surveys in both paper and web-based form. Additionally, resources should be specifically devoted to enhancing response – for example, through using computer assisted telephone calls, or by placing an incentive on return of survey (such as continuing education recognition).
The second reason for low response rates is the relatively lengthy roll-out of the program, as it was delivered through organisations across Australia. This meant that some of the participants received the training too late in the study period to participate in the formal three-month follow-up survey. We did not find any geographic bias in our study, and although the follow-up data overall includes a slight over representation of medical practitioners, the matched-pairs analysis enables this potential bias to be contained. Modelling for attrition bias in follow-up data suggest that attrition rates up to 60 % are acceptable if the missing cases are missing at random  Our attrition rate is around 69 % (74 % for social network data), and our survey is underpowered to detect subtle changes in social network characteristics. The responses from the three month survey may have been susceptible to respondent bias, in that those who felt most positively about the program may have been more likely to respond to repeated surveys. If this were the case, the level of improvement at three months in this study may be overstated.
Despite these limitations, our evaluation suggests that there were positive outcomes in relation to content knowledge, skills and collaborative networks through the use of inter-professional educational workshops addressing health care for primary care patients with co-morbid illnesses. The evidence for the effectiveness of interprofessional education on collaboration or teamwork has not to date been particularly strong for primary care settings . It is worth noting therefore that particular structural drivers existed in the Australian primary care setting, which increased the likelihood of success of Mind the Gap. The changes that were sought, at least in general practice, were not complex, and could be readily incorporated into existing organisational and clinical work. The person-power to support enhanced case-finding now exists with the widespread uptake of practice nurses into general practice over the last decade . Private health professionals other than GPs had structural incentives to collaborate with GPs, since GPs may refer patients with psychological and medical co-morbidities to private allied health professionals and psychologists for services fundable under the national insurer, Medicare. Mental health professionals in the public sector had an interest in reducing the size of their complex workload through better collaboration. Thus, there were few attendees at the workshops who could not identify immediate benefits to themselves arising from interprofessional collaboration for patients with psychological and medical co-morbidities.
Simply placing health professionals in proximity to one another in an educational session is not sufficient to generate changes in attitude or collaborative practice. If run poorly, it may even cement prejudicial attitudes towards each other , Collaboration is a complex, polyfactorial organisational outcome. Education designers should be cognizant of local referral and collaborative networks, and structural drivers for collaboration, in order to identify and capitalise on effective drivers for change. Mind the Gap was “localised” – delivered by local facilitators, with local interprofessional participants, by the local support organisation - while also being framed by financial and organisational drivers to productive collaboration. A follow-up learning exercise that enabled the continuation of reflective engagement would have helped consolidate gains. Nevertheless, our evaluation suggests that there is likely to be value in educating groups of primary care professionals together on matters that are of direct clinical relevance to them, and for which there are existing structural drivers of collaboration.
Clarke DM, Currie KC. Depression, anxiety and their relationship with chronic disease: a review of the epidemiology, risk and treatment evidence. Med J Aust. 2009;190:54–60.
Ford DE. Optimizing outcomes for patients with depression and chronic medical illnesses. Am J Med. 2008;121 Suppl 2:S38–44.
Ani C, Bazargan M, Hindman D, Bell D, Rodriguez M, Baker RS. Comorbid chronic illness and the diagnosis and treatment of depression in safety net primary care settings. J Am Board Fam Med. 2009;22:123–35.
Ormel J, Van Den Brink W, Koeter MWJ, Giel R, Van Der Meer K, Van De Willige G, et al. Recognition, management and outcome of psychological disorders in primary care: a naturalistic follow-up study. Psychol Med. 1990;20:909–23.
Lunay B, Bywood P. Co-morbidity of mental and physical illness: meeting unmet care needs. Research Roundup Issue 18. Adelaide: Primary Health Care Research & Information Service; 2011.
Whiteford HA. Depression in primary care: expanding the evidence base for diagnosis and treatment. Med J Aust. 2008;188(Suppl):S101–102.
Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294:716–24.
Kruzikas DT, Jiang HJ, Remus D, Barrett ML, Coffey RM, Andrews R. Preventable hospitalizations: a window into primary and preventive care, 2000. Washington: Agency for Healthcare Research and Quality; 2004.
Kathol RG, Butler M, McAlpine DD, Kane RL. Barriers to physical and mental condition integrated service delivery. Psychosom Med. 2010;72:511–8.
Henderson J, Battams S. Mental health and barriers to the achievement of the ‘right to health’. Aust J Primary Health. 2011;17:220–6.
Atlantis E, Fahey P, Foster J. Collaborative care for comorbid depression and diabetes: a systematic review and meta-analysis. BMJ Open. 2014;4:e004706. doi:10.1136/bmjopen-2013-004706.
McGregor M, Lin EH, Katon WJ. TEAMcare: An integrated multicondition collaborative care program for chronic illnesses and depression. J Ambul Care Manage. 2011;34:152–62.
Morgan MAJ, Coates MJ, Dunbar JA, Reddy P, Schlicht K, Fuller J. The TrueBlue model of collaborative care using practice nurses as case managers for depression alongside diabetes or heart disease: a randomised trial. BMJ Open. 2013;3:e002171. doi:10.1136/bmjopen-2012-002171.
Department of Health. Chronic disease management (formerly Enhanced Primary Care or EPC) - GP services. http://www.health.gov.au/internet/main/publishing.nsf/content/mbsprimarycare-chronicdiseasemanagement Updated 2 Apr 2014. Accessed 21 Aug 2016.
Department of Health. Better access to psychiatrists, psychologists and general practitioners through the MBS (Better Access) initiative. http://www.health.gov.au/mentalhealth-betteraccess. Updated 16 Dec 2015. Accessed 21 Aug 2016.
Health Workforce Australia. Australia’s Health Workforce Series – psychologists in focus HWA: Adelaide; 2014. http://pandora.nla.gov.au/pan/133228/20150419-0017/www.hwa.gov.au/sites/default/files/HWA_Australia-Health-Workforce-Series_Psychologists%20in%20focus_vF_LR.pdf Accessed 21 Aug 2016.
Australian Bureau of Statistics. Characteristics of people using mental health services and prescription medication, 2011. Cat No. 4329.0. ABS: Canberra. Released 28 Oct 2014. http://www.abs.gov.au/ausstats/abs@.nsf/productsbytopic/39A5EDBC47806B28CA257D7E000F488B Accessed 21 Aug 2016.
King D, Tan Y, Wainer J, Smith L, Fitzpatrick D, Sun L, et al. Evaluation of the Better Access initiative component C: analysis of the allied mental health workforce supply and distribution, 2010. http://www.health.gov.au/internet/publications/publishing.nsf/Content/mental-ba-eval-c-toc Updated Sep 2010. Accessed 21 Aug 2016.
Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2013;3:CD002213. doi:10.1002/14651858.CD002213.pub3.
Schoenborn NL, Boyd C, Cayea D, Nakamura K, Xue QL, Ray A, et al. Incorporating prognosis in the care of older adults with multimorbidity: description and evaluation of a novel curriculum. BMC Med Educ. 2015;15:215.
Veterans and Veterans Families Counselling Service. About VVCS. http://www.vvcs.gov.au/AboutVVCS/index.htm Accessed 21 Aug 2016.
Reigeluth C. Elaborating the elaboration theory. Educ Technol Res Dev. 1992;40:80–6.
Bushnell DS. Input, process, output: A model for evaluating training. Train Dev J. 1990;44:41–3.
King G, Shaw L, Orchard C, Miller S. The interprofessional socialization and valuing scale: a tool for evaluating the shift toward collaborative care approaches in health care settings. Work. 2010;35:77–85.
Hanneman RA, Riddle M. Introduction to social network analysis. Riverside CA: University of California; 2005. Available online: http://www.faculty.ucr.edu/~hanneman/nettext/. Accessed 21 Aug 2016.
Shershneva MC, Wang M, Lindeman GC, Savoy JN, Olson CA. Commitment to practice change: an evaluator’s perspective. Eval Health Prof. 2010;33:256–75.
Borgatti SP, Everett MG, Freeman LC. Ucinet for Windows: software for social network analysis. Harvard: Analytic Technologies; 2002.
Gloster AT, Rhoades HM, Novy D, Klotsche J, Senior A, Kunik M, et al. Psychometric properties of the Depression Anxiety and Stress Scale-21 in older primary care patients. J Affect Disord. 2008;110:248–59.
Sheik JI, Yesavage JA. Geriatric Depression Scale (GDS): recent evidence and development of a shorter version. In: Brink TL, editor. Clinical gerontology: a guide to assessment and intervention. New York: Haworth Press; 1986.
Royal Australian College of General Practitioners. Supporting smoking cessation: a guide for health professionals. Melbourne: The Royal Australian College of General Practitioners; 2011.
Green L. Evaluation and measurement: some dilemmas for health education. Am J Publ Health. 1977;67:155–61.
Kristman V, Manno M, Cote P. Loss to follow-up in cohort studies: how much is too much? Eur J Epidemiol. 2004;19:751–60.
McCarty C, Killworth PD, Rennell J. Impact of methods for reducing respondent burden on personal network structural measures. Social Networks. 2007;29:300–15.
Coromina L, Coenders G. Reliability and validity of egocentered network data.Collected via web: a meta-analysis of multilevel multitrait multimethod studies. Social Networks. 2006;28(3):209–31.
Zwarenstein M, Reeves S, Perrier L. Effectiveness of pre-licensure interprofessional education and post-licensure collaborative interventions. J Interprof Care. 2005;19 Suppl 1:148–65.
Phillips C, Pearce CM, Hall S, Kljakovic M, Sibbald B, Dwan K, Porrit J. Enhancing care, improving quality: the six roles of the general practice nurse. Med J Aust. 2009;191:92–7.
Carpenter J, Dickinson H. Understanding interprofessional education as an intergroup encounter: the use of contact theory in programme planning. J Interprof Care. 2016. doi:10.3109/13561820.2015.1070134.
This project was funded by the Australian Medicare Local Alliance, through a grant from the Department of Veterans Affairs.
Availability of data and materials
The datasets supporting the conclusions of this article are available in the Open Science Framework repository, DOI 10.17605/OSF.IO/JKTM7 | ARK c7605/osf.io/jktm7; https://osf.io/jktm7/.
CP and SH designed the study; MI oversighted quality assurance and data acquisition; CP, MI and SH analysed the data; SH performed the social network analysis; SH and CP interpreted the results, CP drafted the paper. All authors reviewed and approve final version. The Mind the Gap Advanced Learning Module (ALM) was developed by the Australian General Practice Network.
The authors declare that they have no competing interest.
Consent for publication
Ethics approval and consent to participate
This study was approved by the Australian National University Human Research Ethics Committee, #2011/546. All participants provided informed consent to participate in this evaluation.
Pre-workshop Questionnaire, Mind the Gap Program Evaluation. Questionnaire delivered before the workshop. (PDF 79 kb)
Professional Network Questionnaire, Mind the Gap Program Evaluation. Questionnaire delivered before the workshop, and three months after the workshop (PDF 85 kb)
Disciplines nominated as members of respondents’ professional networks. (DOCX 14 kb)
Post-workshop Questionnaire, Mind the Gap Program Evaluation. Questionnaire delivered at the conclusion of the Mind the Gap workshop. (PDF 81 kb)
Follow-up Questionnaire, Mind the Gap Program Evaluation. Questionnaire administered three months after completing Mind the Gap. (PDF 111 kb)
Changes in size and characteristics of health providers’ networks (matched pairs analysis, n = 116). (DOCX 14 kb)