- Research article
- Open Access
- Open Peer Review
Process of care in outpatient Integrative healthcare facilities: a systematic review of clinical trials
BMC Health Services Researchvolume 15, Article number: 322 (2015)
Patients currently integrate complementary medicine (CM) and allopathic, choosing a combination of therapies rather than a single therapy in isolation. Understanding integrative healthcare (IHC) extends beyond evaluation of specific therapies to encompass evaluations of multidisciplinary complex interventions. IHC is defined as a therapeutic strategy integrating conventional and complementary medical practices and practitioners in a shared care setting to administer an individualized treatment plan. We sought to review the outcomes of recent clinical trials, explore the design of the interventions and to discuss the methodological approaches and issues that arise when investigating a complex mix of interventions in order to guide future research.
Five databases were searched from inception to 30 March 2013. We included randomized and quasi-experimental clinical trials of IHC. Data elements covering process of care (initial assessment, treatment planning and review, means for integration) were extracted.
Six thousand two hundred fifty six papers were screened, 5772 were excluded and 484 full text articles retrieved. Five studies met the inclusion criteria. There are few experimental studies of IHC. Of the five studies conducted, four were in people with lower back pain. The positive findings of these studies indicate that it is feasible to conduct a rigorous clinical trial of an integrative intervention involving allopathic and CM treatment. Further, such interventions may improve patient outcomes.
The trials in our review provide a small yet critical base from which to refine and develop larger studies. Future studies need to be adequately powered to address efficacy, safety and include data on cost effectiveness.
Systematic review of clinical trials of integrative healthcare
You think that because you understand “one” that you must therefore understand “two” because one and one make two. But you forget that you must also understand “and”. —Sufi teaching story 
Patients are integrating complementary medicine (CM) and allopathic medicine in a variety of ways depending on their health status and health beliefs [2, 3]. They frequently use “bundles” of therapies rather than just one therapy in isolation [4, 5]. Typically the unit of integration in clinical environments is the patient rather than the practitioners [6, 7]. Increasingly consumers are requesting that there be improved communication and coordination between their allopathic care and CM providers . In response to this consumer demand integrative healthcare (IHC), (also referred to as integrative medicine), has emerged over the past two decades [9, 10]. For healthcare policy makers, care providers, practitioners and the consumer, we need to know whether IHC is effective and, if so, what structures and processes combine to exert the positive outcome. Early experimental efforts have been made to investigate the effectiveness and efficacy of this evolving IHC model of patient care [11–13]. We sought to review the outcomes of recent clinical trials, explore the structure of the interventions and also to discuss the methodological approaches and issues that arise when investigating a complex mix of interventions in order to guide future research.
The varied definitions for IHC reflect differing interpretations and implementation [11, 14–16]. There is a common general underlying philosophy that IHC aims to treat the whole person (physical, emotional, energetic, spiritual), using the body’s innate ability to heal itself with a blend of conventional and complementary therapies [11, 14]. However, definitions of IHC splinter off when the structure and process of care are considered. For example, IHC may refer to the process of patient care where allopathic and CM clinicians work as a team. The team may operate in a multi-disciplinary or inter-disciplinary way. There may be a democratic referral system or the allopathic physician may be the gatekeeper and the CM practitioner is adjunct practitioner [17–19]. In other cases, ‘integrative’ may refer to a clinician who ‘integrates’ both allopathic and CM. Further confusion is added to the typology of this field by regional variations, where “integrated” is used in the United Kingdom (UK) and parts of Europe, in the same way in which “integrative” is used in the United States (US) and Australia.
The focus of this paper is IHC defined as a patient-centred, inter-disciplinary approach where there is a combination of conventional medicine with CM with shared patient assessment, treatment planning, review and/or shared practice guidelines that are constructed and utilised during the care process. This therapeutic strategy enables each practitioner, often in conjunction with the patient, to contribute their knowledge and expertise towards providing individualized healthcare plans.
In this emerging field, there are no ‘right’ or ‘wrong’ ways for patients to experience a combination of CM and allopathic medicine. Some conditions or people may be better suited to different processes of care. However, it is important to distinguish clearly between approaches which are adjunctive or complementary therapy and not integrative as they are different entities, have different organisational and resource implications, and likely, different benefits [14, 20]. Adjunctive or complementary is where a therapy is used in addition to allopathic medicine but not involving any shared assessment, management or review in the process of care.
Research methodology to evaluate IHC has tended to use observational designs, pre and post-test, or descriptive methods, but more recently there has been a move to the use of experimental design [21, 22]. IHC is a ‘complex intervention’, using experimental design with a complex intervention is challenging but feasible . By ‘complex intervention’ we mean that there are many active components, which may combine to provide outcomes greater than a sum of its individual parts [11, 24]; that may involve complex mechanisms for delivery; may be difficult to replicate (tailored to individual); and may be influenced by the environment and social context. As such it is extremely challenging to clearly articulate mechanisms of action [23, 25]. IHC is often highly individualized, taking into consideration all aspects of a person not only one condition or symptom, has varied participant responses, with different practitioners in different settings [26, 27]. Determining the outcomes of a complex intervention requires a combination of qualitative and quantitative methods  and a consideration of the different experimental designs available . In addition to measuring outcomes, understanding the complex intervention may be aided by including a nested process evaluation or similar. Process evaluations within trials explore the implementation, receipt, and setting of an intervention and help in the interpretation of the outcome results .
Hence, our review has three primary objectives:
To systematically review the quality and outcomes of clinical trials of IHC
To explore the design of IHC interventions, including process of care
To review research methodology employed in IHC clinical trials
Eligible CM therapies were defined according to the National Institute of Complementary Medicine (NICM) . NICM has adopted, with revisions, the four domains of CM articulated by the US National Centre for Complementary and Integrative Health (NCCIH). These domains are mind-body medicine, biologically-based practices, manipulative and body-based practices and energy medicine . We included any clinical trial or protocol that was conducted in an IHC setting where there was a combination of at least one biomedical practitioner and one CM practitioner; and included an element of shared care or co-management of the patient at the stage of initial assessment, treatment plan development, case management or review. Clinical trials of multidisciplinary care where there was no element of co-management or patient sharing were excluded. Similarly, we excluded trials where the CM treatment was adjunctive and there was no shared management. Trials in integrative oncology were not examined in this review as the process of care in oncology settings differs from primary care settings, where the lead therapy is allopathic medicine and the lead clinician is the oncologist. Those conducted in an in-patient hospital setting were also excluded. Studies were also excluded which were not transferrable to a Western setting due to the different infrastructures in non-Western settings (e.g. Chinese medicine in China).
We searched PubMed (Medline), EMBASE, CINAHL, Clinical Trials Register and the Cochrane Library from inception to 30 March 2013. The following search terms were used, in various combinations: ‘integrated or integrative medicine’ or ‘healthcare’; ‘multidisciplinary care’; ‘complementary’; ‘alternative’; ‘biomedicine’; ‘conventional’; ‘mainstream’ with ‘medicine’; ‘healthcare’; ‘approach’; ‘therapies’. We also hand searched the citations of relevant papers. We included clinical trials which were randomized, non-randomized or case–control. For the purpose of this study we also included protocols for clinical trials, as we were not only focused on outcomes but on the structure and design of the integrative therapeutic intervention. Our search terms were [(Integrat* OR interdisciplinary) and (medicine OR alternat* medicine OR alternat* therap* OR complementary medicine OR healthcare)]; OR [(allopathic OR conventional OR mainstream OR orthodox OR biomedicine) AND (alternat* medicine OR alternat* therap* OR complementary medicine OR intergrat*].
To explore the structure and design of the IHC intervention we applied the structure-process-outcome model . Structure was defined as the environment in which healthcare is provided, the process as the method by which healthcare is delivered and outcome as the consequence of the healthcare provided. We were particularly interested in the process component of this model. We defined process as the way in which the healthcare is delivered – triage, diagnosis, treatment plan, and review. Central to this process are the means for collaboration to foster the “integrative” nature of the intervention. These concepts are explained below:
‘Triage’ in this context refers to how a patient is ‘allocated’ into an IHC intervention or in a clinical context to a practitioner for initial assessment. In clinical practice the gatekeeper for this process may be the receptionist, the practice nurse, or general practitioner (GP) or may come from an external referral.
‘Diagnosis’ refers to initial assessment where baseline data is collected in a clinical trial or where an IHC assessment is undertaken in clinical practice.
‘Treatment plan’ refers to how the plan is derived, who is consulted, how it is agreed, types of treatments, duration, patient preferences and arriving at responsibility for the patient’s journey through the IHC process.
‘Review’ refers to measuring outcomes at set time points, tracking patient compliance,
‘Means for collaboration’: this may include meetings, shared charting, electronic medical records (EMR), corridor conversation, shared education and training, case conferences.
To review the research methodology, we considered the Medical Research Council (MRC) guidelines  for evaluating complex interventions and extracted data on the following:
Was there a theoretical basis for the intervention and trial design?
ave the authors sufficiently described their intervention and the control group?
How was the intervention evaluated ?
What were the context and the environment of the intervention?
How was implementation of the intervention assessed?
What was the rationale for the duration of the intervention?
Two investigators (SG and JF) independently screened all identified titles and abstracts. Full text reports were retrieved. The final decision for inclusion in the review was made by two investigators (SG and JF). The quality of the trials was also reviewed using risk of bias criteria . Each of the retrieved papers was read in its entirety. A standardized data extraction form was developed to extract data according to the set elements. Where data was missing we attempted to contact authors of the study. Data elements were then extracted and entered into a matrix according to the method of Garrard . This approach provided a systematic and concise organisation of the literature.
A total of 6256 papers were screened. Of these 5772 were excluded based on the title and/or abstract. The main reasons for exclusion were not relevant (n = 4865), study of CM usage or attitudes to CM (n = 304), and conceptual paper or commentary (n = 258). A total of 484 full text articles were retrieved for consideration. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram (Fig. 1) shows details of the study selection. Papers were excluded where the intervention or clinical practice was a set multi-disciplinary program or CM was adjunctive not integrative and there was no shared management (n = 49), the article was about CM usage (n = 296), the setting was oncology or inpatient hospital based (n = 149), the paper was conceptual or commentary (n = 117), examples of IHC clinical practice (n = 63), or limited to a discussion of models of IHC (n = 28). Five clinical trials met the inclusion criteria. We excluded the six clinical trials that had been included in the previous review  as they did not meet our criteria of integrative management or were hospital-based programs.
Study design and setting
Four randomized controlled trials (RCT) and one case control study met the inclusion criteria of our study. Table 1 outlines the study settings and trial designs. Three studies were conducted in the US, one in the UK  and one in Sweden . There was one three arm trial comparing, IHC with chiropractic and medical care, and with conventional medical care alone . Two were pragmatic trials comparing individualised, integrative care with usual care [36, 38, 39]. The study in the UK was quasi-experimental design with a wait-list control group .
Settings varied; three were family or primary care medical centres and the others were conducted within an academic setting (Northwestern Health Sciences University) or out-patient hospital setting. In the Sundberg  study, an IHC clinic was set up within a primary care unit in Sweden. Patients were treated within this clinic or in some cases treatments were provided off-site in the private clinics of participating providers. Participants were required to pay five Euros per treatment for the first six treatments, after which no further payment for CM would be required. In the Richardson study , patients were treated within the context of a CM outpatient service operating in line with other National Health Service (NHS) outpatient provisions in the UK. In all other studies treatments were free of charge.
Four of five studies were conducted in people with lower back pain (LBP), with one study treating neck pain in addition to LBP . The fifth study was a case control study that investigated the treatment of referred patients with over 20 different conditions, although the majority of the patients treated presented with neck/back pain . All studies were conducted in adults and enrolled participants with diverse backgrounds and co-morbidities. A total of 751 people were involved in the studies and sample sizes ranged from 20 participants in a pilot study to 330 adults in the case control study.
Process of care: Triage and diagnosis
Four of the five studies published at least one preliminary paper outlining the background of the study , the protocol [7, 37], the clinical care pathway  or the model  to be used in the clinical trial. In Table 2, the initial assessment process is outlined. In two studies, the initial assessment was conducted by a single allopathic physician [36, 40]. In the other three studies, one study used an allopathic physician and a CM practitioner together ; another used assessments conducted by a combination of either a chiropractor, an osteopath or a medical doctor . The initial assessment was reported in detail in two of the studies [37, 39].
In the Maiers’  study, an initial baseline profile was compiled by research study staff. This evaluation included informed consent, self-report questionnaire, health history, physical examination and x-rays if indicated. The self-report questionnaire includes patient perspectives such as previous experience with treatment, preferences and expectations. This profile was reviewed by the IHC team and a second visit scheduled and then assessed by the multidisciplinary team for eligibility  If eligible the patient then completed a second baseline visit, was enrolled and randomized to a treatment group.
The Goertz  study similarly used research study staff to conduct an initial baseline evaluation to gather back pain and chiropractic treatment history, beliefs about chiropractic, health assessments and a treatment expectations questionnaire. Once eligibility of the patient was confirmed by all physicians, a second baseline visit conducted by a medical or osteopathic physician with a focus on age-related concerns and the participant is then randomised to conventional care, dual care or shared care.
In the Sundberg  study, after an initial consultation with a GP at the primary care unit, a conventional treatment plan was developed for every participant. The research team randomised participants to receive either integrative care or continue with conventional care only.
In the Richardson  quasi-experimental study, referrals were accepted to the study from general practitioners (GPs) and hospital doctors and patients. In the Eisenberg  study, participants were initially assessed by both a medical doctor and a CAM clinician.
Process of care: treatment plan
Treatment plan refers to how the plan is derived, who is consulted, how it is agreed, types of treatments, duration, patient preferences and arriving at responsibility for the patient’s journey through the IHC process. Table 2 provides some detail on how the treatment plans were derived. Four of the studies used a team approach to developing one or more treatment plans for each patient [36–39]. In the team approach, typically non-hierarchical, consensus based decision making was sought. None of the case conferences or team meetings to develop treatment plans involved patient participation.
Patient preferences were considered in two studies. In one of these studies, the outcome of the team approach was typically three treatment plan options that were then presented to the patient by a non-clinician case manager. The patient was then able to express their treatment plan preferences . In the other study, a treatment plan was communicated to the participant and participant feedback was incorporated into the management plan .
Oversight for the patient’s journey in the trial was not always clearly documented. In one study it was clearly the general practitioner (GP) , as only medical doctors in Sweden are able to access the full range of medical services. In the UK study, a nurse coordinated the patient’s journey through the study .
The type of therapies and combinations were comprehensively documented in two of the three fully reported studies [41, 42]. In the Maier’s study, manual therapy, exercise and education were the most commonly recommended and selected modality combination followed by a combination of exercise therapy, education and oriental medicine. An average of 3–4 (3.4) treatment plans were offered to each of the 101 participants, with an average of three modalities per plan. Sundberg et al. documented that patients received around seven treatments, averaging two different modalities over 10 weeks along with one session of a self-help activity – qigong. Swedish massage was the most commonly combined modality.
All study interventions involved one to two treatment sessions per week with administration of up to two treatment modalities per session. The interventions included combinations of different practitioners and therapies that were deemed appropriate to the condition being treated. The four studies on LBP typically utilized differing combinations of acupuncture, osteopathy, chiropractic, massage, mind body techniques and/or exercise. The case control study used acupuncture, homeopathy and osteopathy as therapies.
The study comparator was usual care in three studies [36–38]. The case control study used a waitlist for comparison, whilst another study compared the IHC intervention to chiropractic treatment alone .
The length of the intervention was 10–12 weeks for all five studies and none of the studies documented any data collection of co-morbidities or treatments for co-morbidities. Only one study provided a rationale for the length of the intervention, hypothesizing that a 12 week intervention period was typical in treating the study population . One study had a follow up at 16 weeks  and another study had a follow-up period of 52 weeks .
Process of care: Review
All studies had a review process built into the intervention, in some cases a formal process was in place for review and in others cases were only discussed at weekly staff meetings and informally between practitioners . Re-evaluations were common and of the 101 treatment plans, 38 were re-evaluated during the course of the study. The main reasons for re-evaluation were due to lack of improvement as perceived by the provider, patient or both. Maiers et al. used a modified Measure Yourself Medical Outcome Profile (MYMOP) to assess and modify treatment during the intervention period. This study also used benchmarks for improvement in the condition and the case was returned to the team if benchmarks weren’t met .
Process of care: Means for integration and collaboration
Educational sessions or training of therapists and practitioners was conducted in all of the studies prior to commencement [35, 37, 38, 41, 42]. This ranged from a half day workshop  to a six month course in inter-professional education . In another study, the team met weekly for one full day for 14 weeks to undergo training .
Training typically comprised knowledge sharing of the different modalities to be used in the study, often involving presentations by each of the practitioners and in some cases hands on experiential sessions. Evidence for modalities was also reviewed in these sessions. Four studies referred to sharing patient medical records and treatment notes. In some cases specific computer modules or systems had been constructed specifically for the study.
Team meetings, either face-to-face or via phone, were also seen as a regular part of the integrative approach. It was implied in four of the studies that the treatment plan may be altered as a result of these meetings but this process was only clearly detailed in two of the studies [38, 39]. In one study, patient progress was reviewed using outcome measures being assessed against benchmarks and if progress was not satisfactory the treatment plan was modifies.
Four of the studies [36, 37, 39, 40] referred to using an existing evidence base to guide treatment. Evidence for effectiveness was typically tempered with clinical experience and patient presentation. In some cases, the evidence directed the modalities that were selected for inclusion [37, 39] or guided the referral process . A care pathway was designed for one study. This provided a structured process for clinicians to follow including points along the way to flag treatment review.
Outcome measures and results
Only one of the studies was powered to seek efficacy outcomes  and three trials were pilot studies [36–38]. The objective of the case control study was to establish the feasibility of an IHC service as opposed to determining the efficacy of the service.
The results of the adequately powered study showed that IHC in people with LBP resulted in statistically significant improvements in pain reduction, perceived global improvement and satisfaction with care compared to those patients receiving chiropractic care alone [43, 44]. Quality of life was rated as significantly better in the IHC group compared to the chiropractic group at 52 weeks. The feasibility and pilot studies all reported favourable trends towards the IHC group.
All but one study also used a standardized quality of life instrument. Only one study  used a patient generated measure, a modified MYMOP . Generalizable cellular or molecular markers such as inflammation markers were not collected in any study.
Practitioner and/or patient expectations and perceptions of the integrative care approach were reviewed through semi-structured interviews in four of the studies [35, 37, 39, 42]. Patients found the holistic approach of integrative care to be empowering, facilitating self-help in contrast to the conventional managements which was disease focused and reductionist, but were concerned about the challenges of treatment costs . Two other studies are yet to fully publish their qualitative results. The Goertz  study has conducted semi-structured interview to ascertain participant’s perceptions of back pain improvement, clinical trial experience, expectations, treatment effects, and interprofessional collaboration amongst the treating clinicians. Clinician expectations of care were also recorded along with their thoughts on expected improvement of overall quality of life and health status and their rationale for the treatment provided.
Crude cost data was collected in one study only . Extra costs of treatment were calculated at 365€. In this study there was a trend to decreased prescription and non-prescription medications. The cost of the treatment only was collected in one other study but has not been reported .
One minor adverse event was reported out of 170 IHC interventions provided in one study .
Quality of trials
Four of the five studies were randomized controlled trials. Table 3 provides a brief assessment of the risk of bias of the studies. All trials used appropriate methods of randomization (computer generated) and reported allocation concealment. Blinding of care provider and participants is not possible in an IHC intervention, although in some cases it is possible to blind the outcome assessment. Only one trial reported blinding of outcome assessors where questionnaires were administered by research staff not involved in clinical care. The risk of bias of the RCTs is low if the difficulty of blinding is considered. Blinding of participants and care providers in a trial where treatment is individualised is not possible. However, outcome assessment may have been blinded, one study achieved partial blinding of outcome assessment .
The case controlled study had no randomization . Patients were treated on a first come, first serve basis and urgent cases were given priority. To address the bias inherent in such a design the authors followed a quasi-experimental approach for non-equivalent groups without controlled selection. Sensitivity analysis was conducted on the urgent referrals, and the poor response in the waitlist group. This waitlist control study has limitations due to expectation bias, and selection bias. There are also limitations due to the complexity of the intervention and evaluation of the effect applies to the whole treatment and not to the individual components.
Review of the research methodology
All studies provided an adequate rationale for the intervention and trial design [36–40]. The Richardson  quasi-experimental study was based on a pragmatic approach and used the convenience of a ready-made waiting list. The rationale for all of the studies was primarily 2-fold: patients are currently integrating various CM; and the value of this integration remains largely unknown in terms of outcomes and cost effectiveness.
Table 1 provides a summary of the interventions and comparators used in the studies. The therapies available and the comparators are adequately described but the dosage is not always apparent. In only two of the studies have the authors sufficiently described the number of modalities and frequency of sessions of the intervention on average for the participants [36, 41].
Outcome measures detailed in Table 1 show all studies sought to evaluate the interventions using a quantitative measure for quality of life such as SF36 and a symptom specific tool. All but one study  used interviews to explore participant perceptions of the IHC model and two studies also included interviews with providers [37, 43]. The implementation of the intervention was assessed through provider interviews in two of the studies. No other process evaluation methodology was reported.
The rationale for the duration of the intervention was not always apparent in the reporting of the studies. One study provided the following rationale: ‘a 12 week intervention period was perceived by study clinicians and investigators to be typical when treating this population’ .
The small number of IHC trials found in this review was disproportionate to the amount of literature identified. This supports the findings of Coulter et al.  which concluded that research is still focused on conceptualizing and practicing IHC not on efficacy . The paucity of trials also likely reflects the difficulty in undertaking research on complex interventions such as IHC. However, there is a need for research that evaluates this integration, reflects clinical practice and provides an understanding of efficacy [4, 47]. Reviews or summaries of this evolving field are sparse and few have focused specifically on clinical trials [19, 48]. It is perhaps timely that we examine the reporting, structure and outcomes of recent trials within the context of emerging frameworks for researching complex interventions so as to guide future work in this field.
The theoretical rationale reported for undertaking a clinical trial of an IHC intervention was manifold. Firstly, in ‘real life’, patients frequently don’t restrict their CM use to one therapy and often seek to use a combination . Patients are usually the point at which integration occurs - ferrying results, treatment plans and x-rays between providers. It is argued that the most appropriate point of integration is the provider through IHC . Secondly, there has been much discussion about conceptual IHC models but little testing to examine how IHC may be implemented, at what cost and effectiveness . Thirdly, the combined effect or synergy of multiple interventions is hypothesised as likely to be greater than that which can be achieved by a single modality, particularly for chronic conditions . Provision of an individualised multidisciplinary intervention provides patients with greater choice, and facilitates patient participation in the decision-which is associated with better health outcomes and satisfaction [50, 51].
Not all health conditions or diseases are suited to an IHC intervention. In the trials examined in this paper, lower back pain was the condition selected for four or five trials. It is likely that this was selected for a number of reasons. Lower back pain is a prevalent condition which has a high community and economic cost . Musculoskeletal conditions are one of the most common conditions for which people consult a CM practitioner utilizing both conventional and CM treatments alongside one another [53, 54]. There was a clear intent within the studies to include modalities for which a positive evidence base was available. A recent systematic review and meta-analysis of the efficacy, cost-effectiveness and safety of selected CMs for neck and LBP found that CM treatments such as acupuncture, massage, spinal manipulation and mobilization, were more effective at reducing pain in the short term than no treatment, placebo, physical therapy or usual care . Trials typically examined single CM modalities in isolation, and multidisciplinary approaches rarely included CM therapies. Other types of conditions where IHC interventions might be useful are likely to be those where CM use is high, such as the for the treatment of health problems where there is unmet need and conventional care has not been able to help; that have an unpredictable course and prognosis; and that are associated with substantial pain, discomfort, or side effects from prescription drug medicine [4, 55]. In a recent paper, IHC leaders nominated headache, back pain, arthritis and diseases across the metabolic spectrum as areas where the body of CM evidence was strong and suited an IHC trial . In another study of general practice, conditions such as musculoskeletal problems, depression, eczema, chronic pain, and irritable bowel syndrome were suggested .
To understand the IHC interventions – what works and what doesn’t – we need as much transparency as possible. In the studies we reviewed development plans or protocols documented a consistent structure, process and functional intent around the variable components of IHC intervention. The completed trials in the review provided several papers reporting the results of various aspects of the intervention. In the majority of the studies this was sufficient detailed to enable the replication of elements of the design, although a clear parallel evaluation of the process would be of benefit. Only one study in our review included a nested ‘process evaluation’. Process evaluations within trials explore the implementation, receipt, and setting of an intervention and help in the interpretation of the outcome . A process evaluation embedded within future trials may assist in documenting the time taken to construct a patient profile; the skills needed to collect the information for the patient profile; the time needed to devise a treatment plan, present the plan and reach consensus; help to distinguish between essential and non-essential components of the intervention; investigate contextual factors that affect an intervention; patient responsiveness; practitioner delivery; and monitor dose [23, 59, 60].
Equally useful to understanding the IHC interventions is to know what ‘guides’ the treatment planning and management process beyond the integrative care and management. Table 4 includes a set of ‘guiding principles’ articulated by Maiers et al.. These principles serve to clearly delineate the intent or function of the intervention, and the approach the team should aim to take. Of these five principles, all studies were guided by Principles 4, 5 and 6. It was less clear the extent to which studies were guided by the other three principles.
The basic approach to the provision of care in each of the studies was guided by Principle 4: using evidence informed interventions and translate existing complementary and integrative therapies into clinical practice bringing together practitioner expertise, patient presentation and preference to form the treatment plan . In some studies, the organisational process of constructing a treatment plan was documented and this is useful for future replication. Some of the trials then documented the frequency and types of treatments patients received although the details and rationale of the actual treatments are not provided. Journal article length makes this level of reporting unfeasible. Understanding frequency and intensity of the individualised treatments may be graphically depicted in ways suggested by Perera . This may provide an indication of ‘dose’ per individual. The use of an inadequate dose may be safe and less resource intensive but ineffective.
IHC practitioners in all studies were provided with education and training of variable intent and intensity. The provision of education and training prior to study commencement demonstrated an understanding of the complexity of bringing different professional health disciplines together . Opportunities for dialogue between different practitioners and group development build mutual understanding which is considered important for success of IHC [65, 66]. Future studies should consider the informal and formal means for collaboration and team building not only at study commencement but throughout the study. It is likely established IHC teams would operate in a more timely and cost effective fashion. Increased team cohesiveness may lead to improved safety, sharing resources, less opportunity for negative treatment interactions, reduced treatment cycles, reduced primary care visits and cost effectiveness .
The studies in our review used disease specific outcome measures combined with a quality of life measure and/or a patient reported measure such as MYMOP. While disease specific measures are important, so too are outcome measures on IHC interventions that cover physical, spiritual, psychological and social domains which go beyond measuring disease specific biomedical outcomes [67, 68]. Combinations of qualitative and quantitative measures are best placed to provide comprehensive outcome data. Philosophically, IHC and CM practices are not typically based on a mechanistic cause-effect relationship with a specific intervention for a specific symptom. Rather the approach is to bring the whole person back into ‘balance’ . One study in our review reported qualitative outcomes relating to changes in self-concept and empowerment, and benefits arising from treatment that was ‘whole-person’ focused . This supports other studies that indicate that IHC is associated with improved health related quality of life in diverse populations with substantial co-morbidity . Including measures that capture these outcomes is particularly relevant if the cost effectiveness of IHC through its capacity to reduce co-morbidities alongside disease specific symptoms and improve total well-being is to be captured [56, 60, 69, 71].
Measuring outcomes and designing IHC interventions is further complicated by understanding that causality lies for the effectiveness of an IHC intervention lies not just with the treatment component but by enhancing the healing capacity of the patient (salutogenesis) through the social context and healing environment . The patient focused, IHC team based approach is thought to enhance this process as team members contribute unique perspectives, skills and experience to patient care [73, 74]. Bell argues that the whole therapeutic strategy of IHC needs to be evaluated: including the patient-provider relationship, multiple conventional and CAM treatments, and the philosophical context of care as the intervention . Only one of the studies in our review included measures to attempt to evaluate this process . Future clinical trials of IHC interventions should include qualitative elements that seek to understand the ways in which this process may be fostered to maximise the specific and nonspecific healing effect of an IHC intervention.
The strengths of an IHC intervention to provide good external validity need to be considered against the inherent limitations of undertaking the evaluation of a complex intervention using an experimental design (see Table 5). Defining and articulating the “black box” of an IHC intervention is important for internal validity, generalizability and replicability . The difficulty in doing so within this type of trial design is one of the key limitations of a complex intervention. The individualised nature of IHC makes it difficult to know which component of the intervention is exerting the main effect - the combination of the therapies, the extra attention, the patient-practitioner relationship or something else not considered.
For the purposes of research, IHC is a therapeutic strategy not a single drug intervention. Team-based, patient-centred, integrative approaches to care present a challenge to designing rigorous studies, given IHC is typically used to provide many simultaneous treatments for multiple health concerns . Many efforts have been made to propose frameworks for researching complex healthcare such as IHC [28, 59, 60, 69] calling on program theory , whole system theory [69, 72], utilising the Medical Research Council (MRC) framework or employing implementation, process or fidelity evaluation . Research methodology for evaluating IHC probably best involves a combination of understanding the philosophical underpinnings of IHC through whole system theory and examining it within the MRC framework. Whole systems can be defined as “approaches to healthcare in which practitioners apply bodies of knowledge and associated practices in order to maximize the patients’ capacity to achieve mental and physical balance and restore their own health, using individualised, non-reductionist approaches to diagnosis and treatment”. In the case of IHC, it is the individualised integrative therapeutic strategy that is the whole system intervention.
The MRC framework follows a typical drug development pathway but provides guidance for identifying confounders, modelling to predict how components may interact and identifying the constant and variable components of the intervention. Within the MRC framework a nested “process evaluation” within the study would provide insight on the constant and variable components of the intervention. A process evaluation would investigate contextual factors such as setting, team composition and facilitation, and examine patient-provider expectations and relationships . A process evaluation may follow some of the dimensions identified in program implementation theory: fidelity, quality of delivery, participant responsiveness, and program adaptation  Each of these dimensions has been demonstrated to influence outcomes. Strategies may include interviews, focus groups, and observations alongside document reviews of clinical files and correspondence. An additional complexity is that, not unlike conventional interventions for chronic and complex conditions, research needs to be conducted over the longer term to truly capture outcomes. Jonas et al.  document an evaluation model for integrative care specifically for cancer but equally applicable to understanding and evaluating IHC in primary care . The model is designed to collect data on wellbeing, behavior, clinical outcomes, bio-measures, costs and the course of treatment and compare IHC with standard healthcare practices.
A further challenge in IHC research is the preference of researchers and funders typically prefer investigations that are linear, showing a clear cause and effect, with few confounders and cost effectiveness can clearly be determined  . In the US Institute of Medicine examination of contemporary approaches to CM research acknowledge the limitations of researchers trained in Western cultures, where complex “systems” with multiple levels of relationships and multiple factors which are interactive and iterative and do not fit into this preferred type of research . The typical efficacy focused RCT prescribes to the ‘average’ patient and is ‘fundamentally’ at odds with CM orientation to the ‘individual’ patient . The number of patients using CM continues to grow, as does the number of patients that desire a general practitioner who communicates about, and refers to, CM practitioners. Patients are seeking care that is tailored to their individual needs and where CM and conventional medicine collaborate [78–81] with availability of these treatment options in one location being cited as desirable . These considerations aside, patients are already integrating conventional and CM therapies on their own due to a desire to access the best that both healthcare paradigms have to offer . Investigations on the efficacy, safety and cost effectiveness of an IHC model of care are warranted to guide health policy makers and consumers in decision-making and there are sufficient research and statistical methods available that enable such investigation.
Proposed RCT designs for complex interventions include pragmatic trials, factorial design, preference trials and randomised consent designs, N-of-1 designs . These trial designs may be used to address the preferences of patients, which are often strong in CM users, for an integrative approach . In considering comparators and research design, there is a broad consensus that the evaluation of IHC and CM be conducted where possible within a comparative effectiveness framework [28, 83, 84]. The Institute of Medicine defines CER as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care . In selecting comparators for an experimental design, IHC lends itself with some ease to comparative effectiveness studies.
The cost effectiveness data collected in the studies in our review was comprehensive in only one study. Cost effectiveness analysis for IHC interventions should include sick leave, medication use, return-to-work and other cost-related data alongside cost of treatment course, and compliance. Sundberg et al.  recommend future studies include collection of cost through using cost diaries, measuring health care visits, sick leave.
From the reviewed studies, it is difficult to determine the type and level of resources required to conduct IHC patient assessment and treatment planning. Patient assessment and enrolment in the trials were typically undertaken by one or two therapists this may increase cost. Likewise means for collaboration such as meetings are costly. Any additional duties typically come at a direct financial cost to fee-for-service practitioners or need to be compensated for within the IHC model. Conversely it is thought that within the integrative whole person approach there is considerable potential for cost-effectiveness . Some preliminary data shows that various CM interventions may be cost effective . Consumers and policy makers need to know if IHC models are cost effective for effective decision-making.
This review provides an opportunity to consider methodological challenges that arise in undertaking a trial of a multidisciplinary, complex intervention. To enable an understanding of how an individualised intervention is developed the structure and the process utilised needs to be transparent. With single interventions, internal validity is maximised by having a standard dose delivered in the same way at each trial site. Complex interventions can retain some internal validity by standardising the process, guiding principles, and structure of the intervention while allowing the ‘form’ to be adapted .
A limitation of our review is that we may have omitted some studies. Our review is also limited by the inclusion criteria we selected. Our review supports the findings of Coulter et al., that the lack of a clear definition and taxonomy for integrative health care makes reviewing this field challenging .
The trials in our review provide a small yet critical base from which to draw upon, refine and develop larger studies. We recommend pragmatic controlled trials that document the structure of the IHC environment, the process of care and measure outcomes beyond those reported in mechanistic efficacy based trials similar to the efforts being undertaken in the Integrative Medicine PrimAry Care Trial (IMPACT) . Complex interventions have inherent limitations not the least of being low internal validity. To strengthen the quality of the trials, trials should have built in process evaluations and clear guiding principles that provide structure beyond the individualized treatment components. Such trials will provide evidence of efficacy, safety and identify challenges and opportunities of this healthcare strategy. The trials will complement the large scale observational studies of IHC interventions such as that conducted on Chronic Pain by the Bravewell Integrative Medicine Research Network (BRAVENET) collaborators . An evidence base on IHC is needed to inform patients and healthcare providers and policy makers. Future research in the IHC field would do well to build on the recommendations from the existing body of experimental design. In chronic illnesses where there is enormous burden there is considerable merit in investigating an IHC approach to see if it is able to reduce the health costs.
Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy: strategies for combining interventions at multiple levels. J Natl Cancer Inst Monogr. 2012;2012(44):34–41.
Kaptchuk TJ, Eisenberg DM. The persuasive appeal of alternative medicine. Ann Intern Med. 1998;129(12):1061–5.
Wolsko PM, Eisenberg DM, Simon LS, Davis RB, Walleczek J, Mayo-Smith M, et al. Double-blind placebo-controlled trial of static magnets for the treatment of osteoarthritis of the knee: results of a pilot study. Altern Ther Health Med. 2004;10(2):36–43.
Institute of Medicine (US) Committee on the Use of Complementary and Alternative Medicine by the American Public. Complementary and Alternative Medicine in the United States. Washington (DC): National Academies Press (US); 2005.
Brien SB, Bishop FL, Riggs K, Stevenson D, Freire V, Lewith G. Integrated medicine in the management of chronic illness: a qualitative study. Br J Gen Pract. 2011;61(583):e89–96.
Artus M, Croft P, Lewis M. The use of CAM and conventional treatments among primary care consulters with chronic musculoskeletal pain. BMC Fam Pract. 2007;8(1):26.
Goertz C, Salsbury S, Vining R, Andresen A, Hondras M, Jones M, Killinger L, Long C, Lyons K, Wallace R: Development of an interprofessional model of collaborative care by doctors of chiropractic and medical doctors for older adults with low back pain. BMC Complementary and Alternative Medicine Conference: International Research Congress on Integrative Medicine and Health. 2012;12(Suppl 1):P262
Emmerton L, Fejzic J, Tett SE. Consumers' experiences and values in conventional and alternative medicine paradigms: a problem detection study (PDS). BMC Complement Altern Med. 2012;12:39.
Coulter ID, Khorsan R, Crawford C, Hsiao AF. Integrative health care under review: an emerging field. J Manip Physiol Ther. 2010;33(9):690–710.
Grace S, Vemulpad S, Reid A, Beirman R. CAM practitioners in integrative practice in New South Wales, Australia: A descriptive study. Complement Ther Med. 2008;16(1):42–6.
Boon H, Verhoef M, O'Hara D, Findlay B, Majid N. Integrative healthcare: arriving at a working definition. Altern Ther Health Med. 2004;10(5):48–56.
Maizes V, Rakel D, Niemiec C. Integrative medicine and patient-centered care. Explore (NY). 2009;5(5):277–89.
Eisenberg DM. Trends in integrative medicine: A U.S. perspective. European Chin J Integr Med. 2010;2(4):160.
Gamst A, Haahr N, Kristoffersen AE, Launso L. Integrative care and bridge building among health care providers in Norway and Denmark. J Altern Complement Med. 2006;12(2):141–6.
Mann D, Gaylord S, Norton S. Moving toward integrative care: rationales, models, and steps for conventional-care providers. Complement Health Pract Rev. 2004;9(3):155–72.
Gaboury I, April K, Verhoef M. A qualitative study on the term CAM: is there a need to reinvent the wheel? BMC Complement Altern Med. 2012;12(1):131.
Kligler B. The Integrative Family Practice. Explore: The Journal of Science and Healing. 2006;2(6):522–30.
Boon HS, Mior SA, Barnsley J, Ashbury FD, Haig R. The difference between integration and collaboration in patient care: results from key informant interviews working in multiprofessional health care teams. J Manip Physiol Ther. 2009;32(9):715–22.
Khorsan R, Coulter ID, Crawford C, Hsiao AF. Systematic review of integrative health care research: randomized control trials, clinical controlled trials, and meta-analysis. Evidence-based complementary and alternative medicine : eCAM. 2011;2011.
Coulter ID, Khorsan R, Crawford C, Hsiao AF. Challenges of systematic reviewing integrative health care. Integrative medicine insights. 2013;8:19–28.
Mulkins A, Verhoef M, Eng J, Findlay B, Ramsum D. Evaluation of the Tzu Chi Institute for Complementary and Alternative Medicine's Integrative Care Program. J Altern Complement Med. 2003;9(4):585–92.
Scherwitz LW, Cantwell M, McHenry P, Wood C, Stewart W. A descriptive analysis of an integrative medicine clinic. J Altern Complement Med. 2004;10(4):651–9.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ (Clinical research ed). 2008;337:a1655.
Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.
Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ. 2009;339:b3496.
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694.
Block KI, Burns B, Cohen AJ, Dobs AS, Hess SM, Vickers A. Point-Counterpoint: Using Clinical Trials for the Evaluation of Integrative Cancer Therapies. Integrative Cancer Therapies. 2004;3(1):66–81.
Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A. Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT. Complement Ther Med. 2005;13(3):206–12.
Understanding Complementary Medicine. Retrieved from http://nicm.edu.au/health_information/information_for_consumers/understanding_cm
Medicine NCfCaA. Complementary, Alternative, or Integrative Health. In: What’s In a Name? USA: National Center for Complementary and Alternative Medicine; 2014.
Rademakers J, Delnoij D, de Boer D. Structure, process or outcome: which contributes most to patients' overall assessment of healthcare quality? BMJ Quality & Safety. 2011;20(4):326
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295–309.
Higgins, Julian PT. ed. Cochrane handbook for systematic reviews of interventions. Vol. 5. Chichester, England: Wiley-Blackwell, 2008.
Garrard J. Health sciences literature review made easy: the matrix method: Sudbury, Mass: Jones & Bartlett Learning; 2011.
Richardson J. Developing and evaluating complementary therapy services: part 2. Examining the effects of treatment on health status. J Altern Complement Med. 2001;7(4):315–28.
Sundberg T, Petzold M, Wandell P, Ryden A, Falkenberg T. Exploring integrative medicine for back and neck pain - a pragmatic randomised clinical pilot trial. BMC Complement Altern Med. 2009;9:33.
Goertz CM, Salsbury SA, Vining RD, Long CR, Andresen AA, Jones ME, et al. Collaborative Care for Older Adults with low back pain by family medicine physicians and doctors of chiropractic (COCOA): study protocol for a randomized controlled trial. Trials. 2013;14:18.
Eisenberg DM, Buring JE, Hrbek AL, Davis RB, Connelly MT, Cherkin DC, et al. A model of integrative care for low-back pain. J Altern Complement Med. 2012;18(4):354–62.
Westrom KK, Maiers MJ, Evans RL, Bronfort G. Individualized chiropractic and integrative care for low back pain: the design of a randomized clinical trial using a mixed-methods approach. Trials. 2010;11:24.
Richardson J. Developing and evaluating complementary therapy services: Part 1. Establishing service provision through the use of evidence and consensus development. Jo. J Altern Complement Med. 2001;7(3):253–60.
Maiers MJ, Westrom KK, Legendre CG, Bronfort G. Integrative care for the management of low back pain: use of a clinical care pathway. BMC Health Serv Res. 2010;10:298.
Sundberg T, Halpin J, Warenmark A, Falkenberg T. Towards a model for integrative medicine in Swedish primary care. BMC Health Serv Res. 2007;7:107.
Westrom K, Bronfort G, Evans R, Maiers M: Psychosocial outcomes from the chiropractic and integrative care low back pain randomized clinical trial. BMC Complementary and Alternative Medicine Conference: International Research Congress on Integrative Medicine and Health 2012, 20120515(20120518).
Bronfort G, Maiers M, Evans R, Westrom K. P02.129. Individualized chiropractic and integrative care for low back pain: a randomized clinical trial. BMC complementary and alternative medicine. 2012;12 Suppl 1:P185.
Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ. 1996;312(7037):1016–20.
Andersson S, Sundberg T, Johansson E, Falkenberg T. Patients' experiences and perceptions of integrative care for back and neck pain. Altern Ther Health Med. 2012;18(3):25–32.
Lewith G. A Provider's Perspective: Current Issues in Providing and Funding Complementary Medical Care. Forschende Komplementarmedizin und klassische Naturheilkunde. Research in complementary and natural classical medicine. 2000;7:242–6.
Chung VCH, Ma PHX, Hong LC, Griffiths SM. Organizational Determinants of Interprofessional Collaboration in Integrative Health Care: Systematic Review of Qualitative Studies. PLoS One. 2012;7(11):e50022.
Hsiao A-F, Ryan GW, Hays RD, Coulter ID, Andersen RM, Wenger NS. Variations in provider conceptions of integrative medicine. Soc Sci Med. 2006;62(12):2973–87.
Gattellari M, Butow PN, Tattersall MH. Sharing decisions in cancer care. Soc Sci Med. 2001;52(12):1865–78.
Griffin SJ, Kinmonth AL, Veltman MW, Gillard S, Grant J, Stewart M. Effect on health-related outcomes of interventions to alter the interaction between patients and practitioners: a systematic review of trials. Ann Fam Med. 2004;2(6):595–608.
Furlan AD, Yazdi F, Tsertsvadze A, Gross A, Van Tulder M, Santaguida L, et al. A systematic review and meta-analysis of efficacy, cost-effectiveness, and safety of selected complementary and alternative medicine for neck and low-back pain. Evidence-based complementary and alternative medicine : eCAM. 2012;2012:953139.
Broom A, Kirby E, Sibbritt D, Adams J, Refshauge K. Use of complementary and alternative medicine by mid-age women with back pain: a national cross-sectional survey. BMC Complement Altern Med. 2012;12:98.
Murthy V, Sibbritt D, Adams J, Broom A, Kirby E, Refshauge KM. Consultations with complementary and alternative medicine practitioners amongst wider care options for back pain: a study of a nationally representative sample of 1,310 Australian women aged 60–65 years. Clin Rheumatol. 2013;33(2):253–62.
Coates JR, Jobst KA. Integrated healthcare: a way forward for the next five years? A discussion document from the Prince of Wales's Initiative on Integrated Medicine. J Altern Complement Med. 1998;4(2):209–47.
Grant S, Bensoussan A. The process of care in integrative health care settings - a qualitative study of US practices. BMC Complement Altern Med. 2014;14(1):410.
Fisher P, van Haselen R, Hardy K, Berkovitz S, McCarney R. Effectiveness gaps: a new concept for evaluating health service and research needs applied to complementary and alternative medicine. J Altern Complement Med. 2004;10(4):627–32.
Wight D, Obasi A: Unpacking the black box: the importance of process data to explain outcomes. In. Edited by Stephenson JM, Imrie, J. and Bonell, C. (eds.) Series: Oxford University Press, Oxford, UK, pp. 151–166. Oxford, UK: Oxford medical publications. 2003: 151–166
Dodds SE, Herman PM, Sechrest L, Abraham I, Logue MD, Grizzle AL, et al. When a whole practice model is the intervention: developing fidelity evaluation components using program theory-driven science for an integrative medicine primary care clinic. Evidence-based complementary and alternative medicine: eCAM. 2013;652047.
Herman PM, Dodds SE, Logue MD, Abraham I, Rehfeld RA, Grizzle AJ, et al. IMPACT--Integrative Medicine PrimAry Care Trial: protocol for a comparative effectiveness study of the clinical and cost outcomes of an integrative primary care clinic model. BMC Complement Altern Med. 2014;14:132.
Maiers M, Bronfort G, Evans R, Westrom K, Legendre C. Evidence as a Common Ground for a Multi-disciplinary Health Care Team. Presentation: Evidence 2011 - Transforming Healthcare. October 24-26, 2011; London, England
RB H, DL S, JAM G, DJ C, GH G. Transferring evidence from research into practice, I: the role of clinical care research evidence in clinical decisions. Evidence Based Medicine. 1996;1:196–7.
Perera R, Heneghan C, Yudkin P. Graphical method for depicting randomised trials of complex interventions. BMJ. 2007;334(7585):127–9.
Orchard CA, Curran V, Kabene S: Creating a Culture for Interdisciplinary Collaborative Professional Practice, Medical Education Online, 2009,10
Welch M. Action research, critical realism and new knowledge for delivering integrative medicine: Sharing research results. European Chin J Integr Med. 2012;4(3):e354.
Reason P. Power and conflict in multidisciplinary collaboration. Complement Med Res. 1999;5(3):144–50.
Verhoef M, Vanderheyden L, Dryden T, Mallory D, Ware M. Evaluating complementary and alternative medicine interventions: in search of appropriate patient-centered outcome measures. BMC Complement Altern Med. 2006;6(1):38.
Deng G, Weber W, Sood A, Kemper KJ. Research on integrative healthcare: context and priorities. Explore (New York, NY). 2010;6(3):143–58.
Bell IR, Koithan M, Pincus D. Methodological implications of nonlinear dynamical systems models for whole systems of complementary and alternative medicine. Forschende Komplementarmedizin (2006). 2012;19 Suppl 1:15–21.
Greeson JM, Rosenzweig S, Halbert SC, Cantor IS, Keener MT, Brainard GC. Integrative medicine research at an academic medical center: Patient characteristics and health-related quality-of-life outcomes. J Altern Complement Med. 2008;14(6):763–7.
Long AF. Outcome Measurement in Complementary and Alternative Medicine: Unpicking the Effects. The Journal of Alternative and Complementary Medicine. 2002;8(6):777–86.
Jonas WB, Beckner W, Coulter I. Proposal for an Integrated Evaluation Model for the Study of Whole Systems Health Care in Cancer. Integr Cancer Ther. 2006;5(4):315–9.
Launso L, Skovgaard L. The IMCO scheme as a tool in developing team-based treatment for people with multiple sclerosis. J Altern Complement Med. 2008;14(1):69–77.
Grumbach K, Bodenheimer T. Can health care teams improve primary care practice? JAMA. 2004;291(10):1246–51.
Verhoef MJ. Evaluating integrative medicine: science and art. Commentary on "A descriptive analysis of an integrative medicine clinic". Journal of Alternative & Complementary Medicine. 2004;10(4):595–6.
Ritenbaugh C, Verhoef M, Fleishman S, Boon H, Leis A. Whole systems research: a discipline for studying complementary and alternative medicine. Altern Ther Health Med. 2003;9(4):32–6.
Berkel C, Mauricio AM, Schoenfelder E, Sandler IN. Putting the pieces together: an integrated model of program implementation. Prev Sci. 2011;12(1):23–33.
McCaffrey AM, Pugh GF, O'Connor BB. Understanding patient preference for integrative medical care: results from patient focus groups. J Gen Intern Med. 2007;22(11):1500–5.
Franzel B, Schwiegershausen M, Heusser P, Berger B. Individualised medicine from the perspectives of patients using complementary therapies: a meta-ethnography approach. BMC Complement Altern Med. 2013;13:124.
Joos S, Musselmann B, Szecsenyi J. Integration of complementary and alternative medicine into family practices in Germany: results of a national survey. Evidence-based complementary and alternative medicine : eCAM. 2011;2011:495813.
Jong MC, van de Vijver L, Busch M, Fritsma J, Seldenrijk R. Integration of complementary and alternative medicine in primary care: What do patients want? Patient Educ Couns. 2012;89(3):417–22.
Verhoef MJ, Leis A. From studying patient treatment to studying patient care: arriving at methodologic crossroads. Hematol Oncol Clin North Am. 2008;22(4):671–82.
(US) IoM. What is comparative effectiveness research? Washington DC, USA: The National Academies Press; 2009.
Fischer HF, Junne F, Witt C, von Ammon K, Cardini F, Fønnebø V, et al. Key issues in clinical and epidemiological research in complementary and alternative medicine–a systematic literature review. Forschende Komplementärmedizin. 2012;19 Suppl 2:51–60.
Witt CM, Chesney M, Gliklich R, Green L, Lewith G, Luce B, et al. Building a Strategic Framework for Comparative Effectiveness Research in Complementary and Integrative Medicine. Evidence-Based Complementary and Alternative Medicine. 2012;2012:5.
Herman P, Craig B, Caspi O. Is complementary and alternative medicine (CAM) cost-effective? a systematic review. BMC Complement Altern Med. 2005;5(1):11.
Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004;328(7455):1561.
Abrams D, Dolor R, Roberts R, Pechura C, Dusek J, Amoils S, et al. The BraveNet prospective observational study on integrative medicine treatment approaches for pain. BMC Complement Altern Med. 2013;13(1):146.
Westrom K, Maiers M, Evans R, Bronfort G. Decision making within an integrative care team: an experimental model. J Chiropr Educ. 2009;23(1):119–9.
The authors declare that they have no competing interests.
SG collected and analysed data, and wrote manuscript. JF analysed data and reviewed/edited manuscript AB contributed to discussion, reviewed/edited manuscript. All authors read and approved the final manuscript.