This article has Open Peer Review reports available.
Identifying the barriers to conducting outcomes research in integrative health care clinic settings - a qualitative study
© Verhoef et al; licensee BioMed Central Ltd. 2010
Received: 21 July 2009
Accepted: 14 January 2010
Published: 14 January 2010
Integrative health care (IHC) is an interdisciplinary blending of conventional medicine and complementary and alternative medicine (CAM) with the purpose of enhancing patients' health. In 2006, we designed a study to assess outcomes that are relevant to people using such care. However, we faced major challenges in conducting this study and hypothesized that this might be due to the lack of a research climate in these clinics. To investigate these challenges, we initiated a further study in 2008, to explore the reasons why IHC clinics are not conducting outcomes research and to identify strategies for conducting successful in-house outcomes research programs. The results of the latter study are reported here.
A total of 25 qualitative interviews were conducted with key participants from 19 IHC clinics across Canada. Basic content analysis was used to identify key themes from the transcribed interviews.
Barriers identified by participants fell into four categories: organizational culture, organizational resources, organizational environment and logistical challenges. Cultural challenges relate to the philosophy of IHC, organizational leadership and practitioner attitudes and beliefs. Participants also identified significant issues relating to their organization's lack of resources such as funding, compensation, infrastructure and partnerships/linkages. Environmental challenges such as the nature of a clinic's patient population and logistical issues such as the actual implementation of a research program and the applicability of research data also posed challenges to the conduct of research. Embedded research leadership, integration of personal and professional values about research, alignment of research activities and clinical workflow processes are some of the factors identified by participants that support IHC clinics' ability to conduct outcomes research.
Assessing and enhancing the broader evaluation culture of IHC clinics prior to implementing outcomes research may be a critical step towards ensuring productive and cost-effective research programs. However, as IHC clinics are often complex systems, a whole systems approach to research should be used taking into account the multidimensional and complex nature of such treatment systems so that the results are useful and reflect real life.
Integrative health care (IHC) is an interdisciplinary blending of conventional medicine and complementary and alternative medicine (CAM) for the purpose of enhancing patients' health . The blending of these approaches ranges from "combining treatments from conventional medicine and complementary/alternative therapies for which there is some high-quality evidence of safety and effectiveness"  to "the practice of medicine that reaffirms the importance of the relationship between practitioner and patient, focuses on the whole person, is informed by evidence, and makes use of all appropriate therapeutic approaches, healthcare professionals and disciplines to achieve optimal health and healing" . The increasing development of and demand for IHC clinics to provide a comprehensive evidence-informed approach to health and healing has prompted the need for regular assessment of patient-centered outcomes to assess the value of such models. Despite this realization, few CAM/IHC clinics in North America have achieved robust and productive outcomes research programs.
Based on our team's experiences working in IHC clinics [4, 5], outcomes assessment is challenging, especially finding and choosing outcome measures that are appropriate and relevant to practitioners, patients, and administrators, and that do not burden the patient or practitioner. In addition, many of the outcome measures used to date tend to reflect a biomedical paradigm and measure disease oriented clinical and health status outcomes, rather than outcomes that capture the patients' lived experiences and self-identified health goals. This may, partly, be attributed to the need to demonstrate 'credible' results to the biomedical community.
In addition to physical health and well-being, the goals of IHC include outcomes such as self-awareness, transformation, motivation, balance, feeling connected and patient empowerment . However, the measurement and evaluation of these outcomes in IHC remains rare. For example, a review of evaluations of primary and community care services that include CAM in the UK, found that of the 25 services identified, the most commonly used outcome measures were data extracted from referral forms, service billings, patient satisfaction surveys, and patient health status questionnaires . Outcomes specific to CAM and IHC, as identified above, were often not assessed at all. A well-known instrument such as the MYMOP , an individualized assessment of outcomes that are most important to individual patients, was used in only 4 of the 25 service evaluations. IHC evaluations in North America also included only a limited range of outcome measures, and most of these were not CAM and IHC specific [9–13]. One study assessing patient outcomes in a collaborative model involving chiropractors and physicians did include the MYMOP along with pain and quality of life measures . A recent study that evaluated an IHC model at the University of Michigan was one of a few that used a relevant but not yet validated outcome measure (the Holistic Health Questionnaire) that addresses the various components of health identified within an IHC philosophy .
Based on the literature and our previous work we developed a 'package of outcome measures' (Additional file 1) in 2006 to enable IHC clinics to better understand if their unique approaches foster and enhance healing and to facilitate the comparison of different integrative models. This package consisted of instruments to assess concepts that were identified as central to IHC, closely capture the patients' lived experiences with IHC, and align clinical measures and research instruments with the philosophies and broad goals of IHC clinics. This package was piloted at four IHC clinics in Canada and the US. The purpose of the pilot study was to: 1) explore the feasibility of systematic outcomes data collection in IHC clinics, and 2) to assess which combination of outcome measures best captures healing experiences of patients in order to evaluate IHC clinics.
Despite strong study support from clinic leadership, implementation and recruitment for this pilot project was challenging. Although a significant amount of time was spent with each clinic to ensure that study protocols were modified and tailored to each participating clinics' critical paths and intake processes, it never became fully integrated into the daily practices and activities of the clinic. Challenges with patient recruitment were related to which patients should be approached for the study, when they should be approached, and by whom. Retention of participants was also a problem as many participants were lost to follow-up or withdrew from the study. The main reasons cited for withdrawal included health issues, no longer attending the clinic, and lack of time. Administration of the outcomes package at baseline was also problematic as it increased the large amount of paper work that new patients were required to complete. From our observations and interviews it also appeared that the clinics were not just struggling with the research process, but were subtly resistant to the notion of conducting research within their clinic, as such an evaluation process could be construed as interfering with their carefully conceived patient services. Finally, it appeared that participants saw no benefit from a research-in-practice cycle where research informs and enhances clinical practice.
In addition to the logistical issues, we learned of some major shortcomings of the outcomes package. Through participant feedback, it became apparent that the package did not focus enough on the positive aspects of well being and that it centered too much on ill-health. Similarly, several participants expressed that the package was more appropriate for individuals who were suffering from serious and chronic health concerns and suggested that most of the questions were not applicable for those who are attending clinics for health promotion and/or prevention reasons. This feedback brings forth the recognition that outcomes used to evaluate IHC not only need to be in line with the IHC philosophy of care but should also consider why people seek out IHC.
Since these issues were not exclusive to one or two clinics, we realized that successful outcomes research in IHC clinic settings is much more than having effective recruitment strategies, feasible data collection processes, and relevant patient-friendly outcome measures. We hypothesized that factors affecting a clinic's ability to successfully conduct outcomes research may arise out of lack of a culture in which research is considered important. We refer to a 'research culture' as a shared system of values, beliefs and attitudes within an organization which shapes and influences how employees engage in research activities. Research culture also describes the degree to which an organization defines itself in relation to research and how integral it is to practice . A lack of research culture is not an issue that is limited to IHC, as it has also affected such fields as primary care, counseling, education and kinesiology [17–21].
To facilitate the development of evidence-informed IHC practices, we developed a second phase to the original study to further explore the issues related to research culture. The objectives of this study were to: 1) understand and identify the reasons why IHC clinics are not conducting outcomes research, and 2) to determine the factors necessary for research programs to be successful and sustainable.
Due to the limited amount of information in the IHC field and the exploratory nature of the study objectives, a qualitative study design was used. Forty IHC clinics were identified across Canada through a combination of internet-based searches, informal networking among Canadian clinics, and a list of IHC clinics identified by Gaboury . Clinics were selected based on the following inclusion criteria: 1) multi-disciplinary approach, 2) a written mission statement outlining its role in healing and curing, 3) There are at least two CAM practitioners who practice different healing modalities, and 4) If there is not a medical doctor who practices on site, there must be some kind of formal communication strategy that outlines some level of collaboration between the clinic and physicians in the community. Letters of invitation were mailed to 80 individuals at 40 clinics. Approximately one to two individuals were contacted at each clinic, holding prominent roles, such as Clinic Director, Clinic Manager, Medical Director, Patient Care Coordinator and practitioners (physicians and CAM practitioners including naturopathic doctors, Traditional Chinese Medicine practitioners, and chiropractors).
Each study participant took part in an in-depth, semi-structured telephone interview, approximately 0.5-1 hour in length. They were conducted by two trained interviewers, based in British Columbia. The interview guide is included in Additional file 2. Because the concept of 'research culture' is somewhat abstract, this term was used interchangeably with the words 'climate' and 'environment' in the interviews. Nondirective prompts were used to encourage discussion of important issues that arose. Participants were also asked about their role in the clinic, time in the position and past research training.
Interviews were audio-taped and transcribed verbatim for subsequent analysis. Data collection and analysis were iterative, meaning that each transcript was analyzed before a subsequent interview took place. This process helped to assess the point where data saturation was reached. A basic content analysis approach was used, which involved a systematic process of manually indexing, coding, categorizing and interpreting . Transcripts were first analyzed line by line to identify concepts and then themes and categories were developed into a coding framework for application to the data. During this process, new themes developed and others changed. Analysis was independently conducted by two of the researchers. Themes emerging from the independent analyses were compared and discussed until consensus was reached.
Ethics approval was received from the Office of Medical Bioethics in the Faculty of Medicine at the University of Calgary (reference number 20222) on July 3, 2009 as a modification of the ethics approval received on May 30, 2006 for the initial pilot study.
All individuals voluntarily participated in the interviews. Prior to the interview, all participants were informed of: 1) the purpose and objectives of the study, 2) their rights to decline participation at any point during the interview or thereafter, and 3) how confidentiality of their persons and respective clinics would be protected. Any questions from the participants were addressed and informed consent was obtained verbally over the telephone by the interviewer prior to each telephone interview.
Description of participating clinics
Number of Practitioners
Who was interviewed
MD, ND, Chiro, Yoga, M/B, TCM, RMT
MD, ND, TCM,
M/B, Nutrition, ND
CAM Practitioner- Manager
ND, Nutrition, Dentist, TCM, M/B, MD
MD, RMT, TCM
Chiro, TCM, ND
ND, TCM, RMT, Chiro, MD, Yoga
ND, MD, RMT, TCM
MD, TCM, Phyio, M/B
MD, TCM, Physio, M/B
MD, ND, TCM
MD, RMT, TCM
MD, ND, Physio, Chiro
MD, M/B, Physio, Chiro
MD, TCM, RMT, ND, Chiro, Physio
RMT, TCM, ND
MD, ND, TCM
Perceived value and role of outcomes research
"I see outcomes research as the only way to really assess the success of our program. Sure we can see if our individual patients are happy and making progress but in order to get the full picture, we need to document the progress and the pitfalls in a formal way. I feel this is important not only to us as practitioners in this clinic but also as a way to bring more credence and I suppose evidence to this field. We have come a long way but I feel there is still so much more to be explored and understood." (CAM Practitioner-Manager, Ontario-I)
Generally, participants supportive of in-house research had very clear ideas on what they would like to achieve through research and its resultant benefits. A few practitioners and administrators believed outcomes research could help them to understand the complex interrelationships and impact of the various healing modalities offered at their clinics, and would engage patients more meaningfully within the clinic and in their healing journey. They felt that research could also help them advocate for change in the health care system. Clinic administrators in particular, thought that research participation would enrich IHC staff's work, making it more interesting and relevant, resulting in enhanced job satisfaction and retention. They also felt that research could bring additional skills and new perspectives into their IHC setting.
"The practice itself will succeed if the approach and treatment is successful. If we as a clinic cannot provide 'better' results than the traditional health care system, then we would not be able to charge for our services. The fact that we have established a thriving business and have 'success' stories, supersedes our need to collect research". (CAM Practitioner, Alberta-D)
An underlying sentiment throughout the interviews suggested that administrators felt that conducting research was the politically correct thing to do and expressed feeling some pressure when they compared themselves to their peers and to what other IHC clinics were doing. Participants who worked in management generally had a more favorable opinion about research than did the practitioners. Several practitioners commented that administrators seemed to "look at research through rose colored glasses" whereas they, as well as front line staff, shared a more realistic perspective about the practical implications of conducting research such as increased workload and lack of research skills. Despite the range of opinions about the value and role of research in the context of IHC clinics, the challenges of conducting research emerged as an important overarching theme in the interviews.
Challenges to conducting outcomes research
Despite the range of opinions about the value and role of research in the context of IHC clinics, the challenges of conducting research emerged as an important theme in the interviews. These challenges were summarized under four emergent categories: organizational culture, organizational resources, organizational environment, and logistical issues.
Challenges stemming from the culture existing within an organization such as the philosophy of IHC and the value and belief system of the organization were common issues among participants.
a) Philosophy of IHC
"Research is not woven into the clinic's operating philosophy like patient education or outreach is. Since research has never been reflected in the clinic's overall governing mandate, the chances are slim that it will ever be reflected in any of our organizational activities. At this point, research does not fit with our organization." (Manager, Ontario-J)
Therefore, research was not a priority nor was it embedded in the daily operations of the clinic.
"Our focus is on healing patients. This is how we prioritize our time and energy. Research is secondary to providing results. We operate from a patient-based model and I don't really see where research can fit into that". (CAM Practitioner-Manager, Alberta-E)
Practitioners who were interested in outcomes research identified the clinic's culture as the main reason they had not incorporated research in their practice because "we do not work in an environment that promotes and fosters a spirit of inquiry". (CAM Practitioner, Ontario-H)
"There is a resistance to outcomes research at our clinic. Research doesn't mesh well with the integrative healing philosophy of whole person care. As clinicians, we evaluate patients as whole people, the sum of all parts, whereas outcomes research seems to want to break everything into individual parts. It doesn't fit with the philosophy of integration." (CAM Practitioner-Manager, Ontario-L)
b) Attitudes and beliefs
"I am a healer. I studied healing systems not research. My reason for being here is to work with patients and help them along their healing journey that is our focus." (Physician, British Columbia-A)
"A great deal of our patients are dealing with very complex health problems. It is a struggle for many to get through their days and their treatments. I feel that asking them to add this to their list of things to do would be unfair and unrealistic. We are trying to support them and create wellness for them, not to overburden them with paperwork and forms. And for what? How will they personally benefit from participation? It just doesn't seem to add up. I would rather have them focus on other things than research forms." (CAM Practitioner, Quebec-S)
"If research can be obtained in a manner that provides minimal impact on the delivery of primary care to the patient, then it is clearly worthwhile. However, the moment that it interferes or removes resources from running the clinic and services to the patient, then its value is diminished." (Physician, Ontario-Q)
Some participants (both managers and practitioners) found that study results introduced conflict within the clinic, as everyone had different ideas about the validity and interpretation of the findings. For some practitioners, research findings became a sore point if their specific area of expertise became a target. For example, data from one clinic's outcomes research revealed that patients were unhappy about receiving mixed messages and conflicting treatment protocols from the nutritionist, naturopath and acupuncturist. Several practitioners found that situations like this did not nurture support for the continuation of outcomes research due to fear of having their professional practice scrutinized. For some, this was enough of a hindrance to discourage them from pursuing or supporting further research projects.
c) Relevance of conducting research
"We spend so much energy getting the data and eagerly await the analysis only to find we can't apply any of the results to our daily practice. The promise of useful findings to impact our practice was like the light at the end of the tunnel. What is the motivation then?" (CAM Practitioner, Alberta-F)
When there was an interest in research, the lack of organizational resources to successfully conduct outcomes research was commonly identified as a key barrier.
"The problem I see with research is mainly financial. How can we make money off of research? It costs us money because we would have to hire someone to carry it out for us and really, how do we get that money back? We are constantly trying to cut corners and save dollars every way we can, I just can't justify it right now with the current system. I have a long list of expenses that must be taken care of before I can even fathom introducing research into this clinic. Larger organizations which have more access to funding and more manpower are in a better position to take this on". (Clinic Director-Manager, Ontario-O)
b) Compensation for services
"We as a small independent clinic do not have the extra funds to hire someone to only conduct research which means ultimately, if we are to conduct research, the responsibility falls on the shoulders of the practitioners. However, I feel that integrative medicine, like many other health care fields, exists in a culture of productivity that I feel contradicts what research is all about. Research requires one to stop, reflect and discuss. In this field, productivity means taking care of our clients. That is rewarded, we aren't rewarded for sitting and thinking and documenting". (Clinic Director-Manager, Ontario-O)
c) Research training and skills
Many of the participants expressed concerns that they were ill prepared to conduct or engage in research on their own as they had little or no previous research training or skills. There was also a lack of incentives within the organization to encourage the development or improvement of their research literacy and capacity skills required to develop relevant research studies or to apply findings to improve practice patterns or service delivery.
"We got to the end of the study and had collected all of this data on our patients but had no clue what to do with it..... We spoke with a couple of people from the university who pointed out some serious flaws in our design and data collection. We felt so discouraged by the end result that we gave up. Nobody wanted to pursue it. The data is still sitting in a box in our storage room. It was a disappointing experience for everyone involved. Such a waste." (Physician, Ontario-R)
d) Clinic's partnerships/linkages
"I see other clinics like ours who have partnered with the local university and they are now hooked into all sorts of research initiatives. That is the route to go. We don't have those connections and contacts and I really think that limits us in a huge way in terms of opportunity to get involved in research." (CAM Practitioner-Manager, Ontario-M)
e) Research materials
In addition to the issues of funding and staffing, participants cited having limited access to research related resources such as books, academic CAM journals, and electronic databases as another barrier to research activities.
Many participants felt that the environment within which their clinics operated was constantly shifting and evolving thus influencing their ability to participate in research. Two of the issues participants struggled with most were the nature of their patient population and high staff turnover.
a) Nature of patient population
Clinics with previous research experience observed how the unpredictable nature of their patients' health concerns and irregularity of patient visits negatively impacted systematic follow-up of data collected. This reality made data collection and follow-up with patients challenging and was perceived as a possible threat to recruitment and retention of participants, ultimately impacting the quality of data:
"Many of my patients are living with illness that is cyclical and unpredictable. Often times they come to see me when they most need help and then I may not hear from them for a year. This lack of consistency is a major hindrance to participating in research. I have trouble enough following up with them, I can't imagine what it would be like to chase after them to get a questionnaire filled out!" (CAM Practitioner, British Columbia-C)
b) High staff turnover
"We work in an unpredictable, unstable environment. The loss of one staff person creates a hiccup in the research system and will inevitably impact results as we are often spread thin and the first thing to fall by the wayside is the research. Then we start up again once someone is trained and the cycle repeats itself." (Manager-Clinic Director, Ontario-N)
"At the clinic, everyone has their own tasks to take care of. It is finally running smoothly. Why would I want to introduce an extra layer to complicate things? It is not a priority largely for this reason. Who has the time to take this on? We struggle so much just to stay afloat at this clinic. Why kill ourselves even more with adding on more?" (Manager-Clinic Director, Quebec-T)
Strategies to support outcomes research
Although participants identified a significant number of barriers to creating a research culture, they also provided insights into ways in which clinics could encourage the development of research culture.
a) Develop and adopt a research philosophy
"Valuing research as an intrinsic component of the organization is critical because values and philosophy are what guide the staff, practitioners and patients within the clinic and gets us thinking research. It should be part and parcel of what we do as an organization. No questions asked. Clinical care and research are intimately linked and they feed each other. Staff, time and infrastructure for research related activities needs to be factored into the budget." (Manager, British Columbia-B)
b) Organizational leaders as research advocates
"It is the leader's job to make sure that research receives equal attention and importance within the clinic as all other aspects including patient education and patient care". (CAM Practitioner-Manager, Ontario-P)
"Appointing a person whose main role is to spearhead the research process would help our clinic immensely. The person could look for grant opportunities and linkages and get the ball rolling for us." (CAM Practitioner-Manager, Alberta-G)
c) Develop viable research processes and structures
In order for research findings to be meaningful and have a direct impact on practice, the research process must be collaborative, consensus-driven and inclusive, where individuals are encouraged to contribute to the research program. Also, it is essential to identify the needs and desired outcomes of the clinic prior to designing the research such that it reflects actual practice but is also responsive to the questions that the clinic team considers important and relevant. Participants felt that such an approach would encourage individuals to become more invested in the research process.
Research activities need to be aligned with clinical workflow processes at an early stage in order to minimize the potentially negative impact of adding new tasks to individuals' workload, particularly if they are not compensated. Research activities should be embedded into administrative and practice duties of front office support staff and practitioners, and explicitly outlined in the clinic policy and procedure manuals. Research protocols should be streamlined with clinical protocols. For example, consent for clinical purposes and research purposes should be merged into one form. In order to reduce the administrative burden on patients, ideally, outcomes collected for research purposes should be useful to practitioners as well. In this way, the research data are directly applicable to practice.
"This kind of formality and process will show a commitment to staff that the results of the research will weigh heavily in organizational decision making and where action will be taken accordingly. We also will be clear on decisions made about handling the final results." (Physician, British Columbia-A)
"I think more than anything, I would just like to be acknowledged for my extra work. I did a workshop for my team here based on some things that I learned at a research conference I attended and did I ever receive positive feedback. That type of recognition felt good and has encouraged me to attend more research related events." (Physician, Ontario-K)
Creating access to research resources was critical. Having access to academic CAM journals, establishing productive relationships with academic "mentors"/researchers, and access to research funds were key resources identified by most participants. Because funding and sufficient staffing were major issues for the participating clinics, several participants suggested the creation of large 'umbrella' research projects based in academic institutions in which smaller, individual IHC clinics could take part. In addition, participants discussed the possibility of having access to previously funded grant applications to use as a template to guide their own in-house proposal development.
Balancing the development of clinic services and research activities remains a challenge for most IHC clinics. Securing resources to do both at the same time is challenging, and superimposing research activities on clinic work processes without adding resources such as staff, space, expertise and time can be a stressful experience for clinicians and administrative staff. Identifying and actively removing the barriers to conducting outcomes research in IHC settings will be key to generating new knowledge in the field.
Many studies evaluating IHC were conducted by people outside of IHC clinics, often by students [22, 24]. This raises an interesting question about research initiated in-house versus research conducted by outsiders: Does the latter address or identify the needs of the clinic equally well? Our team consisted of a mixture of these, which was a major advantage in reflecting upon the ways in which our own values, experiences, interests, beliefs and social identities could have shaped our research. One of us (BF) is a nurse and has directed an Integrative Clinic in the early 2000s. SM is a CAM practitioner and researcher and is involved in developing, implementing and assessing different models of IHC in primary care community settings. AM and MV are not practitioners and have conducted research in IHC for the past 8 years. AK has worked as a CAM practitioner in an integrated health care setting. We believe that the range of our roles and our close collaboration has made it possible for us to maintain a critical and thoughtful attitude towards data collection and analysis.
Our earlier 'optimal outcomes' study set out to address previously identified barriers by choosing measures and research instruments that would more closely capture the patients' lived experiences, however, despite all these considerations, we still encountered significant barriers in implementing the study protocol within IHC sites. This study served to identify the subtle, yet complex issues that continue to hinder evaluation of IHC service delivery models. The relatively low response rate in this study further suggests the lack of interest in or relevance of research in the IHC context. Identifying these barriers proved to be very informative and may contribute to a more complete understanding of what "readiness" to conduct research practically looks and feels like in an IHC clinic .
We found that almost all barriers to conducting outcomes research are directly or indirectly correlated with the IHC clinic's organizational culture as it relates to evaluation and/or research. These include the research-related beliefs and values of staff, the degree of research literacy amongst the practitioners and individuals in management roles, and experience using quality management strategies to improve programs and services. Some would argue that the underpinnings of a supportive research culture should be introduced during a health professional's basic education and that practitioners' education should begin to develop research literacy and capacity. Many CAM institutions focus on training students to practice rather than conducting research, thus they have had little exposure to the relevance and need for research in this context during their training . In education programs for conventional health care providers/practitioners (such as physicians and nurses), research is often presented as an information source to be used in practice but produced "outside" of the practice setting and by "others". It is rarely taught as a component of the clinical practice for the purpose of internal evaluation or to contribute to a broader knowledge base within a given field.
As this study highlights, it is critical for IHC clinics to adopt research as a priority at the leadership level and to build in research capacity from the inception of the clinic. Capacity building activities could include formal mentorship by resident scholars; opportunities for all staff to serve as members of the research team to help design protocols and analyze results; journal clubs that introduce relevant literature and point out gaps in the literature; and, lastly, creating linkages to existing research programs or centers. Translating results into action is key to the sustainability of research. As illustrated by the data in this study, providing patients with a platform to provide ongoing feedback and share their healing experiences, clinics are creating endless opportunities to improve practice. Leaders that adequately weave research goals and related activities through the fabric of the organization will be more successful in creating an evaluation culture, which will in turn inform on-going development of effective services and programs.
The literature on IHC provides little practical guidance to IHC clinics about how to incorporate a viable research component within an IHC program. However, the fields of primary care, social work, higher education, and kinesiology have explored factors needed to build research capacity and develop a research culture in practice settings [17–21]. Barriers to research in these fields have been identified, such as lack of time to conduct research, staffing resources, and lack of interest among staff. It has been determined that protected, dedicated research time, effective managerial support, research training, immediate access to mentorship and a nurturing workplace environment are key to a sustainable research culture [17–21]. These findings draw numerous parallels to those identified in this study, demonstrating the need for individuals working in IHC to seek out knowledge and experience from those in other fields. Based on the findings of this study and those in other fields, the next step in fostering the development of research cultures in IHC clinics could be to develop a set of practical guidelines for clinic managers and directors outlining feasible approaches and options of how to implement a research arm within their clinic setting.
There is increasing public and professional demand to identify the most effective ways to deliver IHC, and agreement that evaluating the outcomes of IHC clinics is a critical step towards creating this new knowledge. In an ideal world, established IHC clinics would respond to this call for evidence by agreeing upon a set of relevant, easy to administer evaluation tools, and collaborate on collecting much-needed outcomes data. However, in an emerging field of study such as IHC with a diverse and broad range of options, this may not be possible in the short term. We acknowledge, from our own lived experiences as supported by these study findings, that IHC clinics are highly complex systems. Designing studies that take into consideration and respond to the multitude of factors and components that contribute and create an IHC system is needed to develop an in-depth and comprehensive understanding of IHC and the potential patient outcomes. Research approaches that view IHC from a whole systems perspective  are more likely to be experienced by stakeholders as 'respectful processes' and in turn, produce more useful knowledge.
As highlighted by this study, clinical and research pioneers of IHC are recognizing that creating a viable evaluation culture in tandem with developing clinic services is an important pre-cursor to successfully conducting outcomes research in complex IHC settings.
Funding for this research was provided by the Lotte & John Hecht Memorial Foundation.
- Boon H, Verhoef MJ, O'Hara D, Findlay B, Majid N: Integrative health care: arriving at a working definition. Altern Ther Health Med. 2004, 10: 48-56.PubMedGoogle Scholar
- What is Complementary and Alternative Medicine. [http://nccam.nih.gov/health/whatiscam/]
- The Consortium of Academic Health centres for Integrative Medicine. About us. [http://www.imconsortium.org/about/home.html/]
- Mulkins A, Verhoef MJ, Eng J, Findlay B, Ramsum D: Evaluation of the Tzu Chi Institute for complementary and alternative medicine integrative care program. J Altern Comp Med. 2003, 9: 585-592. 10.1089/107555303322284893.View ArticleGoogle Scholar
- Mulkins AL, Eng J, Verhoef MJ: Working towards a model of integrative health care: critical elements for an effective team. Complement Ther Med. 2005, 13: 115-122. 10.1016/j.ctim.2005.04.002.View ArticlePubMedGoogle Scholar
- Verhoef MJ, Mulkins A, Boon HS: Integrative health care: How can we determine whether patients benefit?. J Altern Comp Med. 2005, 11 (Suppl 1): 57-65.Google Scholar
- Wye L, Shaw A, Sharp D: Evaluating complementary and alternative therapy services in primary and community care settings: a review of 25 service evaluations. Complement Ther Med. 2006, 14: 220-30. 10.1016/j.ctim.2005.11.004.View ArticlePubMedGoogle Scholar
- Paterson C, Britten N: In pursuit of patient-centred outcomes: a qualitative evaluation of MYMOP, Measure Yourself Medical Outcome Profile. J Health Serv Res Policy. 2000, 5: 27-36.PubMedGoogle Scholar
- Greeson JM, Rosenzweig S, Halbert SC, Cantor IS, Keener MT, Brainard GC: Integrative medicine research at an academic medical center: Patient characteristics and health-related quality of life outcomes. J Altern Comp Med. 2008, 14: 763-767. 10.1089/acm.2008.0114.View ArticleGoogle Scholar
- Sarnat R, Winterstein J: Clinical and cost outcomes of an integrative medicine IPA. J Manipulative Physiol Ther. 2004, 27: 336-347. 10.1016/j.jmpt.2004.04.007.View ArticlePubMedGoogle Scholar
- Michalsen A, Grossman P, Acil A, Langhorst J, Ludtke R, Esch T, Stefano GB, Dobos GJ: Rapid stress reduction and anxiolysis among distressed women as a consequence of a three-month intensive yoga program. Med Sci Monit. 2005, 11: CR555-561.PubMedGoogle Scholar
- Scherwitz LW, Cantwell M, McHenry P, Wood C, Stewart W: A descriptive analysis of an integrative medicine clinic. J Altern Comp Med. 2004, 10: 651-659.View ArticleGoogle Scholar
- Brazier A, Cooke K, Moravan V: Using mixed methods for evaluating an integrative approach to cancer care: a case study. Integr Cancer. 2008, 7: 5-10.1177/1534735407313395.View ArticleGoogle Scholar
- Mior S, Barnsley J: Patient Outcomes in a Model of Interprofessional Patient-Centered Collaborative Practice: In-depth Case Study Design. Proceedings of the World Federation of Chiropractic 9th Biennial Congress: 17-19 May 2007; Vilamoura. 2007, World Federation of Chiropractic, 201-212.Google Scholar
- Myklebust M, Pradhan EK, Gorenflo D: An integrative medicine patient care model and evaluation of its outcomes: the University of Michigan experience. J Altern Complement Med. 2008, 14: 821-826. 10.1089/acm.2008.0154.View ArticlePubMedGoogle Scholar
- Gardner F, Nunan C: How to develop a research culture in a human services organization. Qual Soc Work. 2007, 6: 335-351. 10.1177/1473325007080405.View ArticleGoogle Scholar
- Graffy J: Building research capacity: Lessons from North America. Prim Health Care Res Development. 2003, 4: 191-192. 10.1191/1463423603pc149oa.View ArticleGoogle Scholar
- Bland CJ, Center BA, Finstad DA, Risbey KR, Staples JG: A theoretical, practical, predictive model of faculty and department research productivity. Acad Med. 2005, 80: 225-37. 10.1097/00001888-200503000-00006.View ArticlePubMedGoogle Scholar
- Bland CJ, Wersal L, VanLoy W, Jacott W: Evaluating faculty performance: a systematically designed and assessed approach. Acad Med. 2002, 77: 15-30.View ArticlePubMedGoogle Scholar
- Katerndahl D, Parchman M, Larme A: Cultural (r)evolution: Developing a research culture in family medicine. Fam Med. 2002, 34: 616-618.PubMedGoogle Scholar
- Wye L, Digby K: Building research capacity amongst kinesiologists: Results from a mixed methods study. Complement Ther Clin Pract. 2008, 14: 65-72. 10.1016/j.ctcp.2006.12.004.View ArticlePubMedGoogle Scholar
- Gaboury I: Interprofessional collaboration within integrative health care programs: a qualitative and quantitative assessment [abstract]. J Complement Integr Med. 2008, 5: 31.Google Scholar
- Thorne S, Reimer Kirkham S, MacDonald-Emes J: Interpretive description: a noncategorical qualitative alternative for developing nursing knowledge. Res Nurs Health. 1997, 20: 169-177. 10.1002/(SICI)1098-240X(199704)20:2<169::AID-NUR9>3.0.CO;2-I.View ArticlePubMedGoogle Scholar
- Hollenberg D: Uncharted ground: Patterns of professional interaction among complementary/alternative and biomedical practitioners in integrative health care settings. Soc Sci Med. 2006, 62: 731-744. 10.1016/j.socscimed.2005.06.030.View ArticlePubMedGoogle Scholar
- Findlay B, Verhoef M: Operationalizing the concept of the optimal healing environment in clinical settings: the importance of "readiness". J Altern Comp Med. 2004, 10 (Suppl 1): 121-126. 10.1089/1075553042245836.View ArticleGoogle Scholar
- Committee on the Use of Complementary and Alternative Medicine by the American Public: Complementary and Alternative Medicine (CAM) in the United States. 2005, Washington, DC: National Academies PressGoogle Scholar
- Bell I, Koithan M: Models for the study of whole systems. Integr Cancer Ther. 2006, 5: 293-307. 10.1177/1534735406295293.View ArticlePubMedGoogle Scholar
- Ware JE, Kosinski M, Keller SD: A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34: 220-233. 10.1097/00005650-199603000-00003.View ArticlePubMedGoogle Scholar
- Bell I, Cunningham V, Caspi O, Meek P, Ferro L: Development and validation of a new global well-being outcomes rating scale for integrative medicine research. BMC Complement Altern Med. 2004, 4: 1-10.1186/1472-6882-4-1.View ArticlePubMedPubMed CentralGoogle Scholar
- Spitzer WO, Dobson AJ, Hall J, Chesterman E, Levi J, Shepherd R, Battista RN, Catchlove BR: Measuring the quality of life of cancer patients. J Chron Dis. 1981, 34: 585-597. 10.1016/0021-9681(81)90058-8.View ArticlePubMedGoogle Scholar
- Degner LF, Sloan JA, Venkatesh P: The Control Preferences Scale. Can J Nurs Res. 1997, 29: 21-43.PubMedGoogle Scholar
- Howie JG, Heaney DJ, Maxwell M, Walker JJ, Freeman GK: Quality at general practice consultations: Cross sectional survey. Br Med J. 1999, 319: 738-743.View ArticleGoogle Scholar
- Jenkinson C, Coulter A, Reeves R, Bruster S, Richards N: Properties of the Picker Patient Experience Questionnaire in a randomized controlled trial of long versus short form survey instruments. J Public Health Med. 2003, 25: 197-201. 10.1093/pubmed/fdg049.View ArticlePubMedGoogle Scholar
- IN-CAM Outcomes Database Transformative Outcome Profile (MYTOP). [http://www.outcomesdatabase.org/show/element/id/274]
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/10/14/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.