Research article | Open | Open Peer Review | Published:
Design and validation of a questionnaire to assess organizational culture in French hospital wards
BMC Health Services Researchvolume 16, Article number: 491 (2016)
Although many organizational culture questionnaires have been developed, there is a lack of any validated multidimensional questionnaire assessing organizational culture at hospital ward level and adapted to health care context. Facing the lack of an appropriate tool, a multidisciplinary team designed and validated a dimensional organizational culture questionnaire for healthcare settings to be administered at ward level.
A database of organizational culture items and themes was created after extensive literature review. Items were regrouped into dimensions and subdimensions (classification validated by experts). Pre-test and face validation was conducted with 15 health care professionals.
In a stratified cluster random sample of hospitals, the psychometric validation was conducted in three phases on a sample of 859 healthcare professionals from 36 multidisciplinary medicine services: 1) the exploratory phase included a description of responses’ saturation levels, factor and correlations analyses and an internal consistency analysis (Cronbach’s alpha coefficient); 2) confirmatory phase used the Structural Equation Modeling (SEM); 3) reproducibility was studied by a test-retest.
The overall response rate was 80 %; the completion average was 97 %. The metrological results were: a global Cronbach’s alpha coefficient of 0.93, higher than 0.70 for 12 sub-dimensions; all Dillon-Goldstein’s rho coefficients higher than 0.70; an excellent quality of external model with a Goodness of Fitness (GoF) criterion of 0.99. Seventy percent of the items had a reproducibility ranging from moderate (Intra-Class Coefficient between 50 and 70 % for 25 items) to good (ICC higher than 70 % for 33 items).
COMEt (Contexte Organisationnel et Managérial en Etablissement de Santé) questionnaire is a validated multidimensional organizational culture questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first dimensional organizational culture questionnaire, specific to healthcare context, for a unit level assessment showing robust psychometric properties (validity and reliability). This tool is suited for research purposes, especially for assessing organizational context in research analysing the effectiveness of hospital quality improvement strategies. Our tool is also suited for an overall assessment of ward culture and could be a powerful trigger to improve management and clinical performance. Its psychometric properties in other health systems need to be tested.
The concept of Organizational Culture (OC) has become a major element in organizational research, because it is one determinant of individual behaviour and can be linked with some organizational outcomes as job satisfaction, professional engagement, turn over or achievement of goals [1–6]. In addition, operational studies tend to confirm the links between culture and performance in the health care sector, as for example quality of care, promptness of care, external stakeholders satisfaction, patient satisfaction and mortality rate [7–10]. Moreover, two landmark reports from the Institute of Medicine and calls to reforms of the National Health Service (NHS) have stressed the need for cultural changes in order to deliver improvements in quality and performance [11–14]. It is believed that only a transformation of professional and organizational culture will enable the instillation of new values, beliefs and assumptions to guide and underpin new way of working in healthcare organizations, for examples developing coordination of care across patient-conditions and services, improving cooperation among clinicians, deploying knowledge and information sharing, evidence-based clinical decisions making and the use of information technology.
However, the literature on the link between OC and quality of care is controversial. In their literature review on the link between OC and performance, T Scott et al.  identified four studies finding plausible evidence for a link between culture and performance, four with little evidence for such a link and two providing unclear findings. If many contextual social, economic, political and methodological reasons can explain some of these discrepancies , some authors [16–18] suggest that one possible explanation for the difficulty in finding consistent relationship between culture and effectiveness is that culture may influence effectiveness indirectly. They propose that culture potentially has a direct effect on attitudinal factors such as morale, commitment, job satisfaction, and that these “intermediate” factors then directly impact effectiveness. So, as the relationships between culture and performance are multiple, complex and contingent, we actually need tool allowing the global comprehension of the local organizational context making it possible to understand not only the “if” but also the “how” managerial practices could influence performance in health care settings [4–9]. Consistently, B Schneider et al. call for tools that are able to assess the moderator value of OC on the relationship between OC and performance .
Two main schools of thought can be distinguished, to divide the numerous definitions, meaning and the constitutive elements of OC [19–22]. A first approach considers culture as something that an organization “is”. Here, culture serves as a metaphor for describing an organization rather than being seen as something readily identifiable or separate from the organization itself. In contrast, the second perspective considers culture as something that an organization “has”. That is, culture represents aspects or variables of the organization that can be isolated, described, and manipulated. Under this latter conception, there is agreement about the OC definition, as shared values, common understandings, common beliefs and expectations of people belonging to a working team [23–25], that underpin and reinforce their behaviours and distinguish them from other teams . OC encapsulates not only what members of the organization have learned but also what they believe. It comprises perceptions as well as practices shared within the organization, rather than being solely based on values held by individual members . Schein identifies three layers for OC: at the most basic layer are the underlying assumptions that are the basic “taken for granted” beliefs that structure the thinking and behaviour of an individual; the second level, or values, constitutes the basic foundations for making judgements and distinguishing “right” from “wrong” behaviour; the third level corresponds to artefacts including the physical and behavioural manifestations of culture.
There is also variation in the types of assessment approaches used to analyse OC (qualitative versus quantitative). A commonly used method is the survey. There are many different types of questionnaires differing in terms of whether they are either typological, i.e. based on predefined cultural types with assessment results in one or more “types” of OC (for examples: clan, adhocracy, hierarchy or market cultures in the Competing Value Framework ) or dimensional, i.e. based on a dimensional patterns (for example: leadership, social life and communication, management…) which describes a culture by its position on a number of conditions variables . Typological questionnaires allowing a global view of organizational culture have been often validated at institutional (hospital) level. They constitute summative instruments defining the dominant culture of a setting. On the other hand, dimensional instruments allow precise and detailed description of the different OC components in a setting. They constitute effective operational diagnostic tools for a realistic analysis of managerial practices, which can be useful for guiding an intervention . They are also good research instruments assessing the attitudinal factors needed for understanding the mechanism of OC effectiveness on quality of care. The OC questionnaires also vary in scope, some focusing on the assessment in one or more specific domains of organizational culture [30–32] and others assessing a more comprehensive range of issues . They also vary according to the level of culture they tap into, with none convincingly addressing the deeper underlying assumptions that guide attitudes and behaviour and inform the stable substrate of culture .
An instrument that works well for one investigation may not be so effective for another study. In this regard, the concept of “fitness for purpose” has gained increasing importance within the health measurement literature and requires the evaluation of the quality of a given assessment instruments to be conducted in the context of its application, audience and intended use [33, 34].
CCECQA (Comité de Coordination de l’Evaluation Clinique et de la Qualité en Aquitaine), in partnership with French research teams in management, initiated and coordinated a national project, the TheOReM© project (Organizational Theories, Recommendations and Management) funded by the HAS (Haute Autorité de Santé - French National Authority for Health). The aim of the study was to identify the managerial and organisational characteristics of medical wards that are most strongly associated with performance. A part of the managerial and organisational characteristics were assessed by an OC questionnaire. The analysis level was at the level of the ward or unit, i.e. the smallest management entity at hospital, constituting teams of professionals sharing daily patient management practices. The ward has been shown to be an effective level for OC assessment and for quality and security improvement [35, 36].
For our purpose, a detailed and comprehensive managerial and organizational diagnostic tool, validated at ward level, was needed. OC typological questionnaires were excluded, as not descriptive enough for our needs. Existing OC dimensional organizational questionnaires did not show robust psychometric performance, or, adapted from industrial contexts, were not considered as specific enough for our need [37–39]. T Jung et al’s review of the psychometric properties of 48 OC tools noted that less than half (46 %) of the instruments had published data demonstrating adequate internal consistencies. Additionally, only one in five (21 %) instruments demonstrated adequate evidence for aggregating individual data to be representative of the organizational as a whole. Lastly, T Jung et al’s review noted that only one in five (19 %) of the examined instruments presented adequate evidence of the dimensionality of the instrument. Moreover, the existing OC questionnaires suffer from a lack of construction relying on a generally accepted and common framework that allows for a consistent approach to the conceptualization and measurement of organizational culture.
For all these reasons, the TheOReM© working group decided to design and validate a dimensional questionnaire relevant at healthcare ward level in public or private settings, on the basis of predefined dimensional patterns.
This paper presents the development and validation of this tool called COMEt (Contexte Organisationnel et Managérial en Etablissement de santé).
Design of a questionnaire
Selection of measuring instruments
A comprehensive literature search was conducted to identify existing measuring instruments. The retrieval strategy (data bases explored and key terms used) is shown in Table 1. References indicated in articles found were explored and experts were contacted.
Papers inclusion criteria were: French or English language, date of publication between 1980 and 2014, description of a quantitative organizational culture measurement instrument, based on a typology or not, available in its entirety or not, free or on a paying basis. Out of this first selection, a multidisciplinary expert group, composed of four methodologists and three specialists in management sciences, working in three different French research departments (Bordeaux, Paris, Nantes) specialised in questionnaire validation or management sciences, selected tools on the following criteria: quantitative instruments of organizational culture, having already been used in a healthcare settings, having a good face validity to assess a broad range of cultural dimensions. Priority was given to those instruments for which some data were available on their statistical validity and reliability as measures of organizational culture.
Definition of dimensions and sub-dimensions to be explored
A database including all items and themes of selected instruments was created. From that base, the expert group established by consensus the organizational culture dimensions and sub-dimensions to be explored. During three expert meetings, organizational culture dimensions and sub-dimensions were determined on the basis of OC concept and defined; all the database items and themes were classified in each of them.
The items identified in the chosen instruments were classified into specified dimensions and sub-dimensions. Redundant or irrelevant items to the French health system were excluded. The wording of items was in French with a special attention to the consistency in the editing. Items should have the same formulation. They all were edited as affirmative sentences addressing directly to the questioned professional or to the community of professionals of the unit. Whenever possible, validated scales were introduced into the questionnaire. The response options were a five point Likert scale, from “strongly disagree” to “strongly agree”.
Cross validation of items classification in predefined dimensions and sub-dimensions was done by the expert group. In addition, two organisation sociologists reviewed the questionnaire and confirmed the assumption that: all the items were related to the ward’s OC, each item was belonging to one of the dimensions and each dimension was represented by one or several items.
Pre-test and face validity
Finally, some 15 field health care professionals, from public and private hospital wards, completed the questionnaire and assessed its applicability and acceptance level.
Validation of the questionnaire
A survey was conducted in five French regions: Aquitaine, Bretagne, Franche-Comté, Poitou-Charentes, Rhône-Alpes, representing a population of 10 millions inhabitants.
The study population consisted of all professionals involved in patient care in the participating medical wards (doctors, nurses and orderlies), during the daytime or at night, and whatever their status or length of service. The following were excluded: trainee nurses and orderlies, doctors who were working only temporarily in the department, other paramedical personnel (physiotherapists, psychologists, etc.), and professional staff on extended leave. Each professional received oral and written information about the study, delivered during a meeting organized in each participating ward. They gave all verbal consent to participate.
The group of wards participating in the study was randomly chosen in each region, using a stratified cluster sample design. Four strata were defined, according to type of hospital: 1) university teaching hospitals called university hospital; 2) non-university public hospitals with more than 350 beds called large hospital; 3) non-university public hospitals with fewer than 350 beds called small hospital; and 4) private hospitals. If an hospital declined to participate, a new one was selected from the randomized list until there was a sufficient number of hospitals in each stratum.
Within the included wards, the study population of health care professionals was asked to complete the self administered OC questionnaire. The group of health care professionals having filled in the questionnaire constituted the study sample.
The data collection procedure ensured confidentiality of responses; it was coordinated in each region by one member of the study group.
The psychometric properties of the questionnaire were assessed in three successive steps: a) exploratory analysis, b) confirmatory analysis and c) reproducibility [40, 41]. The programs used were Stata and XL-Stat .
Exploratory analysis presented first the item response rate, the distribution of the responses for comprehensiveness and saturation assessment (floor and ceiling effects) and symmetry of the distribution. Dimensional structure was studied using the Spearman's correlation matrix and the Principal Component Analysis (PCA). The correlation analysis identified redundant items (correlation higher than 0.80), and wrongly located or irrelevant ones (correlation lower than 0.20). PCA allowed to identify grouping of items, called factors, and to check the one-dimensional character of the questionnaire. The number of significant factors was determined by applying the combination of criteria (the Kaiser’s criterion- eigenvalue > 1- and the Horn parallel analysis) and an Oblimin rotation was used to facilitate the factor interpretation. Each factor was described taking into account a squared cosine higher than 0.30 indicating high representativeness and a factor loading higher than 0.40 (absolute value) indicating significant contribution of the item. Internal consistency of the questionnaire and each of its a priori dimensions and sub-dimensions was measured by the Cronbach’s alpha coefficient (if alpha > 0.70, consistency was considered as satisfactory). As a result of the analysis, the questionnaire was trimmed and a new construct of dimensions and sub-dimensions was envisaged.
For the confirmatory analysis, the Structural Equation Model (SEM) was used to confirm the construct assumption of the questionnaire into dimensions and sub-dimensions, and the adjustment of the model to the data . The SEM consisted in an external model representing the relationships between the latent (the dimensions) and manifest variables (their items), and an internal model representing the relationships between the latent variables. The Partial Least Square (PLS) approach was used to estimate both the external and internal models. This approach allows determining latent variables without any assumption on distribution. The SEM model allowed calculating scores for each latent variable and taking into account the latent variables associated items and the causal ratio with the other latent variables [44, 45].
Dillon-Goldstein’s rho coefficient was calculated for each sub-dimension to confirm internal consistency as established by the SEM (threshold at 0.70). External quality of the models was measured by the Goodness of Fit (GoF) criterion. The predictions on the external model gave standardised external weights for quantification of the impact of items on their sub-dimension (percentage of the item effect on the sub-dimension). Finally, the calculation of structural coefficients allowed to make predictions on the internal model and to rank the sub-dimensions on the basis of their impact on OC.
A test-retest was performed to assess reproducibility. OC questionnaire was collected twice by the same group of health care professionals at two times separated by three weeks. For each item a percentage of agreement and an Intra-Class Correlation Coefficient (ICC) (of the 1.1 type) were calculated, reproducibility was good when the ICC value was higher than 0.70 .
OC questionnaire construct
The literature review identified 41 OC measuring instruments. The expert group selection process excluded all but 14, out of which a base of 906 items and 100 themes was created. Were extracted and removed from this database the items that were less relevant to a French context, mainly a few items clearly oriented to productivity and on fierce competition. Healthcare professionals are not confronted with direct competition and productivity requirements in the French healthcare system. It is at least partly explained by the modalities of hospital professional management, based on public sector rules and by patient management means, organized at ward level. Items that could not be found in selected questionnaires were introduced as they were considered key points in sociology of work. They were items on clinical decision making and others related to discriminatory practices (according to specific cultural or religious orientation or to gender).
All the instruments that were selected contributed to the questionnaire construct (Table 2). Eight different OC dimensions were identified. Under those dimensions, 26 sub-dimensions were identified and 131 items were spelled out. Three validated scales were retained in their entirety: professional life satisfaction scale (5 items) [47, 48], intention to stay scale (6 items) , and professional burn out scale (6 items) . They all contain 17 items.
Face validation led to some changes in item wording. Sub-dimensions “Conflict management procedures” and “Frequency of conflicts between professionals” were both treated separately for doctors and for paramedical professionals. The same was done for the sub-dimension “Quality of relationship between professionals” with creation of two new sub-dimensions: “Relations between paramedical professionals” and “Relations with and between doctors”.
A total of 36 medicine wards from 33 hospitals in the five French regions participated; there were 7 university hospital wards, 9 large hospital wards, 14 small hospital wards and 6 private hospital wards. The OC questionnaire was distributed to a total of 1081 professionals working in those wards (176 doctors, 440 nurses, and 465 orderlies).
A total number of 859 questionnaires were returned (79 % return rate, 77 % for doctors, 82 % for nurses, and 78 % for orderlies). There was no noticeable difference in participation rate per region. Table 3 shows the socio-demographic characteristics of the study sample. The vast majority of orderlies and nurses were women (respectively 93.8 and 92.5 %) with average ages under 40 years (respectively 39 – Standard Deviation SD: 9 and 37 – SD: 9). They both were most frequently working at day time only (respectively 81.5 and 63.9 %) with permanent status (71.6 % for both); the majority (57.8 %) of medical doctors were men with an average age of 42 years (SD: 11). In term of seniority, nurses had an average of 6 years of tenure (SD: 6), while orderlies and medical doctors had averages of 13 years (SD: 9.7) and 15 years (SD: 10.6), respectively.
Table 4 presents the main results of the exploratory analysis.
Professionals completed the questionnaire in 15 min in mean. The item response rate varied between 94 and 99 % and in more than 90 % of the cases all the items were completed. 22 items had a ceiling effect. Several items had a low correlation with all the others in the same sub-dimension and with most in the other sub-dimensions. Two items were redundant.
The PCA pinpointed 31 factors. The questionnaire’s unidimensional character was confirmed. In some sub-dimensions, such as 2a- “Satisfaction at work” and 2b- “Likelihood of remaining on staff”, all the items appeared on the same PCA factor after rotation, while in others, such as 3b(a)- “Notion of patient and care”, the items were all on different factors. However, in most cases, a dominant factor appeared and the items that were not on that factor were either irrelevant or in the wrong place. 38 items were multidimensional and 22 items did not appear on their sub-dimension main factor.
The Cronbach's alpha was of 0.95 for the questionnaire and ranged from 0.47 to 0.89 for dimensions and 0.35 to 0.92 for sub-dimensions. Internal consistency was considered as satisfactory in ten sub-dimensions. In 15 other sub-dimensions, removing one or several items increased the alpha coefficient.
On the basis of the results, a new structure of 21 sub-dimensions and 6 dimensions was devised. Table 5 presents the psychometric properties of the final version of the questionnaire. Cronbach’s alpha of this modified version of the questionnaire ranged from 0.54 to 0.91 for sub-dimensions. Internal consistency was considered as satisfactory in ten sub-dimensions. The Cronbach’s coefficients remaining under 0.70 corresponded to 8 sub-dimensions of which 5 contained only two or three items. One sub-dimension had a Cronbach’s alpha lower than 0.6. Two dimensions and nine sub-dimensions were deleted. Five sub-dimensions were split and two have been merged. 52 items have been deleted and 13 were rephrased. For example, all items expressed in a negative way were rephrased to items expressed with a positive way. Finally four items were added to maintain the sub-dimension « 4D – Organisational learning », that have been excluded at the confirmatory analysis step.
A SEM was applied to the 79 items regrouped into 20 sub-dimensions and 6 dimensions (without “Organizational learning” sub-dimension). The Dillon-Goldstein’s rho coefficient ranged from 0.76 and 0.93 (Table 5). The external model quality was high (GoF index equal to 0.99).
The standardized external weights per item varied from 0.08 to 0.86. There was little impact difference between the items of each sub-dimension except in 1c- “Acceptance of department standards”, 1d- “Professional aims”, 2c(2)- “Work exhaustion”, 4b(2)- “Discriminatory practices” and 4 g- “Type of behaviour encouraged in the department”, where one or several items had considerably less impact than the others. The internal model structural coefficients allowed ranking the sub-dimensions as a function of their impact on the OC concept (Table 5). The sub-dimensions which had the greatest impact on OC concept were: “Support from the department head” (Structural Coefficient– SC: 0.223), “Relations with and between doctors” (SC: 0.159) and “taking account of the individual in the group” (SC: 0.125). Dimensions having the smallest impact on OC concept were: “Acceptance of department standards” (SC: 0.020), “Professional aims” (SC: 0.06) and “Discrimatory practice” (SC: 0.032).
Figure 1 corresponds to the SEM’s graphical representation linking the dimensions and the items of the questionnaire. The dimensions are latent variables represented by circles and the items are manifest variables represented by rectangle. For every sub-dimension, the standardized external weight by every item was calculated. The items having the greatest impacts on its dimension are presented in pink. All the latent variables converge on a central concept, the organizational culture. The structural coefficient (Reg) allowed ranking the sub-dimensions as a function of their impact on the organizational culture concept; the biggest were the one that had the greatest impact (colour shading).
The questionnaire which was administered at that stage included 83 items and 21 sub-dimensions. The participating department was a medicine ward with 19 professionals, 13 of whom responded twice. For 70 % of the items, reproducibility was average (25 items) to good (33 items). For three items, the ICC could not be calculated. Six sub-dimensions had more than half of their items with an ICC lower than 0.50, two of them were related to the perception of results of the running of the department (2b- “Likelihood of remaining on staff”, 2c(1)- “Workload”), three of them were department management components (4c- “Allocation of tasks and goals”, 4d- “Organisational training”, 4f(1)- “Frequency of conflicts between professionals”), and the sixth one was 5b(2)- “diffusion of information”.
The comprehensive construct, the one-dimensional character and the reliability of COMEt questionnaire suggest a good validity of the instrument for interventional research.
The high overall response rate of 80 % indicates its applicability and acceptability in interventional research conditions. The item selection process allowed the deletion of 52 items, resulting in a final version with 82 items. The challenge of questionnaire validation is to keep the questionnaire as short as possible while still including enough items to adequately measure the variables of interest. The validation results show that this challenge was successfully met.
Internal consistency was globally satisfactory. The retention of three entire scales (professional life satisfaction scale, intention to stay scale and professional burn out scale) has strengthened this psychometric property. Only eight sub-dimensions had Cronbach’s coefficients under 0.70, with five of them containing only two or three items; that is a factor of Cronbach’s coefficient decreasing. Furthermore, we expect those coefficient would increase after taking into account the final rewordings of the final version of the questionnaire (COMEt final version translated in English presented in Additional file 1).
Dillon–Goldstein’s rho coefficients higher than 0.70 for the sub-dimensions, indicate a high homogeneity of the instrument and more than 70 % of the items have a moderate to good reproducibility index.
COMEt construct followed a strict pattern covering a maximum number of OC dimensions and sub-dimensions. As a result of a comprehensive search of existing tools in literature, our instrument is the only one to cover all the OC components treated in the 14 retrieved OC questionnaires. Those 14 tools include most of the instruments retained by T Scott et al.  in their descriptive review of available instruments and they also appear in T Jung et al’s inventory of instruments for exploring OC .
With such a large number of OC dimensions, our OC questionnaire, provides, in addition to be an intervention assessment tool for research purposes, an operational tool for precise and comprehensive managerial and organizational diagnosis at ward level, thus allowing the identification of priorities of change to improve attitudes, effectiveness and performance. COMEt results have been indeed presented to staff and managers of several participating wards. The wards’ leaders mainly thought COMEt results were concordant with their perception of team climate in their ward and that they help them to adjust their managerial strategies.
M Sibe et al. published an analysis of data collected for COMEt validation that reinforces its internal and external validity . M Sibe et al. showed the discriminative capacity of COMEt instrument. The analysis of responses obtained for COMEt validation, allowed to distinguished two groups of wards: the wards with high development of all COMEt dimensions (9 wards on the 36 included, called “optimistic” wards) and the wards with low development of the COMEt dimensions (13 wards on the 36 included, called “pessimistic” wards). There was also a third group of 12 wards with intermediate results. One of the critical functions of a group’s culture is to establish a distinctive identity and thereby provide a means by which members of the group can differentiate themselves from other groups. COMEt showed its discriminative capacities, in accordance with this property of OC. Moreover, these results give arguments for an external validity of Comet. They are in favour of the strength of the existence of an organizational context that can induce either a positive or a negative effect both on management practices and attitudes; that is in coherence with magnet hospital theory change for management and the explanation of the link between OC and performance.
Questions may rise about the organizational issues assessed by COMEt. The first one concerns whether it really captures OC or rather organizational climate. COMEt investigates into layers 3 (artefacts) and 2 (values) of culture as defined by Schein. It is probably a poorly adapted tool for layer 1 assessment, that is the “assumption” level; actually that limitation appears in all the questionnaires, and it is thought to be a consequence of the questionnaire method . In their recent comparative literature review on organizational climate and culture, B Schneider et al. acknowledge that “what most quantitative measures of culture capture are the espoused values and or behavioural norms in organizations and not the full richness of the construct, including myths, stories, and socialization tactics” . Indeed, culture attempts to address deeper values and assumptions rather than the surface perceptions that are the focus of climate studies assessing specific process and content dimensions of behaviour. OC also emphasises that which is shared by group members rather than the diversity of individual perceptions that can make up climate . However, the theoretical distinction between culture and climate is not always upheld in practice. Schein has more recently characterized climate as providing the behavioural evidence for the culture of a setting, such that those behaviours form the bases for employees’ conclusions about the values and beliefs that characterize their organization. Culture and climate are now considered as two crucial, complementary and interrelated building blocks for organizational description and analysis . Our tool collects the shared participants’ views of the environment in which they work. In its main dimensions and sub-dimensions, it measures the shared values and artefacts of a department, or the “visible” or “expressive” part of the culture; it also captures specific process and content dimensions of behaviour, retrieved in some items of the dimensions “Management of the Department” or “Relation and communication in the department”, characterizing “climate”. This integration of climate and culture assessment has useful implications for practice and organizational changes and for research on the link between organization and performance.
A second question concerns the specificities of COMEt questionnaire in comparison with team work questionnaires. A recent review retrieved survey instruments used to assess dimensions of teamwork . All but one retrieved teamwork dimensions assessed by surveys are found in COMEt questionnaire. However, COMEt questionnaire contains supplementary dimensions and sub-dimensions relative to professionals’ attitudes and management practices (“Relations with the patient and his/her family”, “Type of behaviour encouraged in the department”, “Allocation of tasks and goals”, “Engagement at work”, “Discriminatory practices” and “Likelihood of remaining on staff”). There is clearly overlapping between COMEt OC assessment and team work assessment. However, COMEt specifically collects professional attitudes to work and managerial practices.
COMEt was validated within medicine wards for the TheOReM© project’s purposes. This specificity is a current limitation to the instrument. Validation of this questionnaire in other type of wards (surgery, other medicine specialities) is ongoing in France. The validation of the questionnaire in other health systems and other language is also needed. The trans language validation of the Hospital Survey on Patient Safety Culture questionnaire showed that a core set of dimensions consistently assessed as valid can be defined and measured in all countries and that some other should be adapted to each country according to local ways of being, thinking, behaving and communicating . The extent to which the COMEt questionnaire is specific to the French context can not be known without such translation validation. COMEt development included specificities of French health care system that would probably lead to exclude some items from a COMEt international common core set. From our comprehensive database of OC items and themes, were extracted and removed the items which were less relevant to a French context (mainly few items clearly oriented to productivity, items on fierce competition). Face validation led to some changes in item wording that are probably a direct consequence of French operational specificities. As example, contrary to what could be found in other instruments, conflict management procedures or quality of relationship between professionals were treated separately for medical doctors and paramedical professionals. In a transvalidation process across language and health care systems, attention should be put on these items. Moreover, items relative to productivity or competition should perhaps be added.
We did not study the sensitivity to change of the questionnaire. We aim to study this property as part of an ongoing research studying the impact of tools aiming to improve development of patient centred care at hospital using organizational and managerial changes and to analyze the individual and organizational determinants of their effectiveness. It focuses on 30 acute care hospitals that have the choice of the combination of tools and that are followed during two years with combined quantitative and qualitative analyses. That project uses the COMEt questionnaire for ward organizational context assessment. The comparison of the COMEt questionnaire dimensions scores before and after implementation of the tools and between departments implementing different tools will allow the analysis of COMEt questionnaire sensitivity to change.
The validation strategy was to proceed to exploratory and confirmatory analyses using the PLS (Partial Least Square) approach . Very few studies refer to a subsequent confirmatory analysis. The only example we could find was an Australian publication  which relates the use of SEM in the confirmatory phase, to validate an organizational culture measuring instrument. There are two different SEM types which permit confirmatory analysis but they differ in principle, purpose, rules and applications . Contrary to the Australian who adopted the LISREL (LInear Structural RELation) approach, we opted for the PLS approach in our study because it enabled us to predict individual dimension scores. It is the first step of the confirmatory analysis which will be completed by a LISREL approach trough a planned cross validation study including another sample from different wards specialties. This cross validation also will be able to test the performance of the reformulated items, particularly those of the sub-dimension “organizational learning”.
The SEM confirmatory analysis indeed gives distinctive and relevant information: quantifying items impact on its sub-dimension and ranking sub-dimensions impact on the organizational culture concept. The dimensions which had the strongest impact were “Support from the department head” followed by dimensions related to professional interaction and those related to well being within the group. Dimensions related to work, tasks and performances (workload, discriminatory practices, kind of behaviour encouraged in the department) had a lower impact weight on the organizational culture concept. The validation study of the French version of the Hospital Survey on Patient Safety Culture showed consistently that the three dimensions having the strongest impact on safety culture were: ‘Overall perceptions of safety’, ‘Hospital management support for patient safety’ and ‘Teamwork across hospital units’ .
From those findings one could draw the conclusion of a very emotional work perception culture, value is given to wellbeing within the group and to friendly climate. The dimension hierarchy as established by the analysis provides substantial information in favour of the hypotheses that changes into organizational culture are to be introduced by the management staff. In that type of medicine department, action must undoubtedly be focused on the group, on its internal harmony and on the integration of professionals within the group.
The COMEt questionnaire is a validated multidimensional OC questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first multidimensional OC questionnaire, specific to healthcare context, for a unit level assessment showing robust validity and reliability. This tool is suited for research purposes, especially for assessing organizational context in research analysing effectiveness of hospital quality improvement strategies . Our tool is also suited for an overall assessment of ward culture for team-training activities and could be a powerful trigger to improve management and clinical performance. Its psychometric properties in other health systems need to be tested.
Comité de Coordination de l’Evaluation Clinique et de la Qualité en Aquitaine
Contexte Organisationnel et Managérial en Etablissement de santé
Goodness of Fitness
Haute Autorité de Santé
LInear Structural RELations
Principal Component Analysis
Partial Least Square
Structural Equation Model
- SC TheOReM:
Organizational theOries, Recommendations and Management
Peters T, Waterman R. In search of excellence: lessons from America’s best run companies. New York: Harper And Rowe; 1982.
Deal T, Kennedy A. The new corporate cultures: Perseus. 1999.
Hofstede G. Attitudes, values and organizational culture: disentangling the concepts. Organ Stud. 1998;19:477–92.
Lega F, Prenestini A, Spurgeon P. Is management essential to improving the performance and sustainability of health care systems and organization? A systematic review and a roadmap for future studies. Value Health. 2013;16:S46–51.
Zazzali J, Alexander J, Shortell S, Burns L. Organizational culture and physician satisfaction with dimensions of group practice. Health Serv Res. 2007;42:3.
Gregory B, Harris S, Armenakis A, Shook C. Organizational culture and effectiveness: a study of values, attitudes, and organizational outcomes. J Bus Res. 2009;62:673–9.
Jacobs R, Mannion R, Davies H, Harrison S, Konteh F, Walshe K. The relationship between organizational culture and performance in acute hospitals. Soc Sci Med. 2013;76:115–25.
Scott T, Mannion R, Marshall M, Davies H. Does organizational culture influence health care performance ? A review of the evidence. J Health Serv Res Policy. 2003;8:105–17.
De Bono S, Heling G, Borg M. Organizational culture and its implications for infection prevention and control in healthcare institutions. J Hosp Infect. 2013;86:1–6.
Sinkowitz-Cochran R, Burkitt K, Cuerdon T, Harrisson C, Gao S, Obrosky D, Jain R, Mj F, Jernigan J. The associations between organizational culture and knowledge, attitudes and pracices in a multicenter veterans affairs quality improvement initiative to prevent Methicillin-resistant Staphylococcus Aureus. Am J Infect Control. 2012;40:138–43.
Kohn LT, Corrigan JM, Donaldson M. To Err is human: building a safer health system. Washington: National Academy Press; 2000. Institute Of Medicine.
Iom (Institute Of Medicine). Crossing the quality chasm: a new health system for the 21st century. Washington: National Academy Press; 2001.
Department Of Health. The Nhs plan: a plan for investment, a plan for reform. London: Department Of Health; 2000.
Department Of Health. Shifting the balance of power: the next steps. London: Department Of Health; 2002.
Schneider B, Ehrhart M, Macey W. Organizational climate and culture. Annu Rev Psychol. 2013;64:361–88.
Siehl C, Martin J. Organizational culture: A key to financial performance? In: Schneider B, editor. Organizational climate and culture. San Francisco: Jossey-Bass; 1990. p. 241–81.
Brazil K, Wakefield D, Cloutier M, Tennen H, Hall C. Organizational culture predicts job satisfaction and perceived clinical effectiveness in pediatric primary care practices. Health Care Manage Rev. 2010;35:365–71.
Spurgeon P, Mazelan P, Barwell F. Medical engagement: a crucial underpinning to organizational performance. Health Serv Manage Res. 2011;24:114–20.
Brown A. Organizational culture. London: Pitman; 1995.
Davies H, Nutley S, Mannion R. Organizational culture and quality of health care. Qual Health Care. 2000;9:111–9.
Braithwaite J, Hyde P, Pope C, editors. Culture and climate in healthcare organizations. New York: Palgrave Macmillan; 2010.
Martin J. Organizational culture: mapping the terrain. Thousand Oaks: Sage Publications; 2002.
Hofstede G, Neuijen B, Ohayv D, Sanders G. Measuring organizational cultures: a qualitative and quantitative study across twenty cases. Adm Sci Q. 1990;35:286–316.
Ashkanasy N, Lyndelle E, Broadfoot E, Falkus S. Questionnaire measures of organisational culture. In: Ashkanasy NM, Wilderom CPM, Peterson MFP, editors. Handbook of organisational culture and climate. Thousand Oaks: Sage Publications; 2000. p. 131–46.
Rousseau D. Assessing organizational culture: the case of multiple methods. In: Schneider B, editor. Organizational climate and culture. San Francisco: Jossey-Bass; 1990. p. 153–92.
Schein E. Organizational culture and leadership. 3rd ed. San Francisco: Jossey-Bass; 1995.
Smirich L, Calas M. Organizational culture: a critical assessment. In: Jablin F, Putman L, Roberts K, Porter L, editors. Handbook of organizational communications. Newbury Park: Sage; 1987. p. 228–63.
Quinn R, Rohrbaugh J. A special model of effectiveness criteria: toward a competing values approach to organizational analysis. Manage Sci. 1983;29:363–77.
Freeth D, Sandall J, Allan T, et al. A methodological study to compare survey-based and observation-based evaluations of organisational and safety cultures and then compare both approaches with markers of the quality of care. Health Technol Assess. 2012 May;16(25):Iii-Iv, 1–184. doi: 10.3310/Hta16250.
French B, Thomas L, Baker P, Burton C, Pennington L, Roddam H. What can management theories offer evidence-based practice? A comparative analysis of measurement tools for organizational context. Implement Sci. 2009;4:28. doi:10.1186/1748-5908-4-28.
Ehrhart M, Aarons G, Farahnak L. Assessing the organizational context for Ebp implementation: the development and validity testing of the Implementation Climate Scale (Ics). Implement Sci. 2014;9:157. doi:10.1186/S13012-014-0157-1.
Marchand A, Haines Iii V, Dextras-Gauthier J. Quantitative analysis of organizational culture in occupationsl health research: a theory-based validation in 30 workplaces of the organizational culture profile instrument. BMC Public Health. 2013;13:443. doi:10.1186/1471-2458-13-443.
Fava G, Ruini C, Rafanelli C. Psychometric theory is an obstacle to the progress of clinical research. Psychother Psychosom. 2004;73:145–8.
Fitzpatrick R, Davey C, et al. Evaluating patient-outcome measures for use in clinical trials. Health Technol Assess. 1998;2:1–64.
Wagner C, Mannion R, Hammer A, et al. The associations between organizational culture, organizational structure and quality management in European hospitals. Int J Qual Health Care. 2014;26 Suppl 1:74–80.
Secanell M, Groene O, Arah O, et al. Deepening Our understanding of quality improvement in Europe (Duque): overview of a study of hospital quality management in seven countries. Int J Qual Health Care. 2014;26 Suppl 1:5–15.
Jung T, Scott T, Davies H, Bower P, Whalley D, Mcnally R, et al. Instruments for exploring organizational culture: a review of the literature. Public Adm Rev. 2009;69:1087–96.
Heritage B, Pollock C, Roberts L. Validation of the organizational culture assessment instrument. PLoS One. 2014;9:E92879. doi:10.1371/Journal.Pone.0092879.
Minvielle E, Dervaux B, Retbi A, Aegerter P, Boumendil A, Jars-Guincestre M, et al. Culture, organization, and management in intensive care: construction and validation of a multidimensional questionnaire. J Crit Care. 2005;20:126–38.
Streiner D, Geoffrey R, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford: Oxford University Press; 2014.
Igalens J, Roussel P. Méthodes de recherche En gestion Des ressources humaines. Paris: Economica Ed; 1998.
Jacobowicz E. Pls path modeling and Pls regression with Xlstat. Paris. 2009.
Joreskog K. Latent variable scores and their uses. Lincolnwood: Scientific Software International Inc Ed.; 2000.
Jacobowicz E. Contributions Aux Modèles D’équations Structurelles A Variables Latentes, Thèse De Doctorat. De Paris: Conservatoire National Des Arts Et Métiers; 2007.
Tenenhaus M. L’approche Pls. Revue De Statistique Appliquée. 1999;47:40–5.
Shrout P, Fleiss J. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.
Pavot W, Diener E. Review of the satisfaction with life scale. Psychol Assess. 1993;5:164–72.
Fouquereau E, Rioux L. Élaboration De L’échelle De Satisfaction De Vie Professionnelle (Esvp) En Langue Française: Une Démarche Exploratoire. Can J Behav Sci. 2002;34:210.
Pezet-Langevin, Rolland J. Caractéristiques Des Situations De Travail, Burnout Et Attitude De Retrait. Eur Rev Appl Psychol. 1999;3:239–49.
Estryn-Behar M. Santé Et Satisfaction Des Soignants Au Travail En France Et En Europe, Presses De L’école Des Hautes Etudes En Santé Publique. 2008. p. 383.
Scott T, Mannion R, Davies H, Marshall M. The quantitative measurement of organizational culture in health care: a review of the available instruments. Health Serv Res. 2003;38:923–45.
Sibé M, Domecq S, Kret M, Saillour-Glénisson F. Contextes Organisationnels Et Managériaux Des Services Hospitaliers : Résultats D’une Enquête Transversale Auprès De 36 Services Au Moyen De L’outil Comet©. Journal De Gestion Et D’économie Médicales. 2012;L30:24–36.
Ashkanasy N, Wilderom C, Peterson M, editors. Handbook of organizational culture and climate. Thousand Oaks: Sage; 2000.
Valentine M, Nembhard I, Edmondson A. Measuring teamwork in health care settings. A review of sruvey instruments. Med Care. 2015;53:E16–30.
Occelli P, Quenon J, Kret M, et al. Validation of the French version of the hospital survey on patient safety culture questionnaire. Int J Qual Health Care. 2013;25:459–68.
Sarros J, Gray J, Densten I, Cooper B. The organizational culture profile revisited and revised: an Australian perspective. Aust J Manage. 2005;30:159–82.
Tenenhaus M. Comparison between Pls and Lisrel approaches for structural equation modeling: application to the measure of customer satisfaction. In: Vilare M, Tenenhaus M, Coelho P, Esposito V, Morineau A, editors. Pls and related methods –proceedings of the international symposium Pls’03. Decisia Ed. 2003. p. 111–26.
Kringos D, Sunol R, Wagner C, et al. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res. 2015;15:277. doi:10.1186/S12913-015-0906-0.
Members of the Theorem group: N Angélé Halgand, V Daucourt, S Domecq, J-P Dumont, M Kret, P Michel, V Migeot, L Moret, F Saillour-Glénisson, M Sibé, N Terrien, D Veillard.
This study has be in part funded by the French National Authority for Health (HAS).
Availability of data and materials
The dataset(s) supporting the conclusions of this article is (are) available on request to email@example.com or firstname.lastname@example.org.
FSG developed concept of research, designed methods, led research and edited manuscript. SD worked in conjunction with MK on design and concept, supported data gathering, realized data analyses and interpretation. MS and JPD developed concept of research, designed methods, supported data collection and participated to data interpretation. PM oversaw research, supported data collection and reviewed manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent to publish
Ethics approval and consent to participate
Theorem project fulfilled the requirements for research projects of the French National Authority for Health. Ethics approval was granted by the Bordeaux Ethic Committee. Each professional received oral and written information about the study delivered during a meeting organized in each participating ward and gave verbal consent to participate.
COMEt questionnaire in English. Description of data: Presentation of the final version of the COMEt questionnaire, translated from French to English. (DOC 120 kb)