You are viewing the site in preview mode

Skip to main content

Advertisement

Table 1 Studies utilizing the Delphi technique to develop content for checklists used in clinical settings

From: Developing content for a process-of-care checklist for use in intensive care units: a dual-method approach to establishing construct validity

Study Sample Purpose / Method Findings* / Critique
Setting n / cohort
Huang, Lin & Lin. (Taiwan) [13] College of Nursing 14 / 20 invited panel members accepted; 10 scholars in relevant fields of expertise, 4 clinical nurses. To develop content for a fall-risk checklist Framework presented to panel who were asked to review a 4-point Likert scale checklist (from strong agreement to strong disagreement), submit comments & provide revision suggestions Likert scale used to calculate content validity index (CVI) score for each item, rated along 3 dimensions i.e. content importance, appropriateness and discreteness Scoring calculation method detailed 70% of potential panel members accepted, 3 rounds required, completed over 4-month period Response rates: round 1, 78.5% (3 withdrew); 2, 91% (1 withdrew); 3, 100% Results of each round reported in summarized format Key suggestions & resulting refinements for each round provided Changes to domains and checklist processes documented CVI scores for each domain along the 3 dimensions and total score (range 0.84 – 1.00) in last review round provided Information not provided: complete checklist, criteria for deleting items, variation in responses & scores to individual items (results summarized by domain)
Morgan et al. (Canada) [19] 2 independent academic centers 5 anesthesiologists To develop a simulation performance checklist to evaluate performance of practicing anesthesiologists, using a computer-based Delphi technique Checklist items generated by participants after reading 2 pre-prepared scenarios, error weighting assigned to each item based on risk level Responses collated anonymously & emailed back to participants asking them to check off items to retain or delete & to (re)assign weightings Process repeated until no further items added, deleted or changes to weightings A-priori decision to delete responses endorsed by ≤ 20% respondents 100% response rate Required four rounds to reach consensus Participants generated 104 items for scenario 1 & 99 items for scenario 2 Final percentage weightings for checklist items provided Small sample size Information not provided: variation in error weighting to individual items, key study timeframes e.g. time from survey distribution to response
Hart & Owen. (Australia) [17] Anesthesia Department at a tertiary hospital Not reported - consultants with special interest in obstetric anesthesia To generate checklist items for use prior to commencing non-emergency Cesarean delivery under general anesthesia Participants contacted via email and remained anonymous to other participants Two questionnaires were circulated Two questionnaires were circulated Results of 2 questionnaires informed construction of checklist items Items were later divided into four sub-categories Key information not reported: sample size; contents of questionnaires; response rates; how responses were used to inform 2nd round questionnaire & construct final checklist items e.g. not known whether pre-defined consensus methods were used, how checklist items were grouped & ordered
Ursprung et al. (USA) [16] 20-bed tertiary care medical-surgical neonatal ICU Not reported - experts in neonatology, pediatrics, health services research, systems engineering, infection control, advanced practice nursing To develop a patient safety audit checklist for PICUs Questions formatted into a checklist and refined iteratively by consensus Participants responses based on potential clinical impact of mistakes, system failures, perceived frequency Checklist reviewed and refined by physicians and nursing staff from study NICU to ensure relevance locally 36 audit questions representing a broad range of errors associated with NICU patient care generated Questions later divided into 2 categories Information not reported: sample size and participant designations; contents of questionnaire; number of rounds required; method of obtaining consensus; how checklist items were further reviewed and refined for relevance by local PICU staff after consensus was reached; method of categorization
Pronovost et al. (USA) [18] 13 adult medical & surgical ICUs in urban teaching & community hospitals Interviews: 8 nurses & 5 ICU physicians Focus group: not reported Development and pilot testing of daily goals form Validity of measures: obtaining agreement from ICU physicians and quality experts who developed the measures; semi-structured interviews with nurses & physicians who piloted the measures Face validity: focus group of physicians and nurses from 13 participating ICUs Validity of measures: ICU physicians and quality experts unanimously agreed process measures addressed important aspects of ICU quality Focus group: participants believed measures ‘evaluated the domain of quality they intended to measure and identified important opportunities to improve quality’ [18], p.154 Information not provided: sample sizes for development of measures and focus group; content for focus group discussion & semi-structured interviews; how qualitative data analyzed and interpreted