Research article | Open | Open Peer Review | Published:
Cross-cultural validation of health literacy measurement tools in Italian oncology patients
BMC Health Services Researchvolume 17, Article number: 410 (2017)
The aim of this study was to assess the psychometric characteristics of four Health Literacy (HL) measurement tools, viz. Newest Vital Sign (NVS), Short Test of Functional Health Literacy in Adults (STOFHLA), Single Item Literacy Screener (SILS) and Single question on Self-rated Reading Ability (SrRA) among Italian oncology patients.
The original version of the tools were translated from the English language into Italian using a standard forward-backward procedure and according to internationally recognized good practices. Their internal consistency (reliability) and validity (construct, convergent and discriminative) were tested in a sample of 245 consecutive cancer patients recruited from seven Italian health care centers.
The internal consistency of the STOFHLA-I was Chronbach’s α=0.96 and that of NVS-I was α=0.74. The STOFHLA-I, NVS-I, SILS-I and SrRA-I scores were in a good relative correlation and in all tools the discriminative known-group validity was confirmed. The reliability and validity values were similar to those obtained from other cultural context studies.
The psychometric characteristics of the Italian version of NVS, STHOFLA, SILS and SrRA were found to be good, with satisfactory reliability and validity. This indicates that they could be used as a screening tool in Italian patients. Moreover, the use of the same cross-cultural tools, validated in different languages, is essential for implementing multicenter studies to measure and compare the functional HL levels across countries.
In 2003, the National Assessment of Adult Literacy defined Literacy as “the ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential” . “Literacy is content and setting specific. An individual may have adequate understanding of material with familiar content, but struggle to comprehend information with unfamiliar vocabulary and concepts” . The operational definition of Health Literacy (HL), developed for the National Library of Medicine, describes HL as “the degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions” . Numerous studies have shown that an inadequate HL level is associated with inadequate understanding of written information, lack of medical information, less use of preventive measures, lower medication compliance, impaired health care knowledge, poor health-related outcomes and higher health care costs . Appropriate and valid HL measurement tools are essential in research to evaluate the effect of HL on several health outcomes, such as healthcare use, and the effectiveness of preventive healthcare and targeted interventions . Furthermore, they are useful in clinical practice to improve communication with patients. In the literature, several tools have been proposed in research and clinical settings  with different characteristics, validity and reliability history. Most of these tools have been developed in the Anglo-Saxon context and a cross-cultural process of implementation in other languages and cultures is important. This issue is a crucial aspect for the multicenter collaborative Italian Cancer Patient Education Group project whose aims are: to map patient education (PE) activities in the different Italian cancer research and care institutes , to survey the unmet information and communication needs, and to implement PE programs and activities with the involvement of patient representatives in each phase . Thus, validating the tools for measuring functional HL among Italian cancer patients is a key step of our research, because knowing the level of functional HL is essential for all PE activities.
To date, several instruments have been proposed to measure HL, but none of them have been sufficiently validated for the Italian population. The Newest Vital Sign (NVS), the Short Test of Functional Health Literacy in Adults (STOFHLA) and the Single Item Literacy Screener (SILS) are commonly used in HL research. The Italian Cancer Patient Education Group carried out this study, aiming to translate and to cross-culturally adapt the original English version of these HL measurement tools for Italian oncology patients.
An observational cross-sectional study was conducted within the Italian Cancer Patient Education Group between November 2015 and February 2016. The data was collected from patients attending seven different hospitals in Italy, three in Northern Italy, three in the Centre and one in the South. Consecutive adult cancer patients, aged 18–65 years, were enrolled after being approached by trained research assistants in the waiting rooms of the oncology, surgery or day-hospital wards. Patients were included if they were cognitively able to understand and complete the survey. Language proficiency and cognitive functioning was judged by the research assistant upon inclusion. According to study protocol, the minimum sample size was calculated as at least 5 patients for each questionnaire item of the longest questionnaire . Therefore, for our study aims, at least 200 patients were required, based on STOFHLA (40 items). Since the catchment area was different for each centre, there was no competitive recruitment.
The study was approved by the Ethics Committee of each participating center, eg: CRO Aviano National Cancer Institute -IRCCS, Aviano; Arcispedale Santa Maria Nuova - IRCCS, Reggio Emilia; Azienda Ospedaliero-Universitaria Ferrara, Ferrara; Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori (IRST)- IRCCS, Meldola(FC); Istituto Tumori “Giovanni Paolo II” IRCCS, Bari; Veneto Institute of Oncology IOV- IRCCS, Padua; IRCCS National Cancer Institute of Milan. The patients received detailed information about the aims and procedures of the study, and were enrolled once their written informed consent was obtained according to international standards.
The study included four Health Literacy measurement tools:
The NVS, developed by Weiss et al. , is a verbally administered 6-item tool that asks about information contained in a standard ice cream nutrition label. It requires the ability to interpret the food table, and to answer six questions related to health. The NVS was developed as a rapid and accurate HL screening test. The average time taken to conduct this test is about 6 min.
The STOFHLA, developed by Baker et al. , is a 40-item questionnaire divided into two sections. The first section includes four verbal numeracy questions, requiring interpretation of medication instructions, test results and an appointment date. The second section includes 36 self-administered items regarding the preparation for a radiological examination (passage A) and a health administrative rights management section (passage B). The compilation of this test requires an average of about 12 min.
The SILS, developed by Morris et al. , is a single-item test designed to measure the need for help with reading health-related materials. The item is administered verbally: “How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy?” The patient is asked to choose one of the following answers on a 5-point Likert scale: Never, Rarely, Sometimes, Often, Always.
The SrRA, developed by Jeppesen et al. , is a single- item test related to self-rated reading ability. The item is administered verbally: “How would you rate your ability to read?” A participant is asked to choose one of the following answers on a 5- point Likert scale: Excellent/ Very good, Good, Okay, Poor, Terrible/Very poor.
The translation and cross-cultural adaptation of the original English SILS, NVS, STOFHLA and SrRA, were performed using a standard forward-backward procedure and according to recognized good practices [14, 15]. A pilot study was conducted with 15 patients to assess any ambiguous statements or questions and to make sure the Italian forms were understandable. The final translation form was refined after the feedback from the pilot study and after an agreement among the research group leaders.
During the survey, the translated tools were administered consecutively in the following order: SILS-I, SrRA-I, STOFHLA-I and NVS-I. Since SILS-I and SrRA-I investigate the self-perception of reading ability, they were administered before the STOFHLA-I and NVS-I to avoid any influence on patients’ awareness of reading and numeracy skills. The administration process was standardized among the centers and the interviewers were trained centrally.
The research assistants administered the HL measurement tools to all eligible and consenting patients and recorded their responses. Demographic data such as age, gender, educational level and marital status were collected during the interview.
Reliability analyses were conducted for STOFHLA-I and NVS-I using Cronbach’s alpha, which represent the internal consistency of the items. To estimate the test-retest reliability, the split-half procedure was applied using the equal-length Spearman-Brown formula.
Construct validity was also conducted by computing correlations (Spearman coefficient) between NVS-I and STOFHLA-I, and among STOFHLA-I sections. Known-group validity was tested comparing NVS-I and STOFHLA-I mean scores across age groups, educational level, sex, marital status and geographic area using the analysis of variance (ANOVA). Statistical significance was claimed for p<0.05. All analyses were performed using SAS 9.2 software.
A total of 248 patients were enrolled. Three patients were not able to complete the survey and were excluded from the analysis, thus leaving 245 eligible patients. Patients were aged 18–65 years (median: 54 years) and they were predominantly females (n=153; 62.4%) (Table 1). The majority had a high school education level (i.e. 11–13 years of schooling; n=124, 50.6%), whereas only 16 patients (6.5%) reported <6 years of schooling. Patient answers to SILS-I and SrRA-I questionnaires and the score of the NVS-I and STOFHLA-I items that were completed correctly are reported in Table 2.
Chronbach’s alpha coefficient was 0.96 for the overall STOFHLA-I, 0.45 for STOFHLA-I/Numeracy, and 0.96 for STOFHLA-I/Reading sections. The Spearman correlation between Numeracy and Reading was 0.38. The internal consistency of the NVS-I was 0.74. Test-retest reliability was evaluated through the split- half procedure, reporting satisfactory results for both the STOFHLA-I (r-Spearman=0.59) and NVS-I (r- Spearman=0.65).
Construct validity was evaluated by computing discriminative known-group validity analysis as reported in Table 3, and criterion validity by correlation between the tool analyses (Table 4). Significant differences in STOFHLA-I, NVS-I, SILS-I and SrRA-I scores were found between patients of different age groups (p<0.05), educational level (p<0.01), and geographic area (p<0.01). For all tools, no relationship was found with sex or marital status. Given the lack of a validated tool for the assessment of HL in Italy, external validity was evaluated by correlating each HL instrument with the other three.
The STOFHLA-I reported a good correlation with NVS-I (r=0.58), and the score demonstrated a significant positive correlation with the two questionnaires related to reading ability SILS-I (p<0.01) and SrRA-I (p<0.01). The NVS-I score showed a significant correlation with SrRA-I (p<0.01), and borderline with SILS-I score (p=0.06). The correlation between SILS-I and SrRA-I was significant (< p=0.01).
The purpose of this study was to translate HL measurement tools into Italian, assess their cross content validity, and examine their validity and reliability in a sample of oncology patients. Our results showed that the instruments were sufficiently reliable in the Italian culture, showing psychometric properties similar to those reported in other trans-cultural studies.
The internal consistency value of 0.74 of the NVS-I was good and comparable to data obtained in other cultural settings. Indeed, the values reported in other cultures were: 0.74 in United Kingdom ; 0.72 in Japan ; 0.76 in The Netherlands ; and 0.76 in English-speaking Americans . Slightly lower values was reported in a USA Spanish-speaking group (α=0.69) . The internal consistency of the STOFHLA-I was very good for all items included. The instrument measures both text comprehension (reading) and numeracy skills. Once the Reading and the Numeracy sections were evaluated separately, only Reading showed good consistency, while Numeracy gave a poorer result. This was also reported in the original validation paper by Baker and colleagues , and was even lower in the German language version in Switzerland (α=0.33) . The reason for the weakness of the STOFHLA-I/Numeracy internal consistency could be due to the fact that the few items of these sections, only four, may assess non-homogeneous aspects of HL.
The validity was assessed by the correlations within and between the HL tools. In the analysis of criterion-related (convergent) validity, the score of each instrument was compared with all the others. The STOFHLA-I total score reported a good correlation with the total NVS-I, with the SILS-I and the SrRA-I. This suggests that one dimension of the instrument correlates appreciably with other instruments investigating the same postulated area.
The correlation that we found between the NVS-I and the STOFHLA-I (r=0.58) was analogous to the Iraqi version (r=0.51) . Also, it was comparable to the correlations between the NVS and the similar Test of Functional Health Literacy in Adults (TOFHLA) used in Weiss’ USA version (r=0.59)  and Rowlands’ UK version (r=0.49) . The total NVS-I score did not demonstrate a significant correlation with the SILS-I score. It was once again analogous to the result of the Iraqi NVS version. The NVS-I and STOFHLA-I demonstrated a significant positive correlation with the SrRA-I single item “How would you rate your ability to read?” In our sample, this would suggest that there is a match between what patients think they know and what they really are able to do. Also, Jeppesen et al.  reported a similar observation, concluding that “self-rated reading ability was the single most reliable predictor of limited HL of the predictors tested”.
The four tools under investigation showed a satisfactory discriminatory capacity. For all the instruments, the discriminative known-group validity was confirmed, showing significant correlations with age and educational level. Conversely, no correlation was found with gender or marital status.
Some limitations have to be acknowledged. First, the lack of a validated questionnaire for health literacy in Italian did not allow testing external validity through the comparison with a gold standard. Nonetheless, we cross-tested the four instruments, finding good agreement. Further, the oncologic setting may have impacted on patients’ participation and on survey completion. Indeed, the approached patients may have had cancer-related fatigue, which impacted on refusal rate, and survey could have been interrupted by patient call to visit or treatment. However, only three patients refused study participation (refusal rate: 1.2%) and three additional patients (1.2%) interrupted the survey (1 for fatigue and 2 for calling to visit/treatment). The inclusion of study centers from all parts of Italy should be accounted among the study strengths. Indeed, this guaranteed that study patients were a representative sample of the Italian population. This also allowed the evaluation of the geographic variation in health literacy. Sample size should also be acknowledged among study strengths, since it guaranteed an adequate power.
Implications for practice and further research
The psychometric characteristics of the Italian translations of NVS, S-THOFLA, SILS and SrRA were found to be acceptable with satisfactory reliability and validity, which indicate that they can be used as a screening tool in Italian patients. The findings are useful for planning and re-building health services with a patient-centered approach.
Moreover, the use of the same cross-cultural tools, validated in different languages, is essential for implementing multicenter studies to measure and compare the functional HL levels across countries.
This study examined cross-culturally adapted tools for assessing objective HL in the Italian oncology context. The psychometric characteristics of the Italian translations of STOFHLA, NVS, two single-item SILS and SrRA were found to be good. The reliability and validity values were similar to those obtained from other cultural context psychometric studies. It means that they can be successfully used as a screening tool in Italian patients. The findings are useful for planning and re-building health services with a patient-centered approach. Moreover, the use of the same cross-cultural tools, validated in different languages, is essential for implementing multicenter studies to measure and compare the functional HL levels across countries.
U.S. Department of Education-Institute of Education Sciences - National Center for Education Statistics. Definition of Literacy. [Internet]. 2003; Available from: http://nces.ed.gov/naal/fr_definition.asp.
Selden CR, Zorn M, Ratzan S, Parker RM. Current Bibliographies in Medicine.Health Literacy. US Dep. Health Hum. Serv. Natl. Inst. Health Natl. Libr. Med. [Internet]. 2000; Available from: http://www.nlm.nih.gov/archive/20061214/pubs/cbm/hliteracy.pdf.
Health Literacy | National Network of Libraries of Medicine [Internet]. 2010 [cited 2016 Nov 2] Available from: https://nnlm.gov/outreach/consumer/hlthlit.html#Definitions_of_Health_Literacy.
Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.
Fransen MP, Leenaars KEF, Rowlands G, Weiss BD, Maat HP, Essink-Bot M-L. International application of health literacy measures: Adaptation and validation of the newest vital sign in The Netherlands. Patient Educ Couns. 2014;97:403–9.
Haun JN, Valerio MA, McCormack LA, Sørensen K, Paasche-Orlow MK. Health literacy measurement: an inventory and descriptive summary of 51 instruments. J. Health Commun. 2014;19(Suppl 2):302–33.
Cipolat Mis C, Truccolo I, Ravaioli V, Cocchi S, Gangeri L, Mosconi P, et al. Making patient centered care a reality: a survey of patient educational programs in Italian Cancer Research and Care Institutes. BMC Health Serv Res. 2015;15:298.
Truccolo I, Mis CC, Cervo S, Maso LD, Bongiovanni M, Bearz A, et al. Patient-Centered Cancer Care Programs in Italy: Benchmarking Global Patient Education Initiatives. J Cancer Educ. 2015;31:405–12.
Munro BH. Statistical methods for health care research. vol. 1 BH Munro. 5th ed. Philadelphia: Lippincott Williams & Wilkins; 2005.
Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick Assessment of Literacy in Primary Care: The Newest Vital Sign. Ann Fam Med. 2005;3:514–22.
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38:33–42.
Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: Evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract. 2006;7:21.
Jeppesen KM, Coyle JD, Miser WF. Screening Questions to Predict Limited Health Literacy: A Cross-Sectional Study of Patients With Diabetes Mellitus. Ann Fam Med. 2009;7:24–31.
Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterology. 2004;126:S124–8.
Acquadro C, Conway K, Hareendran A, Aaronson N. Literature Review of Methods to Translate Health-Related Quality of Life Questionnaires for Use in Multinational Clinical Trials. Value Health. 2008;11:509–21.
Rowlands G, Khazaezadeh N, Oteng-Ntim E, Seed P, Barr S, Weiss BD. Development and validation of a measure of health literacy in the UK: the newest vital sign. BMC Public Health. 2013;13:116.
Kogure T, Sumitani M, Suka M, Ishikawa H, Odajima T, Igarashi A, et al. Validity and Reliability of the Japanese Version of the Newest Vital Sign: A Preliminary Study. PLoS One. 2014;9:e94582.
Connor M, Mantwill S, Schulz PJ. Functional health literacy in Switzerland—Validation of a German, Italian, and French health literacy test. Patient Educ Couns. 2013;90:12–7.
Al-Jumaili AA, Al-Rekabi MD, Sorofman B. Evaluation of instruments to assess health literacy in Arabic language among Iraqis. Res Soc Adm Pharm. 2015;11:803–13.
The authors would like to thank: Barry D. Weiss, David W Baker, Nancy S Morris, Kelly M Jeppesen for their consent to translate their Health Literacy measurement tools; Danilo Orlandini, Reggio Emilia, and Renato Talamini, Aviano, for their intellectual and statistical contribution to the study design of this validation work; Mauro Mazzocut, Aviano for his informatics support; Anna Vallerugo, Aviano, for her linguistic support in the cross cultural translation of the questionnaires; Anna Maria Colussi, Aviano, for her linguistic revision of the manuscript.
ICPEG, Italian Cancer Patient Education Group, include the following members participating in this study as interviewers: Marilena Bongiovanni, President of the ANGOLO-National Association of Oncology Survivors Patients Onlus; Marta De Conti, ANGOLO Onlus; Francesca Agostina Masutti, ANGOLO Onlus; Silvia Flora, Radiotherapy, CRO Aviano National Cancer Institute - IRCCS, Aviano; Giulia Fornasier; Pharmacy Clinical Desk, CRO Aviano National Cancer Institute, Aviano; Sara Francescon, Pharmacy Clinical Desk, CRO Aviano National Cancer Institute, Aviano; Valentina Tomasi, Press Office, Azienda Ospedaliero-Universitaria-Ferrara; Carla Bagnacani, Medical Library, Arcispedale Santa Maria Nuova - IRCCS, Reggio Emilia; Angela Nardi, Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori (IRST) IRCCS, Meldola(FC); Cristina Nanni, Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori (IRST) IRCCS, Meldola(FC); Petrina Mariana Braghesiu, Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori (IRST) IRCCS, Meldola(FC), Italy.
This study was funded by the RF-2010-2308141 Multicenter Collaborative Project 2013-2015 “Extending Comprehensive Cancer Centers Expertise in patient education: the power of partnership with patient representatives”, granted by the Ministry of Health, Italy.
Availability of data and materials
The most important data of this manuscript are the Health Literacy tools translated and adapted into Italian.
The raw data related to this study will be deposited in the CRO OpenDocuments Repository that is the CRO Aviano institutional repository. They will be accessible at a specific permanent URL related to this publication, immediately created after the publication of this manuscript and the archiving into the CRO OpenDocuments Repository available at the link http://opendocuments.cro.it/cod/.
PZ contributed to the study design regarding the psychometric aspects, as research coordinator of the multicenter validation study, as instructor of the interviewers’ Team, as referent of the Aviano Team of interviewers. Then to the data input coordination and analysis, and to the writing of the draft manuscript. SC contributed to the conception of the Health Literacy (HL) study research with the literature search and the background of the study. Then he coordinated with CCM the translation/back translation and cross cultural adaptation of each tool. Furthermore, he contributed to the interviewing phase for the Reggio Emilia Centre and to the reviewing of the draft manuscript. JP contributed to the statistical aspects of the validation study design, to the acquisition, analysis and interpretation of data. CCM contributed since the beginning to the conception of the Health Literacy (HL) study with the search and comparison of tools for measuring HL in Italian language. Then CCM coordinated with SC the translation/back translation and cross cultural adaptation ‘activities. Furthermore she performed the study design and its presentation to the CRO Ethical Committee and coordinated the pilot phase at the CRO and contributed to the interviewing phase at the Aviano Centre. DB participated as co-coordinator of the HL validation study at AOU Ferrara with Federica Fiorini and as manuscript reviewer. SC contributed to the statistical aspects of the validation study design and specifically to the discussion related to the choice of this set of tools. AC participated as coordinator of the HL validation study at IRST Meldola (FC) and as interviewer and data coordination for IRST Meldola (FC) and manuscript reviewer. CC participated as co-coordinator of the HL validation study at “Giovanni Paolo II” Bari, and as interviewer and data coordination for this Institute and manuscript reviewer. MDG participated in the interviewing phase at IOV Padua and giving critical suggestions to the discussion part of the manuscript. CAD participated as coordinator of the HL validation study at IOV Padua. Then she participated in the back translation and cultural adaptation of the tools and in the linguistic revision of the manuscript. FF participated as co-coordinator of the HL validation study at AOU Ferrara with Donato Bragatto and as manuscript reviewer. LG participated as coordinator of the HL validation study at INT Milano and as interviewer and data acquisition for INT Milano and giving intellectual contribution to the manuscript. AL participated as coordinator of the HL validation study at “Giovanni Paolo II” Bari, and as interviewer and data coordination for this Institute and manuscript reviewer. RM participated in the interviewing phase at IOV Padua and giving intellectual comments to the manuscript. PM participated since the first phase of the HL validation research with suggestions and comments as well to the reviewing phase of the manuscript with substantial suggestions and comments. AP contributed as head partner at Oncologico Bari of the global project of Patient Education – of which the HL validation study is part. He participated in the study conception. VR participated as co-coordinator of the HL interviewers at IRST Meldola (FC) and manuscript reviewer. IT, corresponding author, participated in the HL study starting from the study design, choice of the tools, translation and cultural adaptation’s activities, presentation to the Ethical Committee at CRO, pilot phase, composition of different point of view about the Italian version of the tools. IT contributed to the manuscript writing with specific regard to the Implications for practice. ICPEG members (see list in the Acknowledgements section and in the Additional file 6) contributed as interviewers and facilitators of the present study in their Institutions. PDP, as Principal Investigator of the Patient Education project of which the HL validation study is part, gave substantial contribution to every phase of the study: the conception and design of the global project and study, the presentation to the Ethical Committee, the interpretation of data, the drafting the article, the revision and final approval of the submitted version. All authors have approved the final version of the manuscript.
The authors declare that they have no competing interests.
Consent for publication
“Not applicable” as this manuscript doesn’t contain any individual person’s data in any form.
Ethics approval and consent to participate
The study was approved by the Ethics Committee of each participating center. The patients received detailed information about the aims and procedures of the study, and were enrolled once their written informed consent was obtained according to international standards.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Newest Vital Signal(SILS)_Italian version. (PDF 301 kb)
Single Item Literacy Screener-Italian version. (PDF 73 kb)
Single question on Self-rated Reading Ability(SrRA)_Italian version. (PDF 72 kb)
Short-Text of Functional Health Literacy in Adults(S-TOFHLA) Numeracy Section_Italian version. (PDF 729 kb)
Short-Text of Functional Health Literacy in Adults(S-TOFHLA) Reading Section_Italian version. (PDF 110 kb)
Italian Cancer Patients Education Group(ICPEG)_investigators research. (DOCX 10 kb)
About this article
- Health literacy
- Cross-sectional studies
- Cross-cultural adaptation
- Patient education & empowerment