Construct Validity and Reliability of a Knowledge, Attitude, Perception, and Behaviors on Dietary Practices Questionnaire for School-Age Children in Panama

Objective: The study aimed to evaluate the psychometric properties of a Spanish questionnaire to assess nutrition-related knowledge, attitudes, perceptions, and behaviors (KAPB) for use among school-aged children aged (6 - 10 years) in Panama.

Methods: The questionnaire development and construct validation consisted of a four-stage process-first, conceptualization and construction of the instrument, primarily based on Panama’s Food-Based Dietary Guidelines. Second, assessing content validity with expert reviews. Third, piloting the questionnaire with a small group of respondents (n=30); and, fourth, psychometric properties, reliability considering internal consistency (alpha Cronbach) and construct validity with confirmatory factor analysis (CFA). For the psychometric testing, the sample size included 248 school-age children of 8-13 years old, from five public primary schools in Panama West Province.

Measurable Outcome: A questionnaire with construct validity, reliable, and culturally appropriate to assess healthy eating related KAPB in school-age children in Panama.

Results: The adequacy of the sample was acceptable of 0.8052, determined by The Kaiser-Meyer-Olkin index. The CFA yields four factors related to dietary practices, with a high internal consistency (Cronbach alpha > 0.8).

Conclusion: This study provides a tested questionnaire with acceptable psychometric properties. The questionnaire contributes to a questionnaire to measure healthy eating habits and lifestyles in school-age children in Panama within the context of multi-sectoral obesity prevention programs.

Keywords: Reliability; Aptitude; Behavior; Nutrition Education; Health Policy

List of abbreviations: KAPB: Knowledge, Attitudes, Perceptions, and Behaviors; lFBDG: Food-Based Dietary Guidelines; FNE: Food And Nutrition Education; KMO: Kaiser-Mayer-Olkin index; EFA: Exploratory Factor Analysis; CFA: Confirmatory Factor Analysis; BAZ: Body Mass Index Z Score

According to the World Health Organization (WHO), overweight and obesity are serious health problems related to an excessive accumulation of body fat, resulting in an imbalance between consumption and energy expenditure [1]. Overweight and obesity represent a severe public health problems worldwide affecting all age groups [2]. However, this definition is limited when considering these conditions under an individualistic approach. Therefore, an environment-centered approach allows a complete study of the phenomenon and its potential solutions. In school-aged children (aged 5-11 years) of Latin American countries, the prevalence of overweight is from 18.9% to 36.9%, which is equivalent to 22.2 to 25.9 million affected school-age children [3].

School-age is a critical period for the acquisition, modification, and reinforcement of habits, attitudes, and behaviors for a healthy diet that will persist into adulthood [4]. Various studies from different countries indicate that nutritional interventions in schools are adequate to prevent overweight and obesity [5-8]. However, diet is also influenced by the quality of local health and food systems [9], particularly those defining the obesogenic environments and indirectly changing attitudes and behaviors towards food at the population level [10,11]. Currently, the global food system is characterized by the greater availability of highly processed products that provide excess sugars, salt, and fats, as well as the decreasing demand for healthy foods, for example, fruits, vegetables, legumes, nuts, and seeds [12,13].

Although overweight and obesity are recognized as serious public health problems affecting a large proportion of the population in the Latin America region [3,13], studies on the attitudinal and behavioral aspects of the problem are not as abundant as in other settings such as the United States of America. The most substantial proportion of studies conducted in Panama have described the nutrition environment at large or examined nutritional biomarkers more specifically [14,15]. The lack of social and behavioral research may be due in part to a paucity of valid and reliable instruments available for measuring important social and behavioral factors underlying nutrition outcomes, especially among school-aged children. After all, healthy dietary practices should also focus on valid and reliable psychometric measurement of eating habits and lifestyle choices [16].

Although there are different methods to assess attitude, perception, and behavior change against obesity in schoolchildren, there are no instruments to inform policy processes aligned with the national food-based dietary guidelines and with cultural relevance [17]. The objective of this study is to design a construct valid and reliable questionnaire in Spanish to assess nutrition-related knowledge, attitudes, perceptions, and behaviors (KAPB) for use among school-aged children aged (8 - 13 years) in Panama. The study is part of an overarching project seeking to develop a multi-sectoral public policy for effectively addressing school-aged overweight and obesity in Panama. Such a valid questionnaire was needed to accurately assess nutrition-related KAPB among school-aged children who participate in food and nutrition education (FNE) programming in this context.

The formative work had a two-phase, iterative design and was conducted from June-December 2018. Phase 1 included the process of planning and developing the questionnaire to evaluate KAPB in school-age children, based on the review of the national and international literature and the food-based dietary guidelines (FBDG) [18] using internet-based tools. Phase 2 involved the validation of the measurement instrument.

The 25 knowledge items of the initial questionnaire were established according to Panama’s FBDG for the general population [18]. Using an environment-based behavior change approach [19], the items assessing attitude, perception, and self-reported behavior focused on content related to dietary habits and healthy lifestyles, including consumption of fruits, vegetables, water, school snacks, and physical activity. The first version of the questionnaire was developed and consisted of 67 items after combining those four sections. The research team included nutritionists and psychologists.

Likert scales for the attitudinal and perception questions ranged from 1 -5, for example, "Healthy condiments make food that is eaten at home and school healthier," the Likert scale was (1 = entirely disagree, 2 = disagree, 3 = indecisive, 4 = agree, and 5 = agree entirely). For self-reported behavior questions, for example, "I wash my hands before eating," the Likert scale was also used a 1 – 5 scale (1 = never, 2 = rarely, 3 = sometimes, 4 = almost always and 5 = always). The scales of attitude, perception, and behavior measurement were based on criteria described by Morales-Vallejos (2011) [20].

From the 42 attitude, perception, and behavior questions on the first instrument, only just 37 remained after the review. The knowledge section consisted of 14 multiple-choice questions where one choice was correct. The attitude section consisted of 17 items, with a maximum score value of 85 points, the perception and behavior section of 10 items each, with a maximum score value of 50 points.

Step 1: Expert Review Of Questionnaire Content: Six peer judges were chosen as peer reviewers with experience in FNE to evaluate the questionnaire content validity. The peer judges were from the School of Nutrition and Dietetics of the University of Panama and the Food and Agriculture Organization of the United Nations (FAO). Evaluation form, along with the questionnaire, was sent to the reviewers, and two weeks were given to complete the evaluation. Those judges proposed suggestions and recommendations for changes to the questionnaire content, with a focus on strategies for improving the clarity, pertinence, wording, and relevance of each item. An evaluation matrix was prepared for reviewers to use in their assessment and feedback (Table 1).

Four of the six reviewers provided feedback on questionnaire content. From 25 knowledge questions on the first instrument, only just 14 remained after the review. For the other three sections, from 42 questions on the primary instrument, only just 37 remained after the review.

Step 2: A First Pilot Test of Questionnaire Questions (n = 30 Students): To triangulate those reviewer recommendations, we conducted a pilot evaluation with the original 67-question in a Spanish questionnaire using a convenience sampling of 30 school-aged children (8-13 years) from third to sixth grades from one public school in the greater Panama West area. We included children of both sexes from third to sixth grades were chosen because, from the third grade, nutrition and healthy eating contents are included in public –school subjects taught in Panama. The lower grades (first and second grades) were excluded because the pedagogical approach prioritizes the strengthening of reading and writing skills, rather than food and nutrition literacy, during that time.

Evaluation was done through a face-to-face Spanish interview with each student at school by a nutritionist from the research team. Those questions that were confusing to students or produced unexpected results were examined by the research team and either modified or discarded. The final KAPB questionnaire then included 51 items: 14 knowledge questions using multiple answer choices, and 37 Likert-like items assessing attitudes, perceptions, and behaviors.

Step 3: Final Test of Revised Questionnaire Questions (n = 248 students): The revised instrument, based on steps 1 and 2 above, was conducted among a larger sample from third to sixth-grade students of five geographically dispersed public primary schools of the Province of Panama West, a region with a predominance of low and middle socioeconomic status. Three of the schools were chosen because they participated in the intersectoral project called "Healthy Schools" where students are expected to achieve an adequate state of physical, mental, and social health; and create an environment of institutional and communal well-being through the promotion of healthy lifestyles. The Healthy Schools project also involves families, teachers, school staff, and the community. This project promotes learning as well as strengthening the school health programs, and the training of teachers with improved nutrition knowledge and skills for enhanced teaching effectiveness [21]. The other two schools were chosen for being schools with similar characteristics but not yet participating in the Healthy Schools program. All five schools include students from predominantly medium and low socioeconomic backgrounds.

The sample size was calculated to estimate a proportion of excess weight of 29%, as reported by the Panama Ministry of Health in 2014 [22]. The sample formula used was n=Zα2*p*q/d2, where the confidence level was 95%, p was 0.29, q was 1-p, and d was the precision of 6%. The sample size was 219 subjects. The final sample size was 248 subjects. Participation was voluntary, and no incentive was given. The selection of the participants was made using a convenience sampling in coordination with the Principal and the grade teacher. A sociodemographic questionnaire was carried out that included age, sex, and sociodemographic characteristics. Besides, the measurement of weight and height using an 876 SECA electronic scale and a 213 SECA portable stadiometer (Seca Corporation Medical Scales and Measuring Systems, Hamburg, Germany). With weight and height, the BMI z-score for age was obtained according to the WHO 2007 growth patterns.

Step 4: Data Analysis: The Stata 12.0 Program (StataCorp, College Station, Texas, USA) was used for analysis. First, the sample adequacy measure was analyzed through the Kaiser-Mayer-Olkin index (KMO) [23], which determines if the correlations between the variables are sufficiently small. Second, the exploratory factor analysis (EFA) was performed, which allows us to check the validity of the construct [24]. Third, for the confirmatory factor analysis (CFA), the Jamovi 1.1.9 software was used [25], which through the R lavaan package [26,27], performs this procedure. A model was created to compare its indicators to examine the psychometric properties and fit of the model found by CFA. This second model ordered the items according to the theory of attitudes [28], which considers that it is made up of three dimensions: "beliefs," "behavior," and "emotions." Thus, if the item used a verb associated with an emotion (i.e., I want, I like), it would be considered for the emotional dimension. If the item indicated, an action performed directly by the participant (i.e., I wash my hands), it was recognized as a behavior. Finally, statements that describe how they saw reality were beliefs (e.g., "It is good for health"). Lastly, fourth, the analysis of the internal consistency of the primary scales was performed using Cronbach's alpha coefficient that measures the reliability of the measurement scale [29].

Fifth, given the low variability of the answers (zeros "0" and ones "1") in the knowledge section, this was only modified after the content review carried out by the peer judges. For the analysis of the psychometric properties of the instrument, attitude, perception, and behavior sections, the sample adequacy measurement was carried out using the KMO index. The KMO index was considered acceptable when the value was 0.5 to 1.0. An acceptable KMO was also considered appropriate to apply the EFA, which allows evaluating the proportion of the variance explained by the smaller number of necessary factors and its adaptation to the theoretical construct. The EFA was considered acceptable when the factors that explain the variance were ≥0.60 after the varimax rotation suppressing the values ​​≤0.3 to simplify the expression in terms of the items with greater relevance. Subsequently, the reliability was evaluated using an internal consistency analysis estimating Cronbach's alpha; it was considered good when the value was ≥ 0.80; acceptable when it was ≥0.70; questionable when it was ≥0.60; and, low when it was ≤0.59.

This study is part of the FAO technical cooperation project in Panama, symbol TCP/PAN/3606 entitled “Design of a multi-sectoral public policy against overweight and obesity in school-age children in Panama.” The intervention study protocol was reviewed and approved by the Bioethics Committee of the University of Panama. Before the beginning of the field study, written research authorization was granted by the Ministry of Education (MEDUCA, for its acronyms in Spanish) at the central level and in the Province of West Panama. Field study coordination was carried out by the Ministry of health and education staff, as well as with the participation of the schools' Principals. Finally, all students provided oral assent and parental informed consent before study commencement. Participation was voluntary and anonymous; no personal or contact information was collected.

The study was conducted among 248 students from five primary schools of the West Panama Province, with a mean and standard deviation (SD) age of 9.8 ± 1.3 years and a range between 8 and 13 years (54.8% of females and 62.5% of healthy schools). The student participants are mostly mestizos. The overweight proportion is 46.4% z score for body mass index and age (BAZ) ≥ 1 SD). No difference in nutritional status was observed based on BAZ (t-test, p = NS) and overweight (chi2, p = NS) by schools part of healthy schools program compared to regular schools (Table 2).

The final version of the KAPB questionnaire consisted of 51 items, 14 multiple-choice options (15), and 37 Likert types; it was assessing attitude (17 items), perception (10 items), and behaviors (10 items) of eating habits and lifestyles.

The 37 items of the attitude, perception, and behavior sections were analyzed. When analyzing the measure of sample adequacy, an index of KMO = 0.8052 was obtained, which was considered appropriate for the realization of the EFA, through which four main factors were identified. Four factors have eigenvalues ​​higher than 1.00 and explain 62.29% of the total variance.

Table 3 shows the matrix of rotating components for the EFA. The first factor accounts for 36.20% of the variance explained and saturates 16 items related to fruit consumption, vegetable consumption, physical activity practices, water consumption, the importance of breakfast and hygiene, so this first factor is called "aspects related to healthy eating habits and lifestyle." The second factor accounts for 9.00% of the explained variance and saturates three items related to handwashing, water consumption, and hygiene, for what this second factor is called "aspects related to the use of water." Factor III accounts for 8.69% of the variance and includes four items related to the cost of healthy eating, consumption of sugary drinks, effects of exercise on physical performance, and nutritional status, for what this factor is called "aspects of concern about nutrition, food and healthy lifestyles." For its part, Factor IV gathers three items and accounts for 8.41% of the variance, its contents refer to the school as a place to learn about food, knowledge about the benefits of eating vegetables and physical activity after school, for what this factor is called "aspects related to learning and the school environment."

Table 4 shows the comparison between CFA. By comparing the statistics of each model, it is possible to describe that they are similar. For example, both as a comparative fit index (CFI) and Tucker Lewis Index (TLI) show a better model when they reach numbers <0.9 in this case, only the EFA-based model reaches that criterion in CFI. On the other hand, the attitude-based model in Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) statistics presents better performance since lower values in these statistics suggest a better model. Finally, values <0.05 in Root Mean Square Error of Approximation (RMSEA) and Standardized Root Mean-Square (SRMR) are indicated as criteria to identify a better model. Thus, both models only achieve it in RMSEA, although the EFA-based model shows a better indicator in SRMR. In summary, the EFA model is better in two indicators than the attitude. In comparison, the attitude is better in two other indicators; besides, the statistics shown are quite close, so it is not clear which model is superior to the other from this point of view.

Table 5 shows the internal consistency of the questionnaire and the four main factors of the scale estimated by the Cronbach's alpha coefficient. The final instrument, with the 37 items, showed an internal consistency of 0.8200. The first factor showed an internal consistency of 0.8581; factor II showed an internal consistency of 0.5592; factors III, with four and three items, showed internal consistencies of 0.5331 and 0.5341, respectively.

The internal consistency of the measurement scale determined by Cronbach's alpha was high. In general, values ​​of Cronbach's alpha between 0.8 and 0.9 are preferred, emphasizing that the minimum acceptable value for the alpha is 0.70 [30]. Due to the low saturation of items by factors, when Cronbach's alpha was obtained, low internal consistency was observed, except for the first factor, where a Cronbach's alpha of 0.8581 was reported. However, the criterion can be lower if the factor analysis includes few items (less than ten items), and there is evidence of validity, as well as if a solid theoretical and practical reasoning is present for its use [31].

The purpose of this study was to design a construct validity and reliable questionnaire to assess nutrition-related KAPB in school-age children in Panama. The findings meet the requirements for the development and validation of measurement scales and provide a tool to evaluate the effect of FNE interventions that promote healthy eating and lifestyle habits on psychometric markers of human behavior in school-aged children in Panama.

The significant parameters measured on the questionnaire translate the questionnaire into the internal validity and reliability of the scales. In the future and for research purposes, the measurement of KAPB must have a solid base. Therefore, it is necessary to determine the applicability of the instrument, considering the ethnic, cultural, and social aspects. Besides, the psychological elements and the age group to which the interventions are directed should be considered, since they also influence attitudes and behaviors about healthy eating and lifestyles. Likewise, the improvement, adaptation, or modification of the questionnaire is recommended for specific research interests.

The reliability values ​​of the present study are consistent with that reported by Guerrero et al., who found a Cronbach's alpha of 0.81 in a paper whose objective was to design and evaluate a questionnaire of eating habits and physical activity in school-age children from 8 to 12 years old [32]. Also, the findings of the present study are consistent with the work done by Pino et al., who found a Cronbach's alpha of 0.79 in a study to design a reliable tool that quantifies and determines the individual's eating habits in a population sample that attend primary health care centers in the Maule of Chile [33]. However, these instruments are not aligned with the Panamanian FBDG, do not focus on the food environment as a determinant of obesity, and are not culturally relevant to the study population.

Our findings, which are similar to those other studies among similarly aged children in Latin American settings, highlight the ability to use relatively simple procedures for developing valid instruments that assess KAPB among children in school-based nutrition programming. Also, more recently, Koo et al., validity and reliability of questionnaire development study with n=207 children 9-11 years in Kuala Lumpur, Malaysia demonstrated acceptable internal consistency for knowledge (0.70), attitude (0.72) and practice (0.73) [34]. Similarly, Chen et al., validity and reliability study with n=165 trainers or caregivers about nutritional management of persons with disabilities reported right internal consistency for knowledge (0.63), attitude (0.67), and practice (0.82) domains [35]. Too often is monitoring and evaluation of programing done without using validated instruments. Still, our study highlights the ease at which questionnaire validation can be done for more credible results with the potential to influence public policy.

We must emphasize that the differences and similarities between results reveal the need to conduct studies in specific populations to guide and implement preventive actions in each study group. Also, it is needed to make the assessments of the scales used, even more so if we consider that the Cronbach's alpha is a psychometric evaluation dependent on each population [36]. It is also recommended to evaluate the feasibility of using the questionnaire in other ethnic groups in Panama.

Some study limitations exist. First, other important psychometric, social, and cultural properties were not evaluated because the study just focused on dietary aspects. However, nutritional and dietary behaviors are affected by multiple factors. The study provides evidence that the psychometric characteristics of the scale are adequate to reflect changes in knowledge, attitudes, perceptions, and behaviors of healthy diets and lifestyles in school-aged children in the Panama area. It will be a useful and valid tool to assess school-based nutrition interventions among school-aged children in Panama moving forward.

In conclusion, the findings of the present analysis can be used for the development of instruments and scales of measurement, which show that a reduced and more precise number of items on different constructs of psychosocial importance are reached. Besides, culture, social values, socioeconomic situations are fundamental pillars that condition behaviors and attitudes about healthy eating habits and lifestyle in school-age children, as well as in other age groups. In such social groups, if knowledge, attitude, and perception of desired behaviors are low, unhealthy dietary practices will continue to persist over time, as well as the increase in malnutrition early in life. Therefore, it is needed to make changes to cultural and social values to modify healthy dietary cognitive and behavioral dimensions.

The United Nations Food and Agriculture Organization, the Sub Regional Office for Mesoamerica and the Representation of FAO in Panama contributed the resources for the realization of the present study (TCP / PAN / 3606). The authors are grateful for the contributions of Professor Victoria Valdés, from the University of Panama; Nutritionist Verónika Molina, FAO Nutrition Consultant; Nutritionist José Renán De León, Head of Nutrition Division in MINSA; and Nutritionist María Fernanda Álvarez, from the Education Regional Office of West Panama in MEDUCA. The authors also acknowledge the contributions of Nutritionist Katherine Farro, Lisbeth Escala, Ledys Urriola, Francia Sandoval, Zachary McCammon, and Professor Ericka Matus of UDELAS. The authors acknowledge the support of the Nutrition and Dietetics School of the University of Panama, the Regional Directorate of MEDUCA, and the MINSA of the Province of West Panama.

IR-C, RA-P, and AL contributed with the conception and design of the idea; GG-M, and EG-M assisted with the acquisition, analysis, and interpretation of results. IR-C and GG-M developed the analysis and the drafted manuscript. IR-C, EG-M, SRK, GG-M, RA-P, and AL contributed to the interpretation and made a critical review of the final version of the manuscript. IR-C is the final person responsible for the manuscript. All authors reviewed and contributed to the final version of the manuscript.

The opinions expressed in this publication are those of its author (s) and do not necessarily reflect the views of FAO.

The United Nations Food and Agriculture Organization, the Sub Regional Office for Mesoamerica and the Representation of FAO in Panama contributed the resources for the realization of the present study (TCP / PAN / 3606 entitled “Design of a multi-sectoral public policy against overweight and obesity in school-age children in Panama”).

Israel Ríos-Castillo, Elka González-Madden, Stephen R. Kodish, Gabriel González-Medina, Analinnette Lebrija, and Roberto Ávila-Pozos disclose no conflicts of interest.