Psychometric properties of a modified version of Brazilian household food insecurity measurement scale-Pró-Saúde study

Resumo Trata-se dos resultados das etapas iniciais da avaliação psicométrica de uma versão modificada da Escala Brasileira de Insegurança Alimentar com o objetivo de avaliar a insegurança alimentar aos 12 anos. Os dados foram obtidos, por meio de questionários autoaplicados, de servidores de uma Universidade pública do Rio de Janeiro, que participaram da primeira e quarta ondas do Estudo Pró-Saúde. Avaliou-se a confiabilidade teste-reteste (n=58), consistência interna, validade estrutural, convergente e discriminante (n=3.253). Os coeficientes kappa estavam acima de 0,65; o α de Cronbach foi de 0,84. As cargas fatoriais ficaram acima de 0,800. A confiabilidade composta foi superior a 0,90. Os valores encontrados da raiz quadrada da variância média extraída foram positivos e estatisticamente significativos. A insegurança alimentar domiciliar durante a adolescência esteve fortemente associada com maior tamanho familiar e condições sociodemográficas aos 12 anos: mulher como chefe da família, residir na zona rural ou em cidade pequena, pior padrão de vida e alimentação insuficiente devido à falta de dinheiro. Esta avaliação inicial sugere que a escala apresenta boa performance psicométrica. Estudo futuros devem incluir outras propriedades psicométricas e análises em outros contextos populacionais. Palavras-chave Segurança Alimentar e Nutricional, Validade, Confiabilidade, Questionário Abstract We present results of initial steps of the psychometric evaluation of a proposed modified version of the Brazilian Household Food Insecurity Measurement Scale aimed at assessing adults’ recall of food insecurity at age 12. Data were obtained through self-administered questionnaires from civil servants at university campuses in Rio de Janeiro, who participated in the first and fourth waves of the longitudinal Pró-Saúde Study. We evaluated test-retest reliability (n=58), internal consistency, factor structure, convergent, discriminant validity (n=3,253). Test-retest reliability kappa coefficients were above 0.65; Cronbach’s α coefficient was 0.84. Factor loadings were above 0.800. The composite reliability was above 0.90. The square root values of the Average Variance Extracted were positive and statistically significant. Household food insecurity during childhood was strongly associated with larger family size and several sociodemographic conditions at age 12: female head of household, residence in rural area or small town, worse standard of living, and insufficient food due to lack of money. This initial evaluation suggests good performance. Further investigation should include additional psychometric properties and other population contexts.


Introduction
Household Food Insecurity (HFI) has been a recurring theme in national and international scientific literature due to its high magnitude and relevance. In 2010, the publication The State of Food Insecurity in the World showed for the first time a decline in the number of undernourished individuals in the world 1 . However, this number has grown again in 2016 2 , highlighting the importance of this problem in the current world scenario.
With the establishment of the Millennium Development Goals, renewed in 2015 through the 2030 Agenda for Sustainable Development, whose stated objective 2 is to end hunger, achieve food security and improve nutrition, and promote sustainable agriculture 3 , the measurement of household food insecurity (HFI) has become the focus of a large number of epidemiological and governmental studies.
The evaluation of HFI was for many years carried out by indirect methods 4 . Since the 1990s, new methods for HFI identification, monitoring and evaluation have been developed. The first instrument to address HFI in a broader context was developed in the United States by the Community Childhood Hunger Identification Project (CCHIP) 5 . The development of most scales focusing on families' perception or experience of food insecurity 6 took place during the 2000s, featuring the Household Food Security Survey Module (HFSSM) 7 . It is noteworthy that the development of instruments for measuring HFI and its subsequent cross-cultural adaptations occurred in parallel with the development of short versions for use in large study populations [7][8][9][10] .
Marques et al. 6 identified 24 epidemiological instruments aimed at the evaluation and monitoring of HFI. Most instruments used in epidemiological studies are the HFSSM, HFSSM Six-Item Short Form, and the Modified Radimer/Cornell Scale. The HFSSM is the scale with more linguistic variants (translation and cross-cultural adaptation), psychometric and utilization studies 6 .
The cross-cultural adaptation and validation of HFSSM for Brazil were conducted by six research institutions -Universidade Estadual de Campinas, Universidade 11 . The unidimensional instru-ment is composed of 14 items about household availability of food due to limited resources 12 . Six of the 14 items are limited to households with at least one member aged less than 18 years. These items questioned whether children or adolescents (i) did not eat a healthy and varied diet, (ii) did not eat enough, (iii) cut meal size, (iv) skipped meal, (v) did not eat all day, and (vi) felt hungry 12 . The EBIA is being widely used in epidemiology researches [13][14][15][16][17][18] and nationally representative surveys [19][20][21][22][23] . The recall period of the EBIA is the last three months prior to the interview.
Marques et al. 6 observed that the recall period of the food insecurity scales were 30 days, 3 or 12 months prior to the interview. We could not identify in the literature a scale specifically aimed at assessing HFI in childhood among adults. It is of major interest the development and evaluation of psychometric properties of such an instrument, considering the psychological, physical and social repercussions [24][25][26][27] of HFI that may occur in the short, medium or long terms. Some studies have shown the late consequences of childhood HFI, such as asthma 28 , depression and suicide ideation 29 . However, studies assessing medium and long term impacts are scarce 30,31 . Given this situation, the elaboration and validation of an instrument to evaluate the HFI in childhood among adults could allow studies with this approach, helping to fill this gap in the area.
The present paper assesses some psychometric properties (test-retest reliability, internal consistency, factor structure, and construct validity) of a modified scale derived from the Brazilian Household Food Insecurity Measurement Scale, aimed at identifying household food insecurity during childhood.

Study design and participants
This study is part of the longitudinal Pró-Saúde Study, whose overall objective is to investigate the role of socioeconomic and sociocultural determinants of health patterns among civil servants at university campuses in the state of Rio de Janeiro, Brazil.
The current analyses include data obtained in the first (1999) and fourth (2012) waves of Pró-Saúde Study, with 4,030 and 3,253 participants, respectively. In both waves, questionnaires were self-administered at the workplace, with the support of trained and certified research assistants and supervisors.
In the 4 th wave, some scales of the questionnaire were reapplied within a two-week interval to 58 volunteers who were temporary university employees performing activities similar to permanent civil servants (cohort participants). The conditions under which the questionnaire was administered were identical to those of the main study, i.e. during working hours, in university classrooms, and with help available from trained personnel, with the explanation given that the purpose was to test the adequacy of the questionnaire, rather than the participants' responses. In relation to sample size, Donner et al. 32 suggest that test-retest evaluations should include from 25 to 50 individuals; other some authors accept twenty as the minimum sample size for kappa estimation 33,34 .

Instrument
The instrument evaluated in the present study is a modified version derived from the Brazilian Household Food Insecurity Measurement Scale, called "Household food insecurity measurement scale in childhood". This modified version was based on items from the original instrument (EBIA). Four of the six items addressed to individuals under 18 years were kept in this version, except the question about feeling hungry. One item addressed to families without individuals under 18 years was kept ("ate less than you felt you should"). This modified version is introduced by the statement "In some families, there are times when there is lack of money at home to buy food". Five questions follow: "When you were 12 years old, how often were there occasions when you, for missing money at home to buy food ... (a) ... did you not have a healthy and varied diet? (b) ... did you eat only a few types of food? (c) ... did you eat less than you felt you should? (d) ... did you skip meal? ... and (e) ... did you not eat all day or have only one meal a day?". All questions had four response options (frequently, sometimes, rarely, never). This unidimensional scale was assessed in 4 th wave. Individuals who responded "frequently", "sometimes" or "rarely" to at least one of the items were considered to have experienced household food insecurity at age 12.
According to Piaget, at 12 years old, the individual can already reason about the context in which she or he is inserted, using formal logical principles 35 . At this age, adolescents acquire the ability to criticize and discuss social values in order to build their own values and autonomy 35 .
In view of this, the 12-year-old person already presents better understanding about the world around, especially about the socioeconomic and cultural context, thus being able to report more reliably about the experience or not of HFI.

Other variables
In the first wave of the Pró-Saúde study, a multiquestionnaire was applied, which included the following sociodemographic questions: (i) how many children the mother had (1 to 4, 5 or more); (ii) who was the head of household (father, mother/other); (iii) place of residence (Capital/Big City, Small Town/Rural area); (iv) family economic situation (rich/middle class, poor/very poor); (v) family past standard of living (better than now/same as now, worse than now); and (vi) insufficient food due to lack of money in childhood (frequently/sometimes, rarely/never). With the exception of the number of children, all other variables were related to the same period of life (age 12) to which the HFI scale refers.

Assessment of the psychometric properties and other analyses
The psychometric properties evaluated were test-retest reliability, internal consistency, factor structure, and construct validity. To estimate test-retest reliability the scale was used quadratic weighted kappa. According to Streiner and Norman 36 , the most appropriate estimator for assessing intra-or interobserver agreement (test-retest reliability) on Likert scales, as assessed in this study, is weighted kappa. The weighted Kappa calculation considers the observed agreement ratio (p o ), the expected agreement ratio by chance (p e ) and the sample size (N). w ij is the weight given to cell i,j (in this study, weights were quadratic), and k is the upper limit value of the score. It was estimated separately for each item, with 95% confidence intervals obtained by bootstrap. The values obtained were interpreted according to the criteria proposed by Shrout (moderate: 0.60 to 0.79, substantial: ≥0.80) 37 .
To evaluate the internal consistency of the scale, the Cronbach's α coefficient and its one-sided 95% lower confidence limit were calculated. According to Nunnally and Bernstein 38 , values greater than 0.70 are considered satisfac- tory in the early stages of scale development. We also examined the α coefficients after removing each item from the analysis and the item-rest correlations.
The factorial structure of the scale was initially tested by an exploratory factor analysis (EFA) by the principal factor analysis (PCA), using geomin oblique rotation. The number of factors was identified by the eigenvalue criterion above one and the inflection point of the scree-plot. Then, confirmatory factor analysis (CFA) were applied to test the model identified previously. As appropriate to modelling categorical items, all analyses used the robust weighted least squares mean and variance adjusted estimator (WLS-MV) and used polychoric matrices 39 . Potential residual (error) correlations were examined using the modification indices (MI). The assessment of the model's fits used the Comparative Fit Index (CFI), Tucker-Lewis Index (TLI) and Root Mean Square Error of Approximation (RMSEA). The models presenting CFI and TLI greater than 0.90 40 and RMSEA less than 0.08 were considered satisfactory 41 .
Then, the convergent and discriminant factor validity of the scale was formally evaluated by calculating the composite reliability and the Average Variance Extracted (AVE), respectively. Composite reliability takes factor loadings and measurement errors into account. Its estimate ranges from 0 to 1 (in each factor) and values ≥0.70 are considered satisfactory 39,42 . The AVE also ranges from 0 to 1. This estimate assesses the amount of variance extracted in a factor compared to the amount of variance due to random measurement error 39 . A statistically significant positive sign of this difference would endorse factor-based discriminant validity (i.e., non-violation), whereas a statistically significant negative sign would favour rejection.
After the psychometric evaluation, we investigated the associations between the results of the proposed instrument and variables previously described in the literature as related to food insecurity. The presence of HFI during childhood was considered to be the individuals with at least one positive response one of the items of the new instrument. The relationships between the reported experience of HFI during childhood and six sociodemographic variables collected in the 1 st wave aforementioned were tested, using chi-square test. The literature in the area points out that poverty, low family economic situation, higher number of children in the household, female-headed households, and other sociodemo-graphic factors are associated with HFI [43][44][45][46][47] . Thus, it is expected that the HFI positively correlates with the sociodemographic variables mentioned.
In this study, for the evaluation of construct validity, the presence of HFI during childhood was considered for those who answered "frequently", "sometimes" or "rarely" to any of the five items of the scale. Those who answered "never" to the five items were considered to have "no HFI", that is they had household food security. For the other psychometric analyzes, the 4 response options were used. Data analysis was carried in the Stata 15.0 48 and Mplus 7.2 49 softwares.
The research project was approved by the Research Ethics Committees of the Institute of Social Medicine, and University Hospital Pedro Ernesto -State University of Rio de Janeiro, Brazil.

Results
In the 4 th wave, the age of the respondents varied between 34 and 79 years (mean: 52.4 years); 56.6% were female; 52.7% self-reported as white. In baseline (1 st wave), 62.9% respondents were married; 75.8% had at least complete secondary education and approximately 60% had monthly family income above U$ 384.81 (R$ 1,500).
Test-retest reliability estimates are shown in Table 1. Weighted kappa values were above 0.647, being considered moderate for three items and substantial for two items. The frequency of positive responses ("frequently", "sometimes" or "rarely") to each item showed that only items (d) "... did you miss any meal?" and (e) "... did you not eat for a whole day or have only one meal a day?" had an endorsement frequency below 20% (Table 1).
Cronbach's α coefficient for the scale was 0.84, demonstrating a satisfactory internal consistency. The changes in the α coefficients after excluding each of the items were very small and the itemrest correlations were above 0.60 ( Table 2).
The initial exploratory factor analysis indicated that one factor with eigenvalue above 1.0 could be extracted. The loadings ranged from 0.873 to 0.931, and therefore no residual variances were above 0.70. The analysis showed high values for the CFI and TLI, but the RMSEA was above 0.08.
A one-factor model was then tested in a confirmatory analysis (Model A, Table 3). Factor loadings were above 0.873 and despite the high values for the CFI and TLI, the RMSEA was above 0.08, suggesting a poor fit. Since modifi-cation indices indicated a residual correlation between items 1 and 2, a model specifying this correlation was explored (Model B, Table 3). This model had a better fit, with factor loadings above 0.800. The composite reliability found for model A was 0.96 and for model B was 0.94. In both models presented, the square root values of the AVEs were positive and significant. The model B (proposed model) and its factor loadings are shown in Figure 1. Table 4 shows the relationship between the presence of HFI during childhood and selected   construct-related variables assessed in wave 1. Recalled HFI during childhood was higher for participants who at age 12 had a larger family, whose household was headed by the mother or someone other than the father, lived in a small town or rural area, had a poor economic situation, and had experienced insufficient food due to lack of money.

Discussion
To our knowledge, the scale analyzed here is the first instrument aimed at measuring household food insecurity in childhood. As previously mentioned, the existing scales present a recall period of one 50,51 , three [52][53][54][55] or twelve 8,56-59 months prior to the interview. Epidemiologic evidence about adverse long-term consequences of childhood food insecurity is needed in life course studies, but frequently it is not possible to capture this information through longitudinal studies, that is following up individuals from childhood through adulthood.
Although few studies assessed test-retest reliability of household food security scales 51,60 , ours found test-retest reliability estimates similar to those observed by Kleinman et al. 51 , who examined the 18-item Household Food Security Survey Module (HFSSM) subsequently adapted for Brazilian Portuguese 11 . Derrickson et al. 60 evaluated the test-retest reliability of the 18-item HFSSM version via Pearson correlation coefficient, as r=0.75 (p<0.01).
The results of Cronbach's α coefficient of this study were similar to that found by Vargas and Penny 59 , Gulliford et al. 8 and Hromi-Fiedler et al. 61 . The studies that evaluated the internal consistency the HFSSM and your variants presented Cronbach's α coefficient ranging from 0.73 56 to 0.95 55,62 .
We note, however, that comparisons of reliability estimated across studies are problematic; being directly related to the quality of the measurement process, each study may have a unique modus operandi. However, in the long term, several studies with good reliability results (internal consistency, intra-and inter-observer/test-retest reliability) using the original scale tend to confirm its quality 63 .
Regarding validity aspects, the best fit factorial structure presented adequate factorial loadings for all items and fit indexes, with the exception of RMSEA. The studies that evaluated the dimensional validity of the original (18 items) 50,52,53,58,61,64 and short (6 items) 8,57 versions of the Household Food Security Survey Module mostly applied exploratory factorial analyses and do not present the values of the factorial loadings of the items, hampering the comparison of results. However, studies analysing dimensional validity corroborate the structure of one factor proposed by the original instrument 8,52,53,57,58,61,64 , the latter similar to that found in this study.
Inconsistent fit indices are common in psychometric analyses, especially among RMSEA and CFI 65 . Some authors point out that this inconsistency may be underestimated 66,67 , since some surveys report only the adjustment indexes with favorable results. However, Lai and Green 65 and Mueller and Hancock 68 advise the researchers to report all the analyzed estimates, inconsistent or not.
When analyzing the modification indexes of the model with a single factor (model A), a residual correlation between items (a) and (b) was suggested, indicating a content redundancy among the items. A possible overlapping of ideas of the items "(a) ... did you not have a varied and healthy diet?" and "(b) ... do you eat only a few types of food?" -which was confirmed by the high correlation between the residuals of these items. Future studies that include discussions on the writing of the items by specialists and, subsequently, testing on individuals from the target population in order to assess their understanding are encouraged. However, it is important to consider that a possible redundancy among these items was not observed in the Brazilian Household Food Insecurity Measurement Scale 12 70 corroborated the construct validity of the HFSSM and its linguistic and cultural variants by observing a strong association between household food insecurity and food consumption. Derrickson et al. 73 and Gulliford et al. 50 verified a strong association of the scale with socioeconomic variables, low cost foods and lower monthly household income, respectively. Vargas and Penny 59 also corroborated construct validity by associating scale with demographic issues, such as family size.
The present study has some limitations. The HFI and some variables used in the construct analysis present a long recall period (when the respondents were 12 years old), which may lead to recall bias. However, we believe that HFI is such a salient event in the life course of those who go through this experience, that the possibility of this bias is minimal. In addition, a sen-sitivity analysis was carried out excluding older people. The statistics increased, but there was no change in the results found, with no evidence of recall bias. Another possible limitation refers to the large sample size used to validate an instrument with few items since a large number of individuals can influence the psychometric analysis performed. The low value of RMSEA suggests internal validity problem and needs more psychometric studies before being recommended for use. Additionally, our analyzes are based on responses of individuals with a certain degree of schooling, and the possibility of different reliability and validity results cannot be ruled out if applied in a different population group. Further studies should evaluate its psychometric performance in other population contexts, as well as other properties such as scalability.

Conclusion
Our investigation of test-retest reliability, internal consistency, structural, convergent, discriminant, and construct validity represents the first evaluation of a modified version of the Brazilian Household Food Insecurity Measurement Scale proposed to assess food insecurity in childhood. Test-retest reliability kappa coefficients were moderate; Cronbach's α coefficient was satisfactory. Factor loadings were above 0.800; the RMSEA value indicated poor fit. The convergent discriminant, and construct validity were non-violation.
A possible step to be considered in future studies is the evaluation of the items included in the instrument proposed by specialists and the target population, in order to assess the need for inclusion of new items either from the original instrument or not, changes in the wording of the remaining items or even exclusions. In addition, studies exploring possible cut points for this instrument are needed.

Collaborations
ES Marques and AGS Oliveira were responsible for the data analysis, conceiving and drafting the article, and the final critical review. E Faerstein was responsible for designing and coordinating the Pró-Saúde study, conceiving and drafting the article, and final critical review.