Dietary quality varies according to data collection instrument : a comparison between a food frequency questionnaire and 24-hour recall

1 Instituto de Nutrição Josué de Castro, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brasil. 2 Faculdade de Nutrição, Universidade Federal de Mato Grosso, Cuiabá, Brasil. 3 Instituto de Saúde Coletiva, Universidade Federal de Mato Grosso, Cuiabá, Brasil. Paulo Rogério Melo Rodrigues 1,2 Rita Adriana Gomes de Souza 3 Mara Lima De Cnop 1 Luana Silva Monteiro 1 Camila Pinheiro Coura 1 Alessandra Page Brito 1 Rosangela Alves Pereira 1 ARTIGO ARTICLE


Introduction
Given the complexity of individual food consumption, different methods have been proposed to assess dietary quality, with the aim of providing more efficient analyses of the correlation between diet and health outcomes 1,2,3 .One of the methods proposed is to estimate dietary indices elaborated from dietary guideline recommendations.The aim of these indices is to characterize food consumption, estimate overall dietary quality, and represent adherence to dietary recommendations 2,4 .
The Brazilian Healthy Eating Index -Revised (BHEI-R) for the Brazilian population 5 characterizes dietary quality based on components related to food and nutrient groups, considering the recommendations regarding nutritional aspects affording protection against non-communicable chronic diseases.This index was developed according to the Dietary Guidelines for the Brazilian Population 6 and the recommendations of the Brazilian Society of Cardiology 7 and U.S. Institute of Medicine 8 for saturated fat and sodium consumption, respectively.The BHEI-R is also based on the method proposed by Guenther et al. 9 for the Healthy Eating Index 10 , which takes into consideration dietary guidelines for Americans in 2005 and emphasizes aspects such as the consumption of whole grains, vegetables, good quality fats, and energy intake derived from solid fat, added sugar, and alcoholic beverages (SoFAS).Furthermore, Andrade et al. 11 found that the BHEI-R is a reliable instrument and structurally valid to assess and monitor the quality of Brazilian diets.
However, few studies evaluated the concordance of dietary indices by comparing the results obtained by different instruments used to assess food consumption.According to Moeller et al. 12 , the reproducibility and validity of dietary indices rely on the dietary assessment instrument and the objective evaluation of the diet.Thus, the quality of information on food consumption is essential for the success of epidemiological studies aimed at estimating the relationship between diet quality and health outcomes, and may be ensured by studies evaluating the concordance between dietary data estimated from two different instruments 13 .A reasonable concordance and a high correlation coefficient was observed for dietary quality index obtained from food frequency questionnaires (FFQ) and from food records 14,15 .Therefore, the aim of this study was to assess the concordance between BHEI-R estimated from an FFQ and from adjusted means of multiple 24-hour recalls (24h-R).

Methods
This study used data from a study designed to validate an FFQ (applied with the support of food photographs elaborated to assist the application of the FFQ) developed for adolescents.The study was performed in a public elementary school located in the municipality of Niterói, Rio de Janeiro, Brazil.The study was approved by the Ethics Research Committee of the Institute of Social Medicine at the State University of Rio de Janeiro, and participation was dependent on the responsible adult or individual, aged 18 years or older, who provided informed consent.In addition, authorization to develop the study in this teaching unit was obtained from the State Department of Education and the School Director.

Study design and participant selection
The original study was conducted in 2009 to validate an FFQ, with supporting food photographs.For the study, 120 adolescents of both sexes (60 boys and 60 girls) between 12 and 18 years old were selected.The sample size met the recommendations of Cade et al. 16 for FFQ validation studies of at least 50 individuals and preferably 100 or more when the Bland-Altman method is used for the analyses 17 , and a sample of between 100 and 200 individuals for the correlation coefficient.A list of students, including information on their sex and age, from the sixth grade of elementary school up to the third year of high school, was obtained for the random selection of the adolescents.

Food consumption assessment
Data collection was performed by applying a semi-quantitative FFQ developed for adolescents of Rio de Janeiro, which consisted of 93 food items and was properly analyzed for reproducibility 18 and validity 19 .The detailed description of the FFQ elaboration used in this study can be found in the study by Araújo et al. 20 .The FFQ was applied in personal interviews held by trained nutritionists, with the support of food photographs validated for FFQ application in adolescents 21 .
Three nonconsecutive 24h-R (two weekdays, one weekend day) were obtained between three weeks and one month after the FFQ administration.The multiple-pass method was used to apply 24h-R 22 .Photographs of common tools and containers were presented to support the estimate of quantities reported by the 24h-R.The application sequence of the methods used followed Nelson's guidelines 23 , in which the FFQ is applied before the reference method to maintain independence in the questionnaire responses.
Energy and nutrient consumptions were assessed by using the NutWin software (São Paulo Federal University, São Paulo, Brazil).When foods were not available on the NutWin software list, the information was obtained from the Brazilian Food Composition Table 24 and the Nutrition Data System for Research (Nutrition Coordinating Center, University of Minnesota, Minneapolis, USA).Foods were classified according to the groups presented in the Dietary Guidelines for the Brazilian Population 6 .The ingredients of preparations involving more than one food group, such as sandwiches and pizzas, were segregated into their ingredients, which were classified into their respective groups.Standardized recipes, proposed by Fisberg & Villa 25 and Novaes 26 , were used for this procedure.
The intra-and inter-individual variabilities were estimated for the set of three 24h-R and means of energy and nutrients were deattenuated for intra-individual variability, according to the method proposed by the Iowa State University 27,28 , by using the Software for Intake Distribution Estimation (PC-Side version 1.0, Iowa State University, Ames, USA) 29 .

Diet quality
Dietary quality was evaluated by using the BHEI-R for Brazilians 5 , assessed by attributing a score to 12 components belonging to groups of foods, nutrients, and the concomitant presence of nutrients and foods, which characterize different aspects of the diet.
The score assigned to each component is expressed in three levels (maximum, intermediate, and minimum), and the method of calculation is based on nutrient or food group density per 1,000kcal.The score is attributed to different food groups according to the Brazilian dietary guidelines recommendations 6 which consider a 2,000kcal diet and corresponding number of portions per 1,000kcal, to which the highest score was assigned.According to the recommendations of the Brazilian Society of Cardiology 7 , the maximum score for "saturated fat" corresponds to a maximum consumption of 7% of the total energy intake.For "sodium," the maximum score corresponds to diets with less than 0.75g/1,000kcal, as recommended by the U.S. Institute of Medicine 8 .Finally, for SoFAS, the limits (maximum and minimum) were based on results related to the consumption of these components, obtained in a population-based survey that was conducted in 2003 in the city of São Paulo 30 .The maximum score for this component was assigned to diets in which solid fats, alcohol, and added sugar provided less than 10% of the total energy consumption, corresponding to the 85 th distribution percentile for the consumption of these components by the population of São Paulo.Detailed descriptions of the criteria for minimum, intermediate, and maximum scores of each component can be found in Previdelli et al. 5 .
Moreover, it is important to highlight that Previdelli et al. 5 chose to include legumes in the "vegetables" and "meat, eggs, and legumes" components to develop the BHEI-R due to their significant contribution to Brazilian dietary habits, as they are an important source of protein, fibers, and minerals.Hence, the amount of energy provided by the food group of legumes (consisting of beans, peas, chickpeas, fava beans, and lentils), after having contributed to reach the maximum score for "meat, eggs, and legumes," was computed along with "total vegetables" and "DGOVL" (dark green and orange vegetables and legumes), arousing the interest to evaluate the real contribution of this group to each component, according to the collection method.

Weight status
The weight profile was evaluated by using electronic scales with a capacity of up to 150kg, in increments of 100g (Plenna Sport, São Paulo, Brazil).Height was measured in duplicate by using a portable stadiometer, in increments of 0.1cm (Altura Exata, Belo Horizonte, Brazil), and assuming a maximum variation of 0.5cm between two measures and then using their mean value.To measure weight and height, adolescents were asked to wear light clothes, and to remain barefoot and in a standing position 31 .Weight status was evaluated by calculating body mass index (BMI = weight/height 2 ), classified according to cutoff points proposed by the World Health Organization 32 and depending on sex and age.

Data analyses
Statistical analyses were performed by using the Statistical Package for Social Science software version 19.0 (IBM Corp., Armonk, USA).Continuous variables were analyzed for normal distributions by performing the Kolmogorov-Smirnov test.Differences in the distribution, observed according to age group and weight status between boys and girls, were evaluated by using the chi-square test.To assess differences (mean and standard deviation) in total energy and nutrient consumption, estimated by using the FFQ and multiple 24h-R, the paired Student t test was used.
Five methods were used to assess the relative agreement between dietary quality estimated with the FFQ in comparison with the use of multiple 24h-R.The total mean BHEI-R and its components of BHEI-R for the total population were compared according to the collection instrument, sex, and weight status by using the paired Wilcoxon test and paired Student t test.Another indicator, which refers to the total mean BHEI-R and its components, was the mean difference (FFQ -24h-R) and its corresponding 95% confidence interval (95%CI), in which a difference was considered significant when the 95%CI did not include the zero.
The correlation between the scores attributed to the total BHEI-R and its components was evaluated according to the collection instrument, by using the command of partial correlations, adjusted for age.
The concordance between the scores referring to the total BHEI-R and its components, according to the collection instrument, was evaluated as follows: (1) by the intraclass correlation coefficient (ICC) and ( 2) the Bland-Altman method 17 .The ICC and its respective 95%IC estimate of the total proportion can be attributable to the variability between individuals and is considered adequate when higher than 0.75 33 .
The scores referring to the total BHEI-R and its components, obtained by using the two instruments, were transformed into natural logarithms and applied to the method proposed by Bland and Altman 17 , estimating the mean concordance and its respective 95% limits of agreement (LOA).To apply this method, the difference (FFQ -24h-R) and the mean (FFQ + 24h-R/2) between the two instruments were calculated.LOA provides the range containing 95% of individual differences in scores referring to the total BHEI-R and its components between the two instruments, as well as the dissemination of the total sample concordance.Lower and higher LOAs were estimated as follows: [mean difference ± (1.96 x standard deviation of the differences between instruments)].As data were analyzed upon their logarithmic transformation, the mean concordance and LOA were expressed in percentages, as the exponential of a difference is a ratio (x 100).
A mean concordance of 100% indicates a perfect agreement between the two instruments.A concordance of 110%, for example, indicates that on average, the FFQ overestimates the value obtained by multiple 24h-R by 1.1 times.Therefore, LOAs between 50% and 200% indicate that 95% of estimates for all the individuals were from 0.5 to 2.0 times the value estimated by using multiple 24h-R.In this study, LOAs between 50% and 200% were considered as adequate 34 .Moreover, simple models of linear regression were developed between the measurement differences as the dependent variable and measurement mean as the independent variable 35 .In this analysis, the β regression coefficient was expected to be close to zero and not statistically significant, demonstrating that the concordance between the two measurements is not influenced by their magnitude (H 0 : β = 0.00 α = 0.05).

Results
Of the 120 adolescents invited to participate in the study, 110 (92%) provided informed consent.Of these adolescents, 106 (89%) participated in the FFQ validation with supporting photographs and were included in this study.The mean (SD) age of the subjects was 14.5 (1.75) years, and 57% of them were female.Regarding age distribution, 56% of the adolescents were between 12 and 14 years old.Regarding sex, no statistically significant difference was observed (p = 0.35).Excess weight was detected in 25% of the adolescents and was more common among the girls than the boys (32% vs. 15%, p = 0.05) (data not shown in tables).
Table 1 shows the lack of agreement for all the parameters evaluated in scores referring to "total fruits," "whole fruits," "whole cereal," "sodium," and "SoFAS" when the FFQ and multiple 24h-R were compared.No significant difference was observed in scores referring to "dark green and orange vegetables & legumes" (DGOVL) and "total grains," although poor agreement (ICC = 0.29 and 0.08, respectively) and a low correlation coefficient (r = 0.16 and 0.05, respectively) were observed.Better agreement (ICC = 0.54) and correlation (r = 0.37, p < 0.01) were observed for "milk and derivatives," however, a significant difference was observed between the scores estimated by using the two instruments (p < 0.01).Scores referring to the total BHEI-R, "total vegetables," and "saturated fat" were statistically different (p < 0.01), with an ICC and a correlation coefficient showing poor agreement (ICC = 0.35, 0.39, and 0.15, respectively) and correlation (r = 0.21, 0.24, and 0.08).Finally, no differences were observed between the estimated scores for "meat, eggs, and legumes" and "oils, oilseeds, and fat fish content" [oils]) (p > 0.05).However, these were less discriminated with limitations to estimate ICC and the correlation coefficient; it was not possible to calculate the p value, as the estimates were the same for all the individuals.
According to the Bland-Altman method, the mean agreement estimated for the total BHEI-R and its components varied from 68% for "whole cereals" to 147% for "whole fruits," as the estimated concordance between the two instruments varied according to the intake of almost all the components, except for the total BHEI-R, "total vegetables," and "total grains" (p > 0.05).
According to the LOA, for most FFQ components, they could underestimate or overestimate scores estimated by using multiple 24h-R by approxi-mately 50%.For example, a 95% LOA for "total vegetables" was 58% and 152%, which means that for 95% of the adolescents evaluated in this study, the estimate of this component provided by the FFQ was underestimated by 42% and overestimated by 52% when compared with multiple 24h-R.Large LOAs were observed for "total fruits," "whole fruits," "DGOVL," "whole cereals," "milk and derivatives," "saturated fat," and "sodium" (Table 2).Differences between scores referring to the total BHEI-R and its components, according to the collection instrument, presented a significant relationship with sex and weight status of adolescents, except for "DGOVL," "meat, eggs, and legumes," and "oils," as these components showed no significant difference between the instruments in any of the groups evaluated (Table 3).
For data from the FFQ, the contribution of the legume group by "meat, eggs, and legumes" was 40.6kcal/1,000kcal and by "total vegetables" and "DGOVL" was 21.2kcal/1,000kcal.Meanwhile, data from multiple 24h-R revealed the contrary, that is, a contribution by "meat, eggs, and legumes" of 24.0kcal/1,000kcal, and by "total vegetables" and "DGOVL" of 36.9kcal/1,000kcal.The differences observed for the contribution from the legume group by "meat, eggs, and legumes," "total vegetables," and "DGOVL" were statistically significant (p < 0.01 between the two instruments; paired Wilcoxon test).To better understand these differences, the content (kcal/1,000kcal) of "meat, eggs, and legumes" (a component formed by all kinds of beef, pork, lamb, of hunting, poultry, fish, and eggs) was evaluated also before adding calories from the legume group.In this analysis, the contributions from this component were 134.8kcal/1,000kcal (95%CI: 126.4; 143.1) and 167.0kcal/1,000kcal (95%CI: 155.9; 178.2) for data obtained by the FFQ and multiple 24h-R, respectively, with a significant difference between the instruments (p < 0.01; paired Wilcoxon test).Therefore, the consumption of meat and eggs estimated by using multiple 24h-R was higher than that estimated by using the FFQ, reducing the contribution of legumes by "meat, eggs, and legumes" when BHEI-R was evaluated based on multiple 24h-R estimates.Subsequently, the contribution by "total vegetables" and "DGOVL" was higher.Hence, the score referring to "total vegetables" and "DGOVL" by multiple 24h-R was higher than that by the FFQ (data not shown in tables).

Discussion
The results obtained in this study indicate that the BHEI-R was strongly dependent on the instrument used to obtain the information on food consumption and showed poor agreement between the BHEI-R estimated by the FFQ or Table 2 Mean agreement and 95% limits of agreement (LOA) between the score of Brazilian Healthy Eating Index -Revised (BHEI-R) and its components, obtained by food frequency questionnaire (FFQ) and multiple 24-hour recall (24h-R).Adolescents (n = 106), Niterói, Rio de Janeiro, Brazil, 2009.* Analysis was performed for the data transformed into logarithm; ** Mean difference ± 1.96 x standard deviation of the distribution of the differences; *** Coefficients obtained in linear regression analysis between the means (independent variable) and differences (dependent variable) of the measurements obtained with the FFQ and the multiple 24h-R;

Components
# Statistical significance of the β.
## Oils includes mono and polyunsaturated fats, oils from oleaginous vegetables and fish oils.multiple 24h-R.The score referring to the total BHEI-R showed poor agreement.Greater agreement was observed for the components and, in particular, for specific items consumed with higher frequency, such as milk and dairy.For foods that are consumed occasionally, such as fruits, the concordance was poor, although it presented statistically different mean values.For highly consumed food such as oils and meats, the mean values were correlated, although the concordance could not be estimated owing to the distribution of the values.Finally, for poorly consumed foods such as whole grains, the mean difference was small.Poor eating habits during adolescence, such as excessive fat (especially saturated), cholesterol, salt consumption, and poor fruit and vegetable intakes may predict the occurrence of obesity and cardiovascular diseases in both adolescence 36,37 and adulthood 37,38 .Therefore, food intake evaluation is an essential tool for monitoring health status 39 .Hence, the use of food standards, including those previously defined as dietary quality indices, is considered an alternative approach to overcome limitations to assessing the diet-disease relationship 4 , as they aim to cover food consumption in general 1,2,3 .
Indices assessing dietary quality have received increasing attention for being methodological tools for simple collection and evaluation, besides allowing for estimating the adherence to dietary recommendations for healthy diets, monitoring changes in food consumption in the population studied, and investigating the Cad.Saúde Pública, Rio de Janeiro, 32(2):e00047215, fev, 2016 associations with health risk factors 1,2,40 .However, although the validity of these indices has been investigated for nutrient adequacy and risk of developing chronic diseases 1,2,3,40,41 , few studies compared the results obtained by using different instruments such as the FFQ and multiple 24h-R 14,15,42 .Therefore, to our knowledge, this is the first study of its kind to be conducted in a population of Brazilian adolescents.
The two studies that used a similar methodology to the one used in this study, estimating the correlation between a dietary quality index obtained by FFQ and multiple food recalls, found correlations higher than observed among Brazilian adolescents.In the Health Professionals Follow-up Study, Newby et al. 14 used a dietary quality index obtained by using two FFQs and compared it to the index obtained by using two weekly food records among men between 40 and 75 years old.They obtained a correlation of 0.66 for the first FFQ and 0.72 for the second FFQ.Moreover, the authors observed that the similarity of the index obtained by using the second FFQ and that obtained by using food records allowed for establishing a positive association with the intake of vitamins B-6 and C, fiber, folic acid, magnesium, calcium, and carotene; and an inverse correlation with total, saturated, and monounsaturated fats, and cholesterol (p < 0.05) 14 .Therefore, Huybrechts et al. 15 , estimated the quality of preschool diets for children between 2.5 and 6.5 years of age, comparing the index obtained by using one FFQ and that obtained by using three food records.They observed a correlation of 0.82 for the total score and, categorizing the individuals by tertiles, found that 62% of children belonged to the same category and only 3% were classified in the extremes 15 .
According to Ambrosini et al. 42 , the assessment of dietary indicators during childhood and adolescence is an important approach to understand how they can affect short and long-term health.However, no study was found evaluating the agreement between diet quality indices obtained from different dietary assessment methods assessing only adolescents.Waijers et al. 40 conducted a literature review on indices commonly used to assess dietary quality and identified 20 indices.The authors found that arbitrary choices and correlations between dietary components were not properly evaluated.They also highlighted poor associations between indices and nutrient adequacy, and pointed out that the indices existing so far do not predict morbidity or mortality significantly better than the individual evaluation of dietary components 40 .Arvaniti & Panagiotakos 41 identified 23 indices used to assess dietary quality and found failures in identi-fying intercorrelations between its components and estimating specific health outcomes.
However, Román-Viñas et al. 43 defined validity as the degree to which the index measures exactly what it intends to measure.The authors pointed out that the effectiveness of an instrument to evaluate dietary indices should consider the purpose for which it was developed; that is, if an index was not created with the aim of assessing nutrient adequacy, its ability to discriminate the consumption by the individuals is compromised.Therefore, it is important to note that the reproducibility and validity analysis of a dietary index is only the first step in developing a new method to evaluate diet 15 .
Hence, this study contributes to identify the reliance of the BHEI-R and its components on the estimate, according to the instrument used to obtain information on food consumption among adolescents.This notion should be considered because dietary indices such as the BHEI-R are generally estimated to better represent the complexity of eating habits.
Therefore, further studies are required to optimize the performance of dietary quality indices and to minimize their flaws and gaps in order to predict conditions for which they play dominant roles and to provide a measurement summary of dietary quality that can be used in public policy and scientific research.Consequently, individual aspects should be considered in dietary evaluation since dietary indices may correlate with characteristics such as sex 4,44 and weight profiles 45,46 , influencing the validity and comparison between dietary patterns.
In this study, different methodological approaches were used to assess the agreement between estimates obtained by using the two dietary assessment tools.In particular, the mean difference and 95%CI, ICC, and Bland-Altman method were applied to assess the validity of the FFQ in estimating dietary quality in adolescents.With the Bland-Altman method, it was possible to verify that a 95% LOA for most components of the BHEI-R violated the appropriate limits for underestimating and overestimating FFQ when compared with multiple 24h-R.In this analysis, the greatest differences in food consumption, that is, greater 95% LOA range, were observed for "whole fruits," "DGOVL," and "total fruits" and "sodium." This study presents several limitations that should be considered, especially those concerning the FFQ (e.g., depending on memory, food preferences, and availability of access to food that can compromise a retrospective consumption report) 47 .However, this instrument also presents advantages for epidemiological stud-ies, such that it is not subjected to the effect of intrapersonal variation, and is practical and cost-effective with regard to data collection and analysis 48 .Moreover, although the main disadvantage of 24h-R is the inability to estimate usual food consumption from a specific day during data collection 13 , in this study, multiple 24h-R were collected, followed by the deattenuation of data from these recalls to better control the effect of intra-individual variability and, therefore, to obtain estimates of the habitual consumption of adolescents 27,28,29 .Alternatively, one possible explanation for the differences in underestimation/ overestimation effect observed according to the instrument used can be attributed to characteristics of each instrument, i.e., the FFQ is composed by a fixed list of food, while the 24h-R is an open instrument.With the use of 24h-R, the omission of consumed food items may occur, while when applying the FFQ it is possible to observe reduced variability in the consumption of food items and overestimation of dietary intake 49 .
Despite this, one positive aspect of the BHEI-R is the grains segregation, that is, separately scoring refined grains (included in "total grains") and whole cereals, as whole grains present a protective effect against cardiac diseases, whereas refined grains can increase their risk 50 .However, reduced consumption of whole grains was observed among the adolescents evaluated in this study, which limited the discrimination and contribution of this component to the total BHEI-R score.In addition, Andrade et al. 11 evaluating the validity and reliability of BHEI-R, according to the psychometric properties, observed that, despite some limitations, this index was reliable and structurally valid for assessing the quality of Brazilian diet.They also found that the content validity was supported since the BHEI-R includes the main recommendations of the 2006 Brazilian Dietary Guidelines 6 and made it possible to distinguish groups with known differences in diet quality 11 .
Finally, another aspect that should be considered to evaluate previously defined dietary patterns is the theoretical framework on which the creation of each index was based.According to Newby et al. 14 , the creation of dietary quality indices can limit their use when based on nutrition policies, such as the BHEI-R being based on the dietary guide 6 .It is noteworthy that, as previously mentioned, the index evaluated in this study was developed based on the recommendations of the Brazilian Dietary Guidelines published in 2006.However, a new version of these guidelines was recently published 51 .In this new guidelines, dietary recommendations were proposed in a different way to those proposed by the 2006 Brazilian Dietary Guidelines.With the advancement of nutritional science, these indices may become outdated and as good as the components on which they are based, thus needing to be revised as science and nutrition policies evolve.

Conclusion
This study showed that the BHEI-R was strongly dependent on the instrument used to obtain information on food consumption among adolescents.Thus, researchers interested in using the BHEI-R to assess the quality of the diet of Brazilian adolescents, should take into consideration the effect of underestimation and overestimation that may occur in BHEI-R and its components according to the dietary assessment method used in collecting food consumption data.Moreover, these results emphasize the need for further evaluations to estimate the usefulness of the BHEI-R in predicting health risk factors and its validity to diagnose nutritional inadequacies.
* p-value using the Wilcoxon paired test, except for the total BHEI-R rated by paired Student t test; ** Could not calculate the p-value, since the estimates are the same for all individuals ; *** Oils includes mono and polyunsaturated fats, oils from oleaginous vegetables and fish oils.#Correlation coefficient adjusted for age.

Table 3
Comparison of the scores of Brazilian Healthy Eating Index -Revised (BHEI-R) and its components, obtained by food frequency questionnaire (FFQ) and multiple 24-hour recall (24h-R) [mean (SD) scores and their differences] according to sex and weight status.Adolescents (n = 106), Niterói, Rio de Janeiro,