Relative validity of a food-frequency questionnaire developed to assess food intake of schoolchildren living in the Brazilian Western Amazon

This study aimed to assess the relative validity of a food frequency questionnaire (FFQ) developed to assess food intake of schoolchildren from the Brazilian Western Amazon. The dietary intakes of 61 schoolchildren, aged between six and nine 9 years, were measured using two 24-hour dietary recalls and one FFQ, conducted with the children’s, mother or guardians. Validity of the FFQ compared to the mean of the two dietary recalls was assessed using Pearson’s correlation coefficient adjusted for attenuation and energy intake, Bland & Altman plots and evaluation of agreement levels between the two assessment methods. Energy-adjusted and deattenuated correlation coefficients ranged from -0.03 for vitamin C, to 0.93 for calcium. The mean coefficient was 0.46. The mean proportion of subjects classified within one quintile by the two methods was 66%. The Bland & Altman plots indicated good agreement for almost all nutrients, with a mean limit of agreement of 108%. These results indicate that, although there was a lack of accuracy for certain nutrients, such as vitamins A and C, the FFQ ensures reliable estimates of intake of most nutrients. Food Habits; Diet Surveys; Child Introduction Child’s food intake is an important determinant of child health and nutritional status during childhood and adulthood 1,2. Recent data indicates that nine out of ten children are born in developing countries. Studies have shown that in these countries undernutrition is highly prevalent 3,4 in early life and is often followed by rapid weight gain during late childhood and adolescence 4. Child health in Brazil has improved in recent decades; however, improvements have been less marked in the Northern Region, which encompasses the Amazon 5,6. Habitual dietary intake data is extremely important for monitoring the health of children living in the Brazilian Amazon and research into the influence of child’s food intake on health outcomes 7. However, there are many challenges involved in assessing dietary intake in children who live in developing countries 7,8,9. Some studies have used the direct intra-household observation method for this purpose 8,10,11; however, major disadvantages of this method include its high cost and the fact that the presence of an observer may elicit changes in food intake. The most commonly used dietary assessment method in developing countries is the 24-hour dietary recall (24HR), due to its low cost and because there is no requirement for the respondent to be literate 12; however, this method does not detect epiARTIGO ARTICLE Scagliusi FB et al. 2198 Cad. Saúde Pública, Rio de Janeiro, 27(11):2197-2206, nov, 2011 sodically consumed dietary components. Furthermore, the intra-individual variation in food intake is two-times greater in children than in adults 9 and consequently requires a large number of recall days to accurately assess food intake, meaning that this method is unfeasible for many types of studies 7. One possible solution for this dilemma is the use of a food-frequency questionnaire (FFQ), since it facilitates the assessment of habitual food intake without imposing a heavy burden on subjects or researchers. None of the FFQs developed to date for use with Brazilian children 13,14 are specifically adapted to the different eating habits of the population of the Brazilian Amazon. In the present study, we evaluate the relative validity of a FFQ developed for assessing the nutrient intake of schoolchildren living in the Brazilian Western Amazon so as to assess if it is suitable for reporting nutrient intakes in an ongoing prospective study on child growth.

Relative validity of a food-frequency questionnaire developed to assess food intake of schoolchildren living in the Brazilian Western Amazon Validade relativa de um questionário de frequência alimentar desenvolvido para avaliar a ingestão por escolares da Amazônia Ocidental Brasileira Introduction Child's food intake is an important determinant of child health and nutritional status during childhood and adulthood 1,2 .Recent data indicates that nine out of ten children are born in developing countries.Studies have shown that in these countries undernutrition is highly prevalent 3,4 in early life and is often followed by rapid weight gain during late childhood and adolescence 4 .Child health in Brazil has improved in recent decades; however, improvements have been less marked in the Northern Region, which encompasses the Amazon 5,6 .Habitual dietary intake data is extremely important for monitoring the health of children living in the Brazilian Amazon and research into the influence of child's food intake on health outcomes 7 .
However, there are many challenges involved in assessing dietary intake in children who live in developing countries 7,8,9 .Some studies have used the direct intra-household observation method for this purpose 8,10,11 ; however, major disadvantages of this method include its high cost and the fact that the presence of an observer may elicit changes in food intake.The most commonly used dietary assessment method in developing countries is the 24-hour dietary recall (24HR), due to its low cost and because there is no requirement for the respondent to be literate 12 ; however, this method does not detect epi-Cad.Saúde Pública, Rio de Janeiro, 27 (11):2197-2206, nov, 2011 sodically consumed dietary components.Furthermore, the intra-individual variation in food intake is two-times greater in children than in adults 9 and consequently requires a large number of recall days to accurately assess food intake, meaning that this method is unfeasible for many types of studies 7 .One possible solution for this dilemma is the use of a food-frequency questionnaire (FFQ), since it facilitates the assessment of habitual food intake without imposing a heavy burden on subjects or researchers.
None of the FFQs developed to date for use with Brazilian children 13,14 are specifically adapted to the different eating habits of the population of the Brazilian Amazon.In the present study, we evaluate the relative validity of a FFQ developed for assessing the nutrient intake of schoolchildren living in the Brazilian Western Amazon so as to assess if it is suitable for reporting nutrient intakes in an ongoing prospective study on child growth.

Study design and participants
The study was conducted in the town of Acrelândia, located in the State of Acre, in the Brazilian Western Amazon.The city covers an area of 1,607.5 square kilometers and has a population of 8,697, of which 43% live in urban areas.The infant mortality rate is 70.75 per thousand live births and the Human Development Index is 0.680 (Instituto Brasileiro de Geografia e Estatística.Censo Demográfico 2000.http://www.ibge.gov.br).
The current study is part of ongoing followup research on child health 15 .Data collection occurred between December 2007 and June 2008.In December 2007, a cross-sectional populationbased study was carried out involving all children under 10 years of age living in the urban area of Acrelândia (n = 1,216) in which we conducted anthropometric measurements and performed an analysis of intestinal parasites and iron deficiency anemia.
In May 2008, a convenient sub-sample of 80 mothers of children aged between six and nine years was randomly selected to participate in the present study.In the case of children that did not live with their mother, the father or guardian (grandmother) responded.Of the sample, 19 subjects had moved or their homes could not be localized.The remaining respondents (n = 61) participated in the first phase of the study, a "feasibility assessment", that occurred between May 2008 and June 2008.The aim of this phase was to obtain information about the children's eating habits so as to assess the necessity to develop a FFQ to measure their food intake.For this purpose, a 24HR was conducted with respondents regarding the children's dietary intake to obtain information on the number of different foods consumed and the number of children who ate each food cited through the 61 recalls.Furthermore, the research dietitians collected recipes, weighed the food portions consumed by the children and measured the capacity of routinely used food utensils.The FFQ and a second 24HR were conducted in June 2008 with respondents of the 24HR carried out a month earlier.The findings of the first 24HR were used to develop the FFQ and the results of both 24HRs were used to validate the FFQ.
This study was conducted in accordance with the terms of Resolution nº.

24HR
Each respondent answered the two 24HRs reporting the child's dietary intake during the previous day.School children living in the Brazilian Amazon are provided with school meals, but they are free to take their own packed lunches to school and/or buy food items.Most of the children in the sample ate the meal provided by the school and also bought snacks and candies.Most of the respondents knew what the children had eaten at school, but when they did not know the children were asked.The 24HRs were conducted one month apart (beginning of May and beginning of June 2008).Sixty-seven percent of the recalls covered weekdays and 32.8% covered a weekend day; however, interviews were limited to only one weekend day.The recalls were conducted by two trained dietitians (F.B.S. and M.T.G.).To ensure standardized data collection, the dietitians followed a specially prepared procedure manual.Firstly, the respondent was asked to list the meal times and the foods consumed and then was re-Cad.Saúde Pública, Rio de Janeiro, 27 (11):2197-2206, nov, 2011 quested to detail how the food was prepared and portion sizes.If the respondent cited a typical food dish, the dietitian noted the ingredients, method and number of servings on a separate form.The capacity of different food utensils was measured using a digital kitchen scale (Tanita model 1140; Tanita, Tokyo, Japan).
In the 102 24HR assessments that were collected, the intake of energy, macronutrients, saturated fat, cholesterol, fiber, vitamins A, E, C, B6 and B12, folate, thiamin, riboflavin, niacin, calcium, magnesium, iron and zinc was calculated by a trained dietitian (F.B.S.) using the World Food Dietary Assessment System 2.0 software (FAO/ INFOODS, Berkeley, USA).Additional data on typical Amazonian fruits was drawn from the Brazilian Food Composition Table 16 .Double-keying was independently conducted for every recall by two individuals (F.B.S. and A.L.C.I.).Mean values were calculated for two of the recall days and the range of the estimates was observed.

FFQ
The FFQ was designed to provide a general assessment of the children's diets and to be used in our cohort studies to investigate the determinants of anemia and child growth in this population.This method was used to measure habitual dietary intake in the month prior to the interview.The questionnaire was developed based on the data from the first 24HR and on other FFQs previously developed for use with Brazilian adults 17 and adapted for use with Brazilian children aged between 9 and 12 years 13 .It is important to note that these FFQs were targeted at the population of the Southeast Region of Brazil, and thus were not specific to the dietary components of the population of the Brazilian Amazonian.
The FFQ was not self-administered; instead, it was conducted with the respondents by two trained dietitians (F.B.S. and M.T.G.).As proposed by Chiara et al. 18 , the FFQ food list was based on the most commonly consumed foods cited in the 24HR.Firstly, all foods cited in the 24HR were listed and the number of children that consumed each one was calculated.The children consumed a total of 77 different foods and only 34 of these components were consumed by at least 10% of the children.Thus, these 34 items were included in the FFQ food list following the procedure adopted by Eulert et al. 19 .Another 16 items, identified as being important sources of certain nutrients, were added to the list: liver (source of iron and vitamin B12), cheese (source of calcium), yogurt (source of calcium) and instant noodles (source of carbohydrate and lipids).Instant noodles were added also because several studies carried out in the country have shown that they are becoming a common part of the diet of Brazilian children 20 .These procedures also assured that some traditional regional foods were included in the list.Since only three fruits (orange, banana and natural fruit juice) and three vegetables (tomato, cabbage and lettuce) were consumed by the children, other fruits and vegetables available in the city markets and grocery stores were added to the list.The items "other fruits" and "other vegetables" were also included.Fruits in season that the mother affirmed that the children consumed were also added.Thus, the final food list contained 50 items.The name of the foods contained in the questionnaire were those typically used by the local population.
The FFQ did not collect information on portion sizes.Instead, the mean portion sizes obtained by the 24HR were used as default values, considering that: (1) frequency of consumption tends to have a greater bearing on dietary intake than portion sizes; and (2) the inclusion of portion sizes may make the questionnaire more difficult to answer 21 .The FFQ had eight frequency options written in an understandable manner where possible: option 1 -the child never or almost never ate this food (coded as 0); option 2 -the child ate this food a few times a month, but not every week (coded as 3/month); option 3 -the child ate this food every week, but only a few times (coded as 2/week); option 4 -the child ate this food several times a week, but not every day (coded as 4/ week); option 5 -the child ate this food every day, once a day (coded as 1/day); option 6 -the child ate this food every day, twice or three times a day (coded as 2.5 times/day); option 7 -the child ate this food every day, four to six times a day (coded as 5 times/day); and option 8 -the child ate this food every day, more than six times a day (coded as 7 times/day).
This number of frequency options was chosen because it was successfully used in two earlier FFQs developed for use with Brazilian adults 16 and adapted for use with Brazilian children aged between 9 and 12 years 13 , and also because it was observed in the diet recall that some children ate the same dish a number of times during the day (such as milk with cassava flour); thus, the option "more than six times a day" was necessary.
The FFQ was analyzed by a trained dietitian (F.B.S.) using the Dietsys software version 4.01 (National Cancer Institute, Bethesda, USA).The Dietsys database was configured to strictly match the World Food database and the nutrients analyzed in the 24HR were the same nutrients analyzed using the FFQ.Double-keying was conducted independently with every FFQ by two individuals (F.B.S. and A.L.C.I.).

Statistical analysis
Data analysis was performed with the SPSS 12.0 (SPSS Inc., Chicago, USA) and MedCalc 9.3.1 (MedCalc, Mariakerke, Belgium) software.Data was presented in the form of median values and interquartile ranges where p ≤ 0.05 was considered significant.The normality of the distribution of data for nutrient intake was assessed using the Kolmogorov-Smirnov test and through the observation of the histograms.Since some distributions were not normal, nutrient values were log-transformed before analysis.
Relative validity was determined by comparing the estimates obtained using the FFQ with the estimates obtained using the 24HR through paired sample t tests, Pearson's correlation coefficient, Bland & Altman plots 22 , percentage of correct classification of intake into one quintile and percentage of gross misclassification.
Pearson's correlation coefficient was calculated for crude and energy-adjusted nutrient intakes.Energy-adjusted nutrient intakes were computed using the residuals method 21 .Furthermore, attenuation of the Pearson correlation coefficients between the results of the 24HR and the FFQ caused by intra-individual day-to-day variation in the nutrient intakes was corrected to account for the within-to-between person variance ratio calculated from two days of 24HR 21 .
Bland-Altman 95% limits of agreement (LOA) 22 were used to evaluate the level of agreement between the two dietary assessment methods for each nutrient intake (mean ± 2 SD of the differences).Before plotting, data was transformed into natural logarithm (ln) and energy-adjusted.After that, following the method described by Flood et al. 23 , the difference in nutrient intake between the two methods was plotted against mean of paired intake values.The antilog of the LOA were then taken to provide a ratio between the FFQ/24HR data.The ratios were multiplied by 100 to express percentages, where 100% represents perfect agreement.For example, where the LOA of a specific nutrient is 95% and 115%, the FFQ/24HR ratio is 105%.The dependency between the two methods was tested by fitting the regression line of differences for each nutrient intake: where a correlation of zero between the differences means that the two methods are equally variable.The following criteria were also used to interpret the LOA 24,25 : (1) good agreement -the difference between the two measurements is approximately equal to one standard deviation of the average nutrient intake shown by the reference dietary assessment method (i.e., the 24HR); (2) fairly good agreement -the difference between the two measurements is approximately equal to two standard deviations of the average nutrient intake shown by the reference dietary assessment method; and (3) poor agreement -the difference between the two measurements is approximately equal to three standard deviations of the average nutrient intake shown by the reference dietary assessment method.
The intake values of the FFQ data and the mean values of the data of the two 24HRs were divided into quintiles and the percentage of subjects categorized within one quintile by the two methods was calculated.The percentage of subjects categorized in opposite quintiles (i.e. the gross misclassification) was also observed.

Results
Table 1 shows the general characteristics of the children.Thirty-five of the children's fathers had either not completed primary school or had never attended school, 9 had completed primary school, 7 had completed high school and 4 had a college degree.Fifty-four of the mothers and 41 of the fathers were black or mulatto, 5 mothers and 16 fathers were white, 1 mother and 2 fathers were Asian-Brazilians and only 2 fathers were Brazilian Indians, whereas none of the mothers were Indians.
Table 2 compares the nutrient intakes data obtained using the 24HRs (mean of the 2 days) and the FFQ.In most cases, the quantity of nutrient intakes shown by the FFQ was greater than those shown by the 24HR.
The Pearson correlation coefficient for crude energy and nutrient intakes ranged from -0.04 for vitamin C, to 0.59 for calcium, and the mean coefficient was 0.30 (Table 3).After adjusting for energy intake, half of the coefficients increased and the other half decreased, ranging from -0.02 for vitamin C, to 0.57 for phosphorus.The mean energy-adjusted coefficient was 0.30 (Table 3).As expected, adjusting for intra and between-individual variation led to an increase in the coef-ficient values, ranging from -0.03, for vitamin C, to 0.93 for calcium (Table 3).The mean energyadjusted and deattenuated coefficient was 0.46.
The mean proportion of subjects classified within one quintile was 66%, whereas the mean proportion of subjects grossly misclassified was 3.5% (Table 3).The nutrients that showed the highest percentage of exact agreement were energy, cholesterol, vitamin E, calcium and iron (Table 3).The highest percentage of gross misclassification was found in vitamins C and E (Table 3).
The Bland & Altman plots analysis showed that the limits of agreement for all nutrients, except riboflavin and vitamins A, E, C and B12, were not too large (Table 3).The FFQ/24HR ratio was close to 100% for all nutrients, except for niacin (53%) and vitamin E (158%).Considering the criteria adopted by Hong et al. 24 and Watson et al. 25 , the analysis showed good agreement for all nutrients, except saturated fat and vitamins E and B6 (fairly good agreement) and fat (poor agreement).For example, for energy (transformed into natural log, ln), the difference between the two measurements was equal to 0.03, whereas the standard deviation of the average energy intake data of the reference method (the 24HR) was 0.34.Thus, adopting the criteria cited above, since the difference was smaller than the standard deviation, it can be said that there is good agreement between the two methods for the energy intake estimate.With respect to fat, however, the difference between the two estimates was equal to 0.09 and * Crude Pearson's correlation coeffi cient between energy and nutrient intake data obtained using the two 24HR and the FFQ; ** Energy-adjusted Pearson's correlation coeffi cient between nutrient intake data obtained using the two 24HR and the FFQ; *** Deattenuated and energy-adjusted Pearson's correlation coeffi cient between energy and nutrient intake data obtained using the two 24HR and the FFQ; # Percentage of subjects whose intake was classifi ed within one quintile, using data from the FFQ and the mean of the two 24HR; ## Percentage of subjects whose intake was classifi ed in opposite quintiles, using data from the FFQ and the mean of the two 24HR; ### For energy intake, the coeffi cient was deattenuated but not energy-adjusted.
the standard deviation of the mean fat intake data of the 24HR was 0.44.Thus, the standard deviation is almost 5 times greater than the difference; thus, it can be said that there is a poor agreement.Furthermore, the fitted regression lines indicate a significant linear trend for all nutrients (p ≤ 0.05), except energy and niacin.This shows that there is a dependency between the difference and the average of the two methods; thus, one can expect a greater level of error associated with the estimation of extreme levels of food intake.

Discussion
The results show that the FFQ developed for this study is a valid method for estimating the nutrient intake of this sample, which has specific characteristics: low levels of literacy, low income, poor food availability and consumption of a small variety of foods.Major efforts were made in order to make the FFQ culture-sensitive.The list of foods was based on information obtained from the 24HR.For these reasons, only the most commonly consumed or available foods were included and therefore limiting the length of the list.Four other features of the FFQ may facilitate the use of this method as an appropriate tool for estimating the nutrient intake of a population sample with low socioeconomic status: (1) it not is self-administered; (2) it does query portion size; (3) the frequency options are presented in a simple and less "mathematical" form; and (4) the foods contained in the questionnaire were those typically consumed by the local population.Most of the estimates obtained using the FFQ were significantly greater than those obtained using the 24HR; these findings are similar to those reported by other studies 14,26,27,28 .This might be due to the fact that the FFQ enabled the reporting of foods which were not consumed on a regular basis.Another possible reason is the fact that we did not query portion sizes in the FFQ.This decision was made considering that: (1) frequency of consumption tends to have a greater bearing on dietary intake than portion sizes; and (2) the inclusion of portion sizes may make the questionnaire more difficult to answer 21 .
As proposed by a consensus paper 29 , several methods were used to evaluate the relative validity of the FFQ in addition to the Pearson correlation coefficient.The relatively high percentage of nutrient intakes classified within one quintile and the small number of grossly misclassified intakes show that, although the FFQ may overestimate the intake of some nutrients, it is capable of adequately ranking the nutrient intake of the subjects.Furthermore, the Bland & Altman plots indicate good or fairly good agreement for almost all nutrients, with reasonably narrow LOA; however, a greater level of error can be expected when estimating extreme levels of food intake and certain nutrients.
This FFQ was designed to provide a general assessment of the children's diets and to be used in our cohort studies whose aim is to investigate the determinants of anemia and child growth in this population.Thus, it was important that the FFQ was able to obtain valid estimates of protein, iron, folate and vitamin B12 intake.With respect to these nutrients of special interest the FFQ met the three recommended validity criteria established for epidemiological studies 30 : correlation coefficients above 0.5 (except vitamin B12, which was 0.48); more than 50% of subjects were correctly classified; and less than 10% of subjects were grossly misclassified.Thus, this FFQ provides valid information for these nutrients of special interest.
Less accurate results were found for some nutrients, such as carbohydrate, fat, and vitamins A, C and E. Similar findings were reported for fat and carbohydrate by another Brazilian FFQ validity study among children conducted in a very different region of the country, suggesting that the assessment of these nutrients is particularly complex 14 .
Correlation coefficients for vitamins A, E and C tended to be small and the Bland & Altman plots showed a greater overestimation of these nutrients among respondents who reported a lower intake in the 24HR.A similar finding was reported by another study 12 of 50 low-income Mexican women.Although this assessment did not include children, the diet and economic conditions of the sample population were similar to those of the present study's population sample.Both samples showed a low intake of fruit and vegetables (which are sources of vitamin A and C).In Acrelândia, the population only eats fruit and vegetables when they are in season (when they are cheaper and more available) or at the beginning of the month, when they receive their wages.In this context, the FFQ is probably better able to detect the intake of these foods than a limited number of 24HR recall days.The findings of a FFQ and two 24HR recall days conducted by Cullen et al. 31 show poor correlation coefficients for intake of vegetables, fruits and fruits juices in American children aged between 10 and 17 years.It should be mentioned however that the intake data of this study was self-reported by the children.The inclusion of more recall days, spread across seasons and a period of months, could lead to a possible improvement in results; however, this was not feasible in the case of this study due to the field conditions.Other strategies may be tested in order to improve these poor validity estimates such as: conducting the FFQ in a diet history format that assesses daily meals, incorporating the successful features of the Brazilian Previous Day Food Questionnaire 32 ; asking cross-check questions regarding intake of fruits and vegetables in different periods of the month; including questions about household food availability; and using biomarkers of nutrient intake.
Although our sample met the guidelines for validation studies 29, the small sample size may have been a limiting factor in the present study.However, a study with a relatively large sample size (n = 224 children) and four 24HR obtained modest correlation coefficients between the 24HR and the FFQ 33.This highlights the many difficulties faced by researchers in assessing the dietary intake of children such as the overestimation of intake by the FFQ method, as found in the present study 34,35 .Another limitation is the fact that the first 24HR was used for both developing and validating the FFQ, which may have increased the observed correlations.Unfortunately, it was not possible to conduct a third 24HR -that could have been used with the second for validation purposes -due to the field conditions and budget restraints.
In conclusion, our FFQ developed for evaluating the diet of schoolchildren living in the Brazilian Western Amazon was effective in adequately ranking most nutrient intakes.However, the addition of questions to obtain information on household food availability and behavior should be considered, since these factors can play an important role in the nutrient intake of this population.Finally, a test of the reproducibility of this FFQ is also recommended.

Table 1
General characteristics of the 61 children aged between six and nine years living in the city of Acrelândia, Acre State, Brazil, who participated in the study.

Table 2
Data on daily dietary intakes of energy and nutrients consumed by 61 children aged between six and nine years living in the city of Acrelândia, Acre State, Brazil, obtained using two 24-hour diet recalls (24HR) and one food-frequency questionnaire (FFQ) conducted with mothers or guardians.

Table 3
Results and summary statistics used for the assessment of the validity of the food-frequency questionnaire (FFQ) showing the mean of two 24-hour dietary recalls (24HR), responded by the mothers or guardians of 61 children aged between six and nine years, living in the city of Acrelândia, Acre State, Brazil.