Validity of Food and Beverage Intake Data Obtained by Telephone Survey

OBJECTIVE: To evaluate the reproducibility and validity of data on food and beverage intake obtained by means of a telephone-based surveillance system. METHODS: Reproducibility and validity analyses were carried out in two random subsamples (n=112 and n=119, respectively) of the total sample (N=2,024) of adults (≥18 years) studied by the system in 2005 in the municipality of São Paulo, Southeastern Brazil. Indicators evaluated included protective factors (daily or almost daily intake of fruit and vegetables) and risk factors (daily or almost daily intake of soft drinks, frequent intake of foods containing saturated animal fat, and abusive intake of alcoholic beverages) for the development of chronic diseases. Reproducibility was studied by comparing the results of the original telephonic interview with those of another interview carried out 7-15 days later. Validity was analyzed by comparing the results of the telephone interview with those of three 24-hour recalls (our gold-standard) carried out up to 5 days following the original interview. RESULTS: The frequency of the studied indicators remained relatively constant between the fi rst and second telephone interviews, with kappa coeffi cients ranging from 0.57 to 0.80, indicating good reproducibility for all indicators. In relation to the gold standard, there was a trend towards overestimating the frequency of intake of protective foods, but of foods associated with increased risk of chronic diseases. Sensitivity and specifi city were high for indicators of consumption of risk-associated foods (close to 80%), and variable in the case of protective foods (42% to 80%). CONCLUSIONS: The evaluation showed evidence of good reproducibility and adequate validity for most indicators employed in the system, which indicates that maintaining this system operational in coming years will provide a useful instrument for evaluating public policies for the promotion of a healthy diet and for control of non-communicable chronic diseases with a dietary component in Brazil.


INTRODUCTION
Non-transmissible chronic diseases (NTCDs) play an important role in the current health profi le of human populations.Estimates from the World Health Organization (WHO) indicate that these diseases are responsible for 58.5% of all deaths worldwide, and 45.9% of the global burden of disease. 17In Brazil, NTCDs account for 62.8% of all deaths for which the cause is known.a Time series of mortality statistics available for Brazilian state capitals indicate that the proportion of NTCD deaths increased more than three-fold between the 1930's and 1980's. 6WHO global estimates also indicate that a limited number of risk factors accounts for the great majority of NTCD deaths and for a substantial fraction of the disease burden related to such diseases.Noteworthy among these factors, in addition to smoking and obesity, are food and beverage intake characteristics and physical activity patterns. 17nce 2006, a surveillance system has been operational in Brazil that monitors the frequency and distribution of NTCD risk factors in the capitals of the 26 Brazilian states as well as in the Federal District.This system, which is called Vigilância de Fatores de Risco e Proteção para Doenças Crônicas não Transmissíveis por Inquérito Telefônico (VIGITEL -Risk Factor Surveillance for Non-Transmissible Chronic Diseases by Telephone Survey), is based on telephone interviews carried out continuously in probabilistic samples of the adult population with access to telephone in each of these cities.Estimates are adjusted for differences in the sociodemographic composition of these samples in relation to the composition of each city's total adult population.b VIGITEL was tested successfully in São Paulo in 2003 8 and retested in this same city and in another four state capitals in 2005.In the second test carried out in the city of São Paulo, the evaluation of the system's operation was coupled to a study of the reproducibility and validity of its indicators.The present article describes the results of this study with respect to food and beverage intake.Reproducibility and validity for indicators of physical activity and inactivity are described in Monteiro et al. 9

METHODS
The study employed subsamples extracted from the entire sample (N=2.204) of subjects studied by the VIGITEL system in the city of São Paulo in 2005.Sampling procedures for this system, described in detail in a previous publication, 8,b ensure that, in each city, representative samples of the population of individuals aged 18 years or older living in households with telephone are selected.Two systematic subsamples (n=115), with proportion of men and women similar to that of the whole sample, were randomly selected for participation in reproducibility and validity studies.Three subjects in the reproducibility subsample and six  May 4].Available from: http://portal.saude.gov.br/portal/arquivos/pdf/relatorio_vigitel_2006_marco_2007.pdf in the validity subsample refused to participate in these studies or did not complete the required interviews.The fi nal study sample comprised 112 subjects (44 men and 68 women) for the reproducibility study and 109 subjects (47 men and 62 women) for the validity study.Distribution in terms of age and schooling among men and women included in the reproducibility and validity studies did not differ signifi cantly from the distributions seen in the entire sample of men and women surveyed by the VIGITEL system in the city of São Paulo in 2005 (chi-square test with 5% signifi cance level).
We evaluated indicators of food and beverage intake associated with protection or increased risk of NTCDs.Factors associated with protection were daily or almost daily (≥ 5 days per week) intake of fruit, vegetables (including greens and other vegetables but excluding tubers such as potato and manioc), as well as fruit and vegetables combined.Indicators associated with risk were frequent intake (thus referred by the subject) of foods rich in saturated fats (red meet with excess fat without removal of visible fat or poultry with skin, and whole milk), daily or almost daily intake of non-sugar-restricted soft drinks (at least one can in ≥ 5 days per week) and abusive intake of alcoholic beverages (ingestion of over four doses for women or over fi ve doses for men in a same day during the last 30 days, considering one dose of alcoholic beverage as equivalent to one beer, one glass of wine, or one dose of distilled spirits).
For the reproducibility study, subjects were contacted by phone seven to 15 days after having participated in the system's original interview, when they were asked to respond once again to the block of questions on food and beverage intake.The second interviewer was always different from that which had conducted the fi rst interview.The results obtained in the two interviews were compared in terms of the proportion of subjects exposed and the agreement between the individual classifi cation of each subject with respect to exposure or not to the factor under study.In this latter case, the degree of agreement between the two interviews was evaluated using the kappa coeffi cient, classifi ed as follows: above 0.80, almost perfect agreement; between 0.61 and 0.80, substantial agreement; between 0.41 and 0.60, moderate agreement; between 0.21 and 0.80, fair agreement; and below 0.21, slight agreement. 5r the study of validity, subjects underwent three 24-hour dietary recalls, carried out also by telephone within a seven-day period up to 15 days after the original interview by the system.Two of the recalls referred to weekdays, and one to a Saturday, Sunday, or holiday.This 24-hour recall consisted in requesting that the subject inform the type and amount of food and drink consumed in the last 24 hours. 1 If the spontaneous report failed to contain mention of consumption of one or more of the items of interest (fruit, vegetables, red meat, poultry, milk, soft drinks, and alcoholic beverages), we asked the subject directly about the occasional consumption of the item or items not mentioned.
In the case of red meat, we asked subjects whether there was any visible fat in the portion served, and whether or not this fat was ingested.For poultry, we asked whether skin was present in the serving and whether or not it was ingested.For milk, we asked whether it was whole, reduced fat, or fat free.For soft drinks, we asked about the amount and type ingested (with or without sugar restriction), and for alcoholic beverage consumption, we asked about the amount of doses ingested.
The validity study consisted in comparing results obtained in the standard VIGITEL telephone interview with the three 24-hour recalls (gold-standard) in terms of the proportion of individuals exposed to the factor of interest and the accuracy of the telephone interview in classifying individuals based on their true exposure status as determined by the gold-standard.We considered as truly exposed to daily or almost daily intake of fruit, vegetables, and soft drinks subjects with recorded consumption of these items in at least two out of three 24-hour recalls.In the case of frequent intake of red meat without removal of visible fat, poultry with skin, or whole milk, we considered as exposed subjects with recorded consumption of these items in at least one of three 24-hour recalls.We considered as exposed to abusive intake of alcoholic beverages subjects that reported daily consumption of more than four (women) or more than fi ve (men) doses in at least one of three 24-hour recalls.
To evaluate the ability of the telephone interview to correctly classify individuals truly exposed to the factor according to the gold-standard, we calculated the proportion of these subjects diagnosed as exposed in the telephone interview (sensitivity of the telephone interview).To evaluate the telephone interview's ability to correctly classify subjects truly unexposed to the factor as established by the gold-standard, we calculated the proportion of these subjects that were not diagnosed as exposed in the telephone interview (specifi city of the telephone interview).
In addition, based on the three 24-hour recalls, we estimated the weekly frequency of intake (days per week) of each of the foods and drinks of interest for each subject.We thus multiplied by 2.5 the number of weekdays, and by 2.0 the number of Saturdays, Sundays, or holidays, with recorded consumption for each type of food and drink.We then calculated, by interpolation, the median weekly frequency of intake of a given food or drink among subject classifi ed by the telephone interview as either exposed or unexposed to the critical frequency of intake defi ned for that indicator (≥5 days per week for fruit, vegetables, and soft drinks; ≥1 day in the preceding month for abusive intake of alcoholic beverages; and frequent intake for the remaining indicators).The statistical signifi cance of the difference between the two groups was evaluated using the test of difference in medians. 7e study was approved by the Research Ethics Committee of the Faculdade de Saúde Pública da Universidade de São Paulo.

RESULTS
Table 1 compares the frequencies of indicators of food and beverage intake estimated based on the original VIGITEL interviews and on repeat interviews with the same subjects carried out by other interviewers seven to 15 days after the fi rst one.Differences were negligible with respect to intake of foods with high saturated fat content, soft drinks, and alcoholic beverages, and relatively minor with regard to fruit and vegetable intake.The kappa coeffi cient indicated "substantial" (0.61 to 0.80) agreement between interviews for all protective and risk factors, with the exception of simultaneous intake of fruit and vegetables, which showed "moderate" agreement (kappa = 0.57).There were variations between levels agreement for men and women, but these did not characterize systematic differences according to gender.
Table 2 compares the frequencies of food and beverage intake indicators estimated based on the original VIGITEL interview and on three 24-hour recalls with the same subjects carried out within a one-week period up to 15 days after the original interview.The frequency of indicators associated with protection against NTCDs was higher when estimated based on the telephone interview than when based on the 24-hour recalls, but differences were relatively small (46.8% vs. 38.5%;61.5% vs. 55.1%; and 33.0% vs. 25.7% for fruit, vegetables, and both fruit and vegetables, respectively).Frequencies for NTCD risk factors obtained in the telephone interview were lower than those obtained in the 24-hour recalls for foods rich in saturated fat and higher for soft drinks and alcoholic beverages.Again, differences were of relatively small magnitude.Sensitivity of the telephone interview, using as a reference the 24-hour recalls, ranged from 46.4% for simultaneous intake of fruit and vegetables to 87.5% for soft drinks.Specifi city based on the same reference ranged from 44.9% for vegetables to 94.1% for soft drinks.Reasonable levels of both sensitivity and specifi city (close to or higher than 70%) were found for fruit, whole milk, soft drinks, and alcoholic beverages.Table 3 presents estimates based on three 24-hour recalls for the median weekly frequency of intake of foods and beverages associated with protection or risk of NTCDs.In this Table, we compare the medians of individuals classifi ed by the telephone interview as exposed or non-exposed to the critical intake frequency defi ned for each of the indicators.We found that median weekly frequency of intake of all foods and beverages was higher in the group of subjects that declared intake higher than the critical frequency in the telephone interview.With the exception of vegetables, differences between these subjects and the remainder tended to be substantial and statistically signifi cant (p<0.05).

DISCUSSION
The present study shows that indicators of food and beverage intake obtained by means of telephone interviews through the VIGITEL system have good reproducibility, be it at the collective level (similar frequencies of risk factors in the population obtained through repeated interviews) or individual level (kappa coeffi cients compatible with substantial or moderate agreement in individual classifi cation of exposure to risk factors).The good reproducibility indicates that interviews are conducted in a standardized fashion.This was expected, given the use of computer-assisted telephone interview system in which the interviewer is induced to read questions to the interviewee, thus avoiding interpretations or answer induction.It also indicates that interviewees understand the questions and have no diffi culty in answering them, providing answers that remain consistent with time.What is expected from a surveillance system such as VIGITEL is the provision of estimates that, in addition to being accurate, are also reproducible, ensuring that time-related variations in indicators translate actual variation in the behavior of the population, rather than instability of the indicators. 2,13e indicators employed yielded results that were in general close to those obtained using a method of greater accuracy.The validity of the indicators was evaluated at both the collective and individual levels.Collectively speaking, there was evidence of a trend towards overestimating the frequency of indicators of intake of fruit and vegetables, which may indicate that subjects are aware of the positive effects of these items on health.However, similar bias was not detected for the intake of items associated with risk.At the individual level, we found high levels of sensitivity and specifi city for indicators of intake of risk-associated items (usually close to 80%) and unsteady levels for indicators of fruit and vegetable intake (ranging from 42% to 80%).On the other hand, with the exception of vegetables, the comparison of median weekly intake levels obtained using the three 24-hour recalls indicated expressive and statistically signifi cant differences in favor of subjects classifi ed by the VIGITEL system as exposed to the critical frequency of food or beverage intake.This indicates a good discrimination capacity for the simplifi ed questionnaire used during the telephone interview.Poorer performance in estimating vegetable intake may be due to greater diffi culty among subjects to recognize foods pertaining to this group.In the version of the questionnaire used by the system in 2007 we attempted to overcome this issue by providing greater detail in the questions on vegetable consumption.
The design employed for assessing reproducibility had no limitations, given that major sources of intra-subject and interviewer variation were accounted for by repeating the same interview with a different interviewer.Moreover, the kappa coeffi cient, used to evaluate the reproducibility of the telephone interview, is the most recommended measure for evaluating reproducibility of instruments that classify individuals as exposed or unexposed to a given condition. 14mmon limitations among validity studies involve the use of inadequate gold-standards as well as the use of samples that are not representative of the population to which the indicators will be applied. 14Regarding the former, it would have been desirable for the present study to extend the gold-standard daily recall questionnaires to an entire week, or even a longer period in the case of the indicator of alcohol abuse, which uses a reference period of 30 days.The use of only three daily recalls as a gold-standard may have underestimated the validity of frequency questionnaires employed in VIGI-TEL, given that the measures obtained in frequency questionnaires refer to the subjects' usual diet whereas daily recalls only estimate the food intake in few and limited days. 10In any case, an increase in the reference period of daily recalls would likely be unfeasible, and could in fact compromise the quality of individual responses.Carrying out at least three 24 hour recalls in different days of the week is the procedure most widely used as a reference method for the validity evaluation of food frequency questionnaires. 1,16garding the representativeness of our sample, the probabilistic selection of studied individuals ensures that the results obtained are applicable to the performance of VIGITEL throughout the city of São Paulo, but not necessarily to other Brazilian cities where this system is being implemented.In this sense, it will be important to carry out similar studies in at least one state capital of each of the country's fi ve Great Regions.Two important strengths of the validity assessment conducted by the present study are the provision of estimates for the specifi city and sensitivity of the indicators, a recommended procedure given the character of the evaluated indicators , 14 and the comparison of median weekly intake of food or beverage items among subjects classifi ed as exposed or unexposed to the intake condition defi ned by the indicator of risk or protection.
The majority of studies of reproducibility and validity in food intake surveys compare different data collection instruments and techniques in terms of the amount of energy, nutrients, and food groups consumed, translated into continuous variables, by applying statistical techniques such as correlation tests and tests for differences in means. 3,4,11,15This limits the comparability of these results to those of the present study.However, in the specifi c case of fruit and vegetable intake, our results on the reproducibility and validity of the VIGITEL indicators are similar to those obtained in the United States by another telephone-based surveillance system. 10,12 conclusion, the indicators of food and beverage intake employed by the VIGITEL system seem reproducible and accurate.Maintaining this system operational in years to come will supply the country with a useful instrument for evaluating public policies for promotion of a healthy diet and for control of diet-related nontransmissible chronic diseases.* Critical intake frequency defi ned in the telephone survey as equivalent to ≥ 5 days per week for fruit, vegetables, and soft drinks and ≥ 1 day in the last 30 days for alcoholic beverages; in the remaining cases, frequent intake of the food item.

Table 1 .
Frequency (%) of indicators of food and beverage intake obtained in two successive telephone interviews administered to the same subjects.Municipality of São Paulo, Brazil, 2005.

Table 2 .
Frequency (%) of indicators of food and beverage intake among adults estimated based on telephone interview and three 24-hour recalls.Municipality of São Paulo, Brazil, 2005.

Table 3 .
Weekly frequency of food and beverage intake estimated based on three 24-hour recalls among adults classifi ed by a telephone survey as exposed or unexposed to the critical intake frequency.*Municipality of São Paulo, Brazil, 2005.