How reproducible is self-reported information on exposure to smoking, drinking, and dietary patterns? Evidence among Brazilian adults in the Pró-Saúde Study

ABSTRACT CONTEXT: Epidemiological studies of the validity and reliability of self-reported information on important risk factors for non-communicable chronic diseases are scarce in Brazil. OBJECTIVE: We evaluated the test-retest reliability of information – overall and stratified by gender, age and education – on active and passive smoking, alcohol intake and aspects of dietary habits. TYPE OF STUDY: Test-retest reliability. SETTING: Universidade do Estado do Rio de Janeiro, Rio de Janeiro, Brazil. PARTICIPANTS: 192 University employees. PROCEDURES: Self-administered questionnaires were completed on two occasions, two weeks apart. MAIN MEASUREMENTS: Kappa Statistics; Intraclass Correlation Coefficient. RESULTS: Information on smoking status and pack-years smoked had almost perfect levels of agreement, respectively, kappa = 0.97 (95% CI, 0.92-1.00), and intraclass correlation coefficient = 0.93 (CI 95%, 0.89-0.96). Characteristics of alcohol intake yielded substantial levels of agreement (kappa ranging from 0.62 to 0.69). The reproducibility of the information on dietary habits varied from 0.67 to 0.79 (kappa). No clear-cut patterns could be identified comparing information by age or gender. There was a slight tendency towards greater reliability among people with higher levels of education. CONCLUSION: The reproducibility of information on smoking, drinking, and dietary patterns ranged from substantial to excellent, as investigated in the Pró-Saúde Study, a longitudinal investigation recently launched in Rio de Janeiro.


INTRODUCTION
There are no published reports of epidemiological studies among Brazilian adults regarding the validity and reliability of self-reported information on smoking, exposure to passive smoking, and alcohol intake, which are important risk factors for non-communicable chronic diseases.A single Brazilian study assessing the quality of information regarding habitual diet has been made. 1 In addition, no information exists about how the quality of self-reported information might vary across population subgroups (defined, for example, by gender, age, or socioeconomic status).Research conducted in the United States, however, has shown that information quality may vary with these and other characteristics of the study population, 2-4 leading to measurement errors that may in part explain discrepant results between epidemiological studies. 5nformation on health-related behavior, generally obtained by means of interviews or self-administered questionnaires, may be distorted by respondents' recall errors, or by their wish to give responses which would be considered socially desirable. 6,7Thus, the evaluation of the quality of information obtained from epidemiological research is crucial, and the estimation of validity and reproducibility indicators is the main tool for conducting such evaluations. 8his paper reports on the test-retest reliability of information on smoking, exposure to passive smoking, alcohol intake, and aspects of diet (fruit and vegetable intake).It also presents stratum-specific estimates by gender, age and educational attainment.Questions on these and other exposures were subsequently included in a multidimensional, self-administered questionnaire applied to 4,030 university employees in Rio de Janeiro, Brazil.This was the Pró-Saúde Study, launched in 1999 with the objective of investigating relationships between social factors and health outcomes.

Study population
The test-retest study was conducted in June 1999.The eligible participants were 1,120 employees who were at that time working at the University where the Pró-Saúde Study was subsequently launched, but who were not permanent employees (i.e. they were not on the administrative "tenure track" that exists in Brazil), although they were not shortterm "temporary" workers either.Their job activities were similar to those performed by permanent employees, but they were considered ineligible for the main study because the prospects for their long-term follow-up were uncertain.A 20% systematic sample of 224 potential respondents (e.g. the 5 th , 10 th , etc.) was drawn from official listings.

Test-retest reliability study
The conditions under which the questionnaire was administered were identical to those of the main study, i.e. during working hours, in University classrooms, and with help available from trained personnel.Among 224 employees sampled, 202 (90.2%) agreed to fill out the questionnaire.After its completion, they were invited to repeat the procedure two weeks later, with the explanation given being that the purpose was to test the adequacy of the questionnaire, rather Responses to the following closed-ended questions are analyzed in the present paper: "How often do you eat fresh fruit?"; "How often do you eat vegetables?"; "Have you ever been a cigarette smoker, that is, have you smoked at least 100 cigarettes (five packs) during your lifetime?";"Do you currently smoke cigarettes?"; "In general, how many cigarettes per day did you smoke or do you currently smoke?"; "In total, for how many years did you smoke or have you been smoking?(do not include time periods when you did not smoke)".The following open-ended questions were also analyzed: "Are there other people who smoke at your workplace or at home?"; "In the last two weeks, did you drink any alcoholic beverage?";"In the last two weeks, on how many days did you drink any alcoholic beverage?";"In the last two weeks, on average, how many drinks did you have on each of those days when you drank?"

Statistical analysis
Agreement was measured for the entire sample, and also stratifying for gender, age (18-39 years vs. 40 years or more), and education (high school or less vs. college or more).The responses were analyzed in terms of assessing smoking status (current, former, or never); exposure to passive smoking (at workplace, at home, both or neither); alcohol intake during previous 2 weeks (yes or no), number of days of drinking (1, 2-5, or more than 5), and average number of drink units (1, 2-4, 5-7, or more than 7); usual frequency of fruit and vegetable intake (less than once a week, 1-3 days/week, 4-6 days/week, or daily).Pack-years of cigarettes smoked were calculated by multiplying the average number of cigarettes smoked daily by the duration in years of the smoking habit.
We compared frequencies reported at the test and retest by using kappa statistics (k) for categorical variables (active and passive smoking, and alcohol intake) and weighted kappa (k w -squared error weighting) 9 for ordinal variables (frequency and amount of alcohol intake; fruit intake and vegetable intake). 10The intraclass correlation coefficient (ICC) was utilized to measure agreement for the continuous variable pack-years smoked. 11According to criteria from Landis and Koch, 12 kappa values greater than 0.80 represent "almost perfect" agreement; those between 0.60 and 0.80 show "substantial" agreement; those between 0.40 and 0.60 show "moderate" agreement, and values below 0.40 represent "poor to fair" agreement.
The independent associations of gender, age, and education with test-retest agreement (outcome variable: "agree versus disagree") were investigated by fitting logistic regression models for each health-related behavior or exposure.

RESULTS
Overall, no substantial differences were observed between test and retest prevalence (Table 1, previous page).Information on smoking status and pack-years smoked was shown to be highly reliable: k = 0.97 (95% confidence interval, CI: 0.92-1.00)and ICC = 0.92 (95% CI: 0.88-0.95),respectively.A high level of agreement was also estimated regarding passive smoking (k = 0.73).Nevertheless, it is worth noting that the prevalence of passive smoking "at home only" estimated in the test (7.4%) was nearly double that of the retest (4.2%), which did not occur with passive smoking "at workplace only" or "at home and workplace".
Among diet-related characteristics, information on fruit intake showed the highest agreement (k w = 0.79), while information on intake of vegetables ranked lower (k w = 0.67).
No clear-cut patterns could be identified by comparing kappa statistics by gender and age strata (Table 2).However, it is worth not-ing that information on pack-years smoked showed lower reliability among younger employees.In addition, there was a slight tendency towards higher reliability among people with higher levels of education.
The association between agreement and potential sociodemographic predictors (gender, age, and education) was further investigatedexcept for smoking, as this yielded reliability values close to 1.0 (data not presented).The responses regarding number of alcohol units, intake of vegetables and fruit showed no association (p < 0.10) with any of these potential sociodemographic predictors.However, women were over three times more likely than men to agree on passive smoking information: crude odds ratio, OR = 3.3; (95% CI: 1.5-7.2);after adjusting for age and education, OR = 3.6 (95% CI: 1.6-8.3)(data not shown in table ).

DISCUSSION
Our results indicate good levels of reliability regarding information on important risk factors for major adult chronic diseases in this Brazilian study population.Reliability of the information on smoking, number of cigarettes smoked and exposure to passive smoking (except at home) ranged from substantial to almost perfect.Reproducibility of the other items regarding dietary patterns and alcohol intake ranged from moderate to almost perfect.
Studies conducted in the United States have also considered information on tobacco use and alcohol intake to be reasonably reproducible. 13,14The reproducibility of information on dietary patterns tends to show more variability depending on the data collection instrument utilized and the frequencies of intake of specific dietary items. 2,4he estimated measures of reliability have practical implications for future analyses of the Pró-Saúde Study data.We were especially interested in assessing possible differences across the responses given by sociodemographic subgroups.Gender, age, and education were each predictive of some variation in agreement for one or more risk factors, but no consistent effect of any of these characteristics on the reproducibility of information was observed.Therefore, the overall similar prevalence of the risk factors measured in both the test and the retest, and the absence of systematic differences in the information across sociodemographic strata suggest that potential misclassification of results in the main study would  result mostly from non-differential errors.Stein et al. 3 found similar results when investigating demographic predictors of agreement in information on chronic disease risk factors: there was no consistent effect of any demographic characteristic across risk factors, although some of them predicted higher levels of concordance for one or more risk factors.In contrast, in two other studies, 2,15 agreement on information about preventive health practices and physical activity variables tended to be lower among non-white than white women.
Although good reliability does not ensure validity, reliability studies are often conducted insofar as they provide information on consistency and stability of the answers. 2- 4, [13][14][15][16] One important limitation of the testretest method in estimating reproducibility must be considered.Any actual change in the condition or exposure under study, which may occur during the period between the test and the retest, will inevitably result in underestimated reliability.This may have been the case with the moderate reliability estimated in this study for the information on alcohol intake during the previous two weeks.

CONCLUSION
The need for reliability studies built into epidemiological investigations should not be overlooked, as levels of reliability can vary according to characteristics of the population and data collection methods.The reproducibility of information on smoking, drinking, and dietary patterns ranged from substantial to excellent, as investigated in the Pró-Saúde Study, a longitudinal investigation recently launched in Rio de Janeiro.

Souza Lopes How reproducible is self-reported information on exposure to smoking, drinking, and dietary patterns? Evidence among Brazilian adults in the Pró-Saúde Study than
the participants' responses.The procedure was repeated by 192 respondents (95.0% of the initial participants), without relevant variations in the proportions of participants in the test and retest according to age, gender, or schooling.

Table 2 . Test-retest agreement of self-reported information on active and passive smoking, patterns of alcohol intake and diet, by gender, age and education. Pró-Saúde Study, Rio de Janeiro, 1999 Strata Gender Age (years) Education Characteristics Men Women 18 -39 40 High school College or more or less or more Cigarette smoking
ANÁLISE ESTATÍSTICA: Estatística Kappa; Coeficiente de Correlação Intraclasse.RESULTADOS: Estatística.As informações sobre o hábito de fumar cigarros e a quantidade de