Comparison of estimates of population-based surveys

OBJECTIVE: To compare estimates obtained through household and telephone surveys for monitoring, intervention and development of health policies. METHODS: The study analyzed data from 2,526 and 1,900 individuals aged 18 and over, living in Campinas, SP, interviewed by the household survey (ISACamp) and by the telephone survey (Vigitel), respectively. Sex, age and schooling were used to characterize the studied population. Prevalence estimates and 95% confi dence intervals were utilized in the analysis. The estimates of the sociodemographic characteristics of the population were compared by t-test for two independent samples and the comparison of the other estimates, according to the type of survey, was performed using Poisson regression. RESULTS: No statistically signifi cant differences were found between the estimates obtained by the two surveys analyzed for overall prevalence of: overweight/obesity, smoking status, mammography in the prior year and Pap smear at least once in life. However, for the estimates of worse perceived health status, health plan affi liation, mammography at least once in life and Pap smear in the prior year, signifi cant differences were found, with a tendency to overestimate data from the telephone survey, except in the case of worse perceived health status. CONCLUSIONS: These fi ndings point to the need for further studies, which may contribute to a better understanding of the differences, given that telephone surveys can provide quick and essential information for monitoring modifi able risk factors for the assessment of interventions and to develop policies promoting health in Brazil. DESCRIPTORS: Chronic Disease, epidemiology. Risk Factors. Prevalence. Epidemiological Surveillance. Health Surveys. Comparison of estimates of surveys Francisco PMSB et al Population-based studies are an important tool for estimating a population’s health condition indicators, health-related behavior, access to and use of services and morbidity.3 Although summary measures of health are necessary, the availability of indicators which enable different dimensions of the disease process to be assessed make it possible to produce formulations for the wider health scene.2 With the increased prevalence of non-communicable chronic illness, there is greater interest in studies whose data on signs, symptoms and risk factors are self-reported by the population. The data may be collected through face-to-face or telephone interviews or through self reported questionnaires. They allow the accompaniment of the population over time by means of probabilistic samples, at a lower cost and with simplifi ed fi eld work.3,15,18 National health surveys carried out in the country describe indicators at a metropolitan level, i.e., they do not provide suffi cient information to assess and monitor health campaigns carried out by the public health system,3 which aims to decentralize management at the municipal level. In the last few decades, the research of these surveys stands out due to their capacity to measure modifi ed risk factors and, increasingly, valid data is becoming necessary for decision making, be they clinical or for public health interventions.3 Household health surveys enable more detailed information on a wide range of topics to be obtained.3,16 The more concise, telephone interviews cost less and are quicker in obtaining and divulging and obtaining information.4,15,16 The ISACamp (Campinas Health Survey) is a crosssectional population-based study carried out by the Health Situation Analysis Collaborating Centre of the Faculdade de Ciências Médicas da Universidade Estadual de Campinas. Household interviews were used to analyze health status, lifestyle profi le (risk factors for chronic illness) and use of health care services for different sectors of the population, assessing the degree of social equality/inequality. The Vigitel (Telephone Survey System for Monitoring Risk and Protection Factors for Chronic Illness) is population based research established by the Ministry of Health in 2006. This research aims to continuously monitor the frequency and distribution of risk and protection factors for chronic disease in the Brazilian state capitals and in the federal district.a In 2008, the municipality of Campinas was included in this research. INTRODUCTION a Ministério da Saúde. Secretaria de Vigilância em Saúde. Secretaria de Gestão Estratégia e Participativa. Vigitel Brasil 2008: vigilância de fatores de risco e proteção para doenças crônicas por inquérito telefônico. Brasília (DF); 2009. b Alves MCGP. ISA-CAMPINAS 2008/09. Plano de Amostragem. Campinas: UNICAMP; 2009 [cited 2012 Jan 5]. Available from: http://www. fcm.unicamp.br/centros/ccas/arquivos/plano_de_amostragem.pdf Telephone interviews are the cheapest and most fl exible method for collecting population data and can achieve estimates comparable with those of face-to-face interviews. Household and telephone interviews provide similar estimates.7,8,10,16 However, the high rates of notresponding or not-covered found in telephone surveys may affect the quality of the estimates.4,20,22 The aim of this study was to compare estimates obtained from household and telephone surveys in order to develop, carry out and monitor health policies.

Population-based studies are an important tool for estimating a population's health condition indicators, health-related behavior, access to and use of services and morbidity. 3Although summary measures of health are necessary, the availability of indicators which enable different dimensions of the disease process to be assessed make it possible to produce formulations for the wider health scene. 2 With the increased prevalence of non-communicable chronic illness, there is greater interest in studies whose data on signs, symptoms and risk factors are self-reported by the population.The data may be collected through face-to-face or telephone interviews or through self reported questionnaires.They allow the accompaniment of the population over time by means of probabilistic samples, at a lower cost and with simplifi ed fi eld work. 3,15,18tional health surveys carried out in the country describe indicators at a metropolitan level, i.e., they do not provide suffi cient information to assess and monitor health campaigns carried out by the public health system, 3 which aims to decentralize management at the municipal level.In the last few decades, the research of these surveys stands out due to their capacity to measure modifi ed risk factors and, increasingly, valid data is becoming necessary for decision making, be they clinical or for public health interventions. 3usehold health surveys enable more detailed information on a wide range of topics to be obtained. 3,16The more concise, telephone interviews cost less and are quicker in obtaining and divulging and obtaining information. 4,15,16e ISACamp (Campinas Health Survey) is a crosssectional population-based study carried out by the Health Situation Analysis Collaborating Centre of the Faculdade de Ciências Médicas da Universidade Estadual de Campinas.Household interviews were used to analyze health status, lifestyle profi le (risk factors for chronic illness) and use of health care services for different sectors of the population, assessing the degree of social equality/inequality.Telephone interviews are the cheapest and most fl exible method for collecting population data and can achieve estimates comparable with those of face-to-face interviews.
Household and telephone interviews provide similar estimates. 7,8,10,16However, the high rates of notresponding or not-covered found in telephone surveys may affect the quality of the estimates. 4,20,22e aim of this study was to compare estimates obtained from household and telephone surveys in order to develop, carry out and monitor health policies.

METHODS
Data on the adult (over 18) population from the 2008 ISACamp and Vigitel (Campinas) surveys were used.
The household survey sample -ISACamp, was obtained through probabilistic two-stage cluster sampling procedures: census tract and households.In the fi rst stage, 50 census tract with probability proportional to size were randomly selected.The draw was systematic and the sectors were in order of percentage of heads of household with university education, producing implicit stratifi cation according to the level of education of the head of the household.In the second stage 2,150, 700 and 3,900 households were randomly selected for adolescents, adults and the elderly respectively.
The questionnaire included information on the interviewees', the households' and the families' state of health, use of health care services, habits related to health, height, weight and socio-economic conditions.This information was obtained for the sample representing the population ≥ ten years old, through household interviews carried out by trained and supervised interviewers.
The sample size was calculated taking into consideration the situation corresponding to maximum variability for the frequency of the events studied (p = 0.50), coeffi cient of confi dence of 95% in determining the confi dence intervals (z = 1.96), sampling error between 4% and 5% and design effect = 2, totaling 1,000 individuals in each of the pre-defi ned age ranges: adolescents (ten to 19 years old), adults (20 to 59 years old) and elderly (≥ 60 years old).The sample size was corrected to 1,250 people for an 80% response level.Information on all of the inhabitants in the selected age range were collected in each randomly selected household.b c Lee S, Davis WW, Nguyen HA, McNeel TS, Brick JM, Flores-Cervantes I. Examining trends and averages using combined cross-sectional survey data from multiple years.CHIS Methodology Paper.Los Angeles; 2007[cited 2010 Sep 19].Available from: http://www.chis.ucla.edu/pdf/paper_trends_averages.pdf The fi nal weight given to each individual resulted from multiplying the design weight, the weight of no responses and the post-stratifi cation weight, taking into consideration the distribution according to gender and age ranges (10-19, 20-39, 40-59, ≥ 60 years old) of the Seade-2007 (State System of Data Analysis Foundation).
The sampling procedures adopted by Vigitel aimed to obtain probabilistic samples of adults resident in households served by at least one landline.The sample selection took place in two stages: random selection of landline numbers and of the inhabitant to be interviewed.A systematic random selection of 5,000 landline numbers was made from the Telefônica phone company's electronic records of residential landlines, ordered by prefi x, leading to implicit stratifi cation according to area of the municipality.In the second stage, the randomly selected numbers were then re-drawn and divided into 25 groups of 200 lines.Each group had the same proportion of numbers by telephone prefi x. a The data from the probabilistic sample of the ≥ 18 year old population were collected through telephone interviews conducted by a company hired for the purpose.The team of interviewers received training and were continuously supervised throughout the data collection.a The questions were about: socio-economic and demographic characteristics; eating and exercise patterns associated with non-communicable chronic disease; characteristics indicating body composition; frequency of alcohol and tobacco consumption; the interviewees' self-rated state of health and references to prior medical diagnoses of high blood pressure, cholesterol and diabetes.
Calls were made to 4,800 numbers distributed throughout 25 replicas, identifying 2,773 eligible numbers in Campinas.The individuals were numbered and one of them was randomly selected to be interviewed for each eligible number, and permission sought from the users to take part.
The fi nal weight applied to each interviewee was the result of multiplying the design weight (inverse of the number of telephone lines in the interviewee's home multiplied by the number of adults in the interviewee's home) by the post-stratification weight (relative frequency of individuals in the population of the 2000 Census divided by the relative frequency in the weighted sample).For the post-stratifi cation, 36 strata were considered according to sex, age group (18-24, 25-34, 35-44, 45-54, 55-64 and 65 and over) and level of education (0-8, 9-11 and 12 or more years of schooling).a In total, 4,426 individuals aged ≥ 18, resident in Campinas in 2008 were included, 2,526 were interviewed by ISACamp and 1,900 by Vigitel.
The indicators (variables) used to compare the survey estimates were: being overweight (BMI ≥ 25 kg/m 2 ), smoking (current smoker), self-rated health (worse perceived health), having health insurance, in women ≥ 40 having had a mammogram and in women ≥ 20, a smear test at least once and in the previous 12 months.
Sex, age group and education (in years of schooling) were used to characterize the population.The data from both surveys were analyzed based on a combined archive.The variables were renamed and classifi ed as values and identical labels for the same categories of response.A new variable was created to identify the origin of the information (ISACamp or Vigitel), making comparisons possible.c The prevalence of the selected variables and their respective 95% confi dence intervals were estimated.In addition to overall estimates, prevalence estimates were also calculated according to gender, age group and education.
The estimates of prevalence ratios used to compare the surveys were obtained using Poisson regression, considering the characteristic studied as a dependent variable and the variable created to identify the origin of the survey (household or telephone) as an independent variable.According to Lee et al c (2007), if the independent variable is statistically signifi cant, there is evidence to reject the hypothesis that the estimates from the two types of survey are similar, with a descriptive level of 0.05 for the Wald test.
In order to control possible differences in the surveys samples' distributions, due to the dissimilar post-stratifi cation weights based on different years (ISACamp -Seade 2007 and Vigitel -2000 Census), age, gender and education were used as adjustments in the regression models.
The Stata 11.0 program, which enables the aspects of the complex sample design to be considered, was used.
The research project was approved by the Ethics Committee of the Universidade Estadual de Campinas (Report n o 079/2007) and the interviewees signed consent forms.In the Vigitel research, the interviewees' verbal consent to participate was given at the time telephone contact was made.The establishment of the Vigitel project was approved by the Ministry of Health Ethical Research Committee (CONEP 13081/2008).
The socio-demographic characteristics obtained by the ISACamp and Vigitel surveys are shown in Table 1.
The percentages for gender, age group and education referred to those of the population of Campinas for 2007 and 2000, based on projections by the SEADE Foundation and the Census, respectively.There were no differences in overall estimates and for the subgroups evaluated for being overweight or smoking (Table 2).The fi gures in italic show differences in the indicators analyzed There were no differences observed in self-rated health for women, those aged 18 to 39 and those who had higher levels of education.For the other categories, the estimates obtained by the telephone survey were lower than those obtained by the household survey (p < 0.05).Considering the other categories, the estimates obtained by the telephone survey were lower than the obtained by the household survey (p < 0.05).
Higher prevalence of having health insurance was observed from the Vigitel data, except among the elderly and for those with higher levels of education.
The overall estimate of health plan coverage among individuals was 25% higher among those surveyed by telephone (Table 3).
The coverage for mammography was 80.1% (95%CI 75.9;84.4) in the ISACamp survey and was almost 12% higher in the Vigitel survey, irrespective of age or education (Table 4).Differences were observed in age groups and for women with lower levels of education.
There was no statistical signifi cance with regards to having had the examination in the preceding 12 months.
There were no differences in overall estimates for ever having had a pap smear test.The prevalence obtained in the Vigitel survey was 8% higher for women with a higher level of education, irrespective of age.The overall estimate reached by the telephone survey for having had the examination in the preceding 12 months was around 10% higher than that of the household survey.The size of this difference when evaluated according to subgroup was 32% for the elderly and 18% for those who reported lower levels of education (Table 4).

DISCUSSION
This study compares estimates of health indicators commonly investigated health indicators, obtained from household (ISACamp) and telephone (Vigitel) interviews.The surveys showed similar overall results of four of the eight items assessed: being overweight, smoking, having had a smear test at least once and having had a mammogram in the preceding 12 months.For the estimates of worse self-rated health, having health insurance, having had a mammogram at least once and a smear test in the preceding 12 months, it was possible to verify differences of a variety of sizes, with signifi cantly higher estimates from the Vigitel survey, except in the case of worse self-rated health.
The validity of self-reported weight and height showed a high correlation between measured and reported data.In spite of the tendency to underestimate weight and exaggerate height. 9,17In this study, similarities were observed between overall estimates and sociodemographic variables for being overweight.Nelson et al 16 (2003) using data from the National Health Interview Survey (NHIS) and the (Behavioral Risk Factor Surveillance System) (BRFSS) compared estimates for weight, height and body mass index (BMI) and observed signifi cant differences in overall BMI estimates for the population aged ≥ 18 and according to specifi c groups, with greater prevalence obtained from the household interview (NHIS) data.Upon comparing estimates of BMI with data from the 20004 BRFSS, the NHIS and the National Health and Nutrition Examination Survey (NHANES), no differences were found in overall estimates between the BRFSS and the NHIS, although the rates were higher for the NHANES data. 7However, as in the study by Nelson et al 16 (2003), cut off points specifi c to BMI were not used.
Epidemiological studies have identifi ed new locations for cancer since the fi rst link between smoking and lung cancer was established, with a causal link to tobacco. 19olicies aiming to reduce and prevent tobacco consumption have contributed to the fall in smoking in Brazil and reduced the burden which smoking related illness represents for the health system. 23For Wunschet al 23 (2010), knowing the extent of the problem in different subgroups of the population is fundamental for adopting and monitoring effective campaigns to control smoking.
No statistically signifi cant differences were observed when comparing the percentage of adult smokers, which implies that obtaining estimates for this indicator via telephone interviews is viable.Overall estimates of the percentages of smokers presented similar values to data from the 2004 BRFS and the NHIS.Upon comparing NHANES with BRFSS, the household survey showed signifi cantly higher overall prevalence than those obtained by the telephone survey. 7Arday et al 1 (1997) used data from the 1985, 1989 and 1992/1993 BRFSS and from the Current Population Survey (CPS), a survey conducted in every American state by the Census Bureau, in order to compare the prevalence of smoking and they observed similarities.
Population based studies including self-rated health as an indicator enables the researcher to know about health conditions as perceived by the individual.In the last few decades, longitudinal studies have shown the ability to predict mortality in adults and functional decline in the elderly.For Macintyre et al 14 (2005), this is a relevant indicator, even at ages when mortality rates are low.
In this study, it was possible to verify higher proportions of worse self-rated health in the ISACamp, for overall estimates and also according to subgroup, except for women, the 18 to 39 age group and among those who reported higher levels of education.Upon comparing these estimates with data from the NHIS and the BRFSS, Nelson et al 16 (2003) observed differences in overall and specifi c estimates for the sections evaluated among those aged 18 to 64 who considered their health to be bad or very bad.However, in contrast to this study, the estimates were higher for the telephone study.
Having health insurance in Brazil is an indicator which enables the assessment of cover by the supplementary health system.The study by Lima-Costa 13 (2004) indicates greater use of preventative services among individuals who have private health insurance, even when the results are stratified by education.This suggests social inequalities with regards to access to health care services.
In this study, except for the elderly and those with higher levels of education, the Vigitel survey showed higher percentages of having health insurance.Bernal & Silva 4 (2009) showed that having at least one health insurance plan was positively associated to access to a residential landline.Data on the population with a landline are biased towards coverage by a health insurance plan. 8,11himi et al 7 (2008) found similarities when comparing overall estimates of lack of health insurance in the US, using data from the 2004 BRFSS and NHIS.Similarities in the overall estimates of these surveys with the 1997 data were observed by Nelson et al 16 (2003).
In spite of recognized methods of prevention and early detection in Brazil, estimates for 2012, valid for 2013, show 52,680 and 17,540 cases of breast cancer and cervical cancer respectively.d According to Viacava et al 22 (2009), estimates of coverage for mammograms obtained from household surveys are used in various countries to monitor campaigns for early detection of breast cancer and to evaluate the effects of increasing coverage in reducing mortality and increasing survival rates.Estimates of mammogram and smear test coverage in Brazil are obtained from data from various population surveys. 6,8,12,21,22 statistically signifi cant differences were found for mammogram coverage in the preceding 12 months, but the estimates from the Vigitel survey were higher for having had the examination at least once, as was the case in the study using data from household and telephone interviews in São Paulo, Southeastern Brazil. 20avis et al 5 (2010) observed higher estimates for the phone surveys in the periods 1997-1999 and 2000-2003 based on NHIS and BRFSS data.Viacava et al 22 (2009) showed signifi cant differences in mammogram coverage in the three years preceding the interview using data from household and telephone surveys for all of the state capitals analyzed, with greater coverage obtained by the Vigitel survey.Studies show that data referring only to the population who have a landline may have signifi cant biases for having had a mammogram, 11,20,21 especially in municipalities with low numbers of landlines. 22e estimates differ for having had a smear test in the preceding 12 months, with coverage around 10% higher among those interviewed by telephone.Data from São Paulo for 2008 show higher coverage for telephone surveys both for having had a smear test at least once (4%) and in the preceding 12 months (14%). 21udies on comparing estimates from different methodologies for collecting data in population surveys are documented in the international literature. 1,5,7,16The increasing number of surveys carried out in Brazil over the last few decades require studies on: methodological questions involving sample design; selection procedures; techniques for assessing data quality and validity; post-stratifi cation strategies for minimizing the no-response effects; evaluating the effects of applying different measuring tools; and comparing estimates obtained from different types of survey.However, national experience is of recent date. 8,10,21,22e ISACamp and Vigitel surveys are population based with different sample designs and different methods of collecting data.However, comparison was possible as they had many similarities: they were carried out in 2008, they did not use proxy data and the questions used for some of the indicators were analogous.
Analyses indicated that part of the overall results were similar and, for those which presented significant differences, a tendency for the telephone survey data to overestimate was noted, except in the case of worse self-reported health.
Interpreting the results of those indicators which showed differences between the two surveys required detailed analysis of its epidemiological signifi cance.In spite of overestimating prevalence, the estimates obtained by the Vigitel survey overall, converged in the same direction as those found by the household survey.
This survey compared estimates of indicators investigated using different types of surveys and enabled the statistical differences in some variables, whose post-stratifi cation adjustments are still not suffi cient to cancel out the defects associated with those without a telephone being excluded, to be viewed in perspective.Moreover, it demonstrated that the estimates obtained by the Vigitel survey show consistency, which has also been observed in other studies. 8,10,21Although there are signifi cant differences, the consequences related to using such estimates in implementing public health campaigns and programs may be considered of minor importance.
Studies which compare estimates of these and other indicators for both types of surveys are necessary to confi rm the fi ndings and validate data relevant to public health.These data can be made available rapidly and systematically through a dynamic and effi cient system of collection such as the telephone survey (Vigitel).
The Vigitel (Telephone Survey System for Monitoring Risk and Protection Factors for Chronic Illness) is population based research established by the Ministry of Health in 2006.This research aims to continuously monitor the frequency and distribution of risk and protection factors for chronic disease in the Brazilian state capitals and in the federal district.a In 2008, the municipality of Campinas was included in this research.INTRODUCTION a Ministério da Saúde.Secretaria de Vigilância em Saúde.Secretaria de Gestão Estratégia e Participativa.Vigitel Brasil 2008: vigilância de fatores de risco e proteção para doenças crônicas por inquérito telefônico.Brasília (DF); 2009.b Alves MCGP.ISA-CAMPINAS 2008/09.Plano de Amostragem.Campinas: UNICAMP; 2009 [cited 2012 Jan 5].Available from: http://www.fcm.unicamp.br/centros/ccas/arquivos/plano_de_amostragem.pdf
n: number of individuals in the unweighted sample ISACamp: Health Survey Campinas; Vigitel: Monitoring system for Risk and Protective Factors for Chronic Diseases Telephone Survey a Prevalence in the weighted sample

Table 2 .
Comparison of estimates for being overweight and percentage of smokers, by sex, age and education.ISACamp and Vigitel.Campinas, Southeastern Brazil, 2008.Adjusted for variables which appear in the Table ISACamp: Health Survey Campinas; Vigitel: Monitoring system for Risk and Protective Factors for Chronic Diseases Telephone Survey a

Table 3 .
Comparison of estimates for worse self-rated health and having health insurance, according to sex, age and schooling.ISACamp and Vigitel.Campinas, Southeastern Brazil, 2008.
a Adjusted for the variables which appear in the Table ISACamp: Health Survey Campinas; Vigitel: Monitoring system for Risk and Protective Factors for Chronic Diseases Telephone Survey The fi gures in italic show differences in the indicators analyzed

Table 4 .
Comparison between estimates for coverage of mammograms and smear tests at least once and in the preceding 12 months.ISACamp and Vigitel.Campinas, Southeastern Brazil, 2008.