Agreement between data obtained from repeated interviews with a six-years interval

The objective of the study was to compare information collected through face-to-face interviews at fi rst time and six years later in a city of Southeastern Brazil. In 1998, 32 mothers (N=32) of children aged 20 to 30 months answered a face-to-face interview with structured questions regarding their children’s brushing habits. Six years later this same interview was repeated with the same mothers. Both interviews were compared for overall agreement, kappa and weighted kappa. Overall agreement between both interviews varied from 41 to 96%. Kappa values ranged from 0.00 to 0.65 (very bad to good) without any signifi cant differences. The results showed lack of agreement when the same interview is conducted six years later, showing that the recall bias can be a methodological problem of interviews. DESCRIPTORS: Interviews. Data Collection. Reproducibility of Results. Bias (Epidemiology).

Minimizing bias in order to produce more valid results is a major challenge to survey-based research.A number of research methodologies are employed to investigate oral behavior and other health habits through face-toface and telephone interviews, mailed questionnaires, diary data and computer-based questionnaires.Such methodologies also attempt to evaluate the validity and reproducibility of these methods. 1,3udies have investigated the reproducibility of research methodologies, comparing computed-based questionnaires to printed ones and face-to-face interviews, mailed questionnaires to telephone interviews. 1,4owever, it is also important to evaluate the agreement of responses when the same method is used with the same population at two different points in time.Do respondents answer the same question in the same way as earlier?It is important to evaluate the precision of responses, especially if the investigation is about events that occurred years earlier.Agreement can be measured by kappa coeffi cient for two observers or between two classifi cations on ordinal or nominal scales. 5e aim of the present study was to compare information collected through face-to-face interviews at fi rst time and six years later.

METHODS
The study was conducted as part of a cohort study on fl uoride intake among 32 children aged 20-30 months in Southeastern Brazil.In 1998, mothers answered a faceto-face interview about their children's current brushing habits with fl uoridated toothpaste.The interview was conducted by a single researcher (SMP).
All 32 mothers interviewed in 1998 agreed to participate in the second interview, conducted six years later (2004), by a second interviewer (CCM) previously trained by the fi rst one.On this second occasion, mothers were asked to recall the past habits of their children when they were 20-30 months of age.Both interviews lasted on average ten minutes.
Both interviews were conducted at the mother's home at a prescheduled time.They followed a protocol of structured questions on the child's brushing habits with fl uoridated toothpaste.The data analyzed comprised six identical questions included in both interviews, from two to four answer options.
Data were entered into Stata and electronic spreadsheet.Interviews were compared for overall agreement (a+d/ a+b+c+d), kappa and weighted kappa, with p<0.05 considered signifi cant at 95% confi dence interval.Strength of agreement was based on criteria defi ned by Altman 2 (1991): 0.00 to 0.20= poor; 0.21 to 0.40= fair; 0.41 to 0.60= moderate; 0.61 to 0.80= good; 0.81 INTRODUCTION to 1.00= very good.The frequency of missing data was compared between the two interviews."I don't know" and "I don't remember" responses were considered missing data and thus were not included for kappa calculation and overall agreement.These responses are not presented in the Table .The study was approved by the Universidade Federal de Minas Gerais Research Ethics Committee.
For the questions "How often did your child brush his teeth per day?" and "How much toothpaste was dispensed on the brush?," weighted kappa values were 0.23 and 0.12, respectively.
The missing data was low, but higher for the interview conducted in 1998 (5.7%) than that conducted in 2004 (0%).

DISCUSSION
The outcome addresses the comparison of responses and their implication in the ability to recall past episodes.
The response rate was 100%, which is not anticipated in longitudinal research since a proportion of subjects is often lost over a study period.Nevertheless, all mothers were located and agreed to participate, which is noteworthy.
The missing data in the present study related to "I don't know" and "I don't remember" answers, which accounted for 11 out of a total of 192 answers.These responses were not included in the kappa calculation and overall agreement.The frequency of missing data was relatively low, but it can be assumed that the small study sample infl uenced this fi nding.Missing data were higher for the interview conducted in 1998 (5.7%) than that conducted in 2004 (0%).The lower frequency of missing data in 2004 may be due to information on oral health gathered over the six-year period, in which mothers might not remember their children's habits.On the other hand, the very low frequency of missing data in 2004 does not imply reliable answers.In a study by Aitken et al 1 (2004), they found 0.5% of missing data in telephone interviews and 3.8% in mailed questionnaires, which is very similar to the present study.
All questions had an overall agreement varying between 41% to 96%, with kappa values ranging from 0.00 to 0.65 (poor to good). 2 Unlike kappa, overall agreement does not consider random agreement (Table ), but it is used to evaluate agreement between two measures of the same individual. 5The present study evaluated the agreement of an interview conducted with the same group of mothers over time.Kappa values ranging from poor to good evidenced low reproducibility of responses over time.As evidenced by the p-value, there was no signifi cant difference, except for the questions "Who dispensed toothpaste on the brush?" and "What kind of toothpaste did your child use?" (k= 0.33, 95% CI: 0.00-0.73,p=0.016).The former had a higher kappa value (0.65, 95% CI: 0.01-1.00,p=0.001).Only one mother changed her answer between the two interviews.The adequate agreement indicated by the kappa value is perhaps due to the fact that this information is already widespread among the population, along with young children's inability to dispense toothpaste on the brush by themselves.
In regard to the question "What kind of toothpaste did your child use?," 11 mothers who responded "the regular one" in 1998 changed their answer to "children's toothpaste" in 2004.Several factors may have infl uenced this change, such as new brands introduced in the market, marketing strategies, new siblings born during the recall time who used children's toothpaste.The child might have changed habits during the six-year period, which may have confused their mothers.The following studies compared two methodological instruments and found considerable agreement.Chestnutt et al 4 (2004) found good to very good agreement when comparing a face-to-face interview to a computed-assisted questionnaire (k=0.68 to 0.90).Berthelsen et al 3 (2000) found an overall reliability of 93% when comparing a computed-assisted questionnaire to a printed questionnaire.Both studies comprised questions on oral practices.The present study is different from these two studies as it did not compare two different methods, but rather the same method over time.
Depending on the type of question and sample size, kappa values may increase or decrease.Many mothers changed their answers in the second interview.These changes may be explained by several reasons: mothers may have received new information on oral health and adopted new habits; some questions may be very subjective and confused the parents with regard to recalling the amount of toothpaste or brushing frequency; mothers may have over-reported some habits to apparently show they are careful; individuals may choose to respond positively to the questions to please the interviewer or adapt their answers to the socially desirable norm; and recall bias, i.e., the faulty memory of mothers in recalling the child's dental practices.
The lack of agreement between both interviews could be explained by the type of information addressed.Brushing habits may not represent important information to mothers, thereby explaining inadequate recall.The six-year period between interviews may be an extensive period of time, but studies regarding childhood habits usually ask parents to recall episodes of their children's life from long time ago.This is the reason why the present study conducted the second interview many years later.Furthermore, a small sample may affect the outcome, whereas kappa values would likely have been different if the study were conducted on a larger sample.
The lack of agreement may be due to recall bias in parent's accounts, which represents a great challenge to epidemiologic research.Many epidemiologic investigations are based on data collected though interviews and questionnaires regarding past events.The results of the present study revealed a lack of agreement when the same interview is conducted after some time.Investigators should be aware of this problem and follow proper training before conducting interviews.Caution should be taken in order not to infl uence the respondents and avoid bias when interpreting the results.

Table .
Comparison of mothers' responses in interviews at different times regarding their child's brushing habits.SoutheasternBrazil, 1998 and 2004.