Reliability of a questionnaire about knowledge concerning exposure to mercury in the production of gold

The primary economic activity of the municipality of Poconé, Central-West Brazil, is the production of gold. In an attempt to reduce exposure to metallic mercury, schools in that location conduct educative programs directed towards their students. A questionnaire used to evaluate the effi cacy of these programs with respect to adolescents’ comprehension concerning exposure to mercury was developed in 2007. This questionnaire is self-applied and has 12 questions. A test-retest reliability study, with a 21-day interval between the two sessions, was performed with this instrument. The sample was comprised of 128 students who were in the 8th grade of elementary school. Reliability was evaluated by Kappa coeffi cient and McNemar’s chi-square test, resulting from “moderate” to “almost perfect” agreement. DESCRIPTORS: Adolescent. Mercury Poisoning. Occupational Exposure. Questionnaires. Reproducibility of Results. Health Education.


INTRODUCTION
Metallic mercury presents elevated toxicity 2 and is utilized in the production of gold to form a gold-mercury amalgam which, after being burned, liberates the mercury into the atmosphere.When it is commercialized, this gold is burned again for "purifi cation", liberating more mercury into the atmosphere. 4Metallic mercury may also be transformed into methylmercury in the river sediments by means of a process of methylization, mediated by aquatic microorganisms. 5study conducted in Poconé (Central-West Brazil) indicated that the the mean levels of mercury in the urine (HgU) of 365 residents not exposed occupationally, that is, residents who lived near the stores that commercialize gold, were four times higher than the control group (4.35μg/L versus 1.25μg/L).Among the latter, 14 people (HgU from 7.0 to 102.4 μg/L) reported that they extracted gold from their backyards and burned the amalgam in their kitchens. 1e objective of this study was to test the reliability of a questionnaire utilized to evaluate the effi cacy of preventive programs in terms of adolescents' knowledge concerning exposure to mercury.

METHODS
This study was conducted in 2007 among 8th grade elementary school students enrolled in public schools in Poconé.The size of the sample of the test-retest reliability study was calculated as 123 pairs, considering that the prevalence of exposure to mercury among students, according to a previous study was 4.5%, 1 with 3% absolute precision, 1% effect of the design and 95% confi dence level.Seven schools were randomly selected and when the instrument was applied, a code system was utilized to guarantee the anonymity of the study subjects.
Within this context, a test-retest reliability study was conducted by means of a self-applied questionnaire that was utilized to evaluate the effi cacy of a program of the Instituto de Estudos em Saúde Coletiva da Universidade Federal do Rio de Janeiro [Institute of Collective Health Studies of the Federal University of Rio de Janeiro] geared towards the prevention of exposure to mercury among adolescent residents living within areas of gold production.
The questionnaire addresses knowledge acquired by adolescents through an educational program composed of ludic activities concerning accidents and diseases related to the environment in general and specifi cally with the environment at work.In this study, during the period between the test and the retest, the program was not developed so that it could not interfere in the evaluation of the reliability of the questions.
The questionnaire has three open questions: 1) What is pollution?2) State some causes of pollution?; 3) What is an occupational accident?Two researchers analyzed if the answers were correct, and, specifi cally with respect to occupational accidents, some plausible answer which corresponded to adolescents' knowledge concerning this theme, that is, a response which included work and some effect it had on health (for example getting hurt on the job) were taken into consideration.The remaining questions were closed (the answers were either yes or no): 4) Do you work outside your home? 5) Do you work at home? 6) Manuel was going to work on his bicycle and he hit a car.Because of this accident, he broke his arm.In your opinion, did Manuel suffer from an occupational accident?7) Yesterday it rained a lot while John was at his job on the street fi xing a post.Upon his arrival at home, his clothes were soaked and he got a lung disease.Do you believe that John's disease is an occupational accident?8) Richard and Bruno fought at the bar where they work because of a soccer match.Bruno punched Richard and, as a result, broke two of his teeth.In your opinion did Richard suffer from an occupational accident?9) Have you ever gotten hurt because of your work?
10) The use of chemical products at home may be harmful to your family's health?11) Has someone in your home gotten sick due to the use of some chemical product?12) Is pollution a problem in your city?
The evaluation of test-retest reliability was conducted by means of the Kappa coeffi cient, which consists of calculating the proportion of matching replies in two applications of the same instrument. 3The values were classifi ed in the following manner: values between 0.81-1.00(almost perfect); 0.61-0.80(substantial); 0.41-0.60(moderate); 0.21-0.40(weak); 0.00-0.20 (slight) and, <0 poor.McNemar

RESULTS
The questionnaire was fi lled out by 207 adolescents, 184 of which participated in the retest.After matching the questionnaires in test-retest pairs, analysis focused on 128 pairs and the 60 remaining questionnaires were considered losses.There was a similar proportion of respondents of each sex (approximately 50%); the mean age for the female sex was 13.9 (sd=0.10)and 16.0 (sd=1.42)for males; according to registration forms, the majority (75%) of students attended school in the morning.Absence from school was the primary cause of losses, being excluded from analysis those that participated only in the test or the retest.It may be presumed that the students who were absent did not differ from the rest of the students, for they were all in the same socioeconomic level that characterizes students attending public schools.
As to the items concerning the working environment, (see Table ), it may be observed that the two questions about work "outside" and "inside" the home had almost perfect kappa (respectively: k=0.81, 95% CI: 0.70; 0.93; k=0.91, 95% CI: 0.84; 0.99).For the remaining questions concerning these occupational environments, the values obtained corresponded to "moderate" to "substantial" reliability.
The questions which referred to chemical products and pollution obtained kappa values which correspond to "substantial" and "moderate" respectively, being that the latter was attributed to questions about whether someone in the house had suffered some effect caused by pollution (k=0.571;95% CI: 0.33; 0.81) and the existence of pollution in the city of Poconé (k=0.551;95% CI: 0.37; 0.73).
General agreement varied from moderate to high for all items of the questionnaire, with a value of 83.5% (95% CI: 75.8; 89.5) when characterizing disease as an occupational accident to 99.2% (95% CI: 95.6; 99.9) when defi ning environmental pollution.Except for the question "what is an occupational accident?"(p=0.025),no other question presented signifi cant statistical difference for the marginal proportions (test-retest) according to McNemar's χ 2 test.

DISCUSSION
Reliability and reproducibility, measured through internal consistence, were evaluated using kappa coeffi cient, were evaluated utilizing the weighted kappa coeffi cient of all items.The reliability obtained varied from "substantial" to "almost perfect" for half the items and the remaining items presented "moderate" kappa coeffi cients.The 21-day interval between the two sessions (test-retest) was longer than what is recommended by other studies conducted in Brazil 3 .It may therefore be inferred that if the period between the applications of the questionnaire was smaller, memory biases may have been reduced and, above all, there may have been less probability of changes occurring in the events being measured.
As to the three open questions, it is possible that the concepts concerning occupational accidents and environmental pollution are already part of these students' symbolic repertoire, since they presented "almost perfect" kappa coeffi cients, despite the small difference in marginal proportions.A plausible explanation for the high value obtained by the concept of environmental pollution may be the diffusion of the results of studies conducted by different institutions that have pointed out the association between use of mercury in the production of gold and the increased exposure of the population to this metal. 1However, reliability was "moderate" when students were asked to point out the causes of pollution.Knowledge concerning the effects of this pollution on health have not yet been incorporated by participants in the present study because 80.5% were not aware of the fact that the use of chemical products in the house could harm family members.
As to the work setting, "almost perfect" reliability was obtained in the identifi cation of the place of work being "inside" or "outside" the home.This result is relevant, because it indicates that adolescents have incorporated basic concepts concerning work.
The items that equated diseases, aggressions at the work place and accidents on the way to work as occupational accidents, the question on accidents on the way to the work place presented a 0.793 coeffi cient whereas the other two questions presented rates of 0.591 and 0.558.
These are more complex questions that extrapolate the cognitive knowledge of adolescents that are still in the eighth grade.If, on the one hand, the existence of only two alternatives (yes and no) may have induced the student to choose one of these options randomly, on the other, it may be observed (see Table ) that for each of these three questions in the questionnaire, respectively 106, 107 and 122 the students maintained the options they chose in both the test and the retest, indicating that these questions were adequately formulated.
As to the other questions in which the kappa was moderate, a high level of agreement was obtained: injuries caused by occupation, the existence of pollution in the city, the use of chemical products at home and the effects of chemical products on the health of someone at home.
Finally, among the limitations of the study, we must mention the losses that occurred due to the number of students that replied only once because he or she were absent from class on the other occasion.Although the instrument presented some questions that obtained moderate kappa values, it expressed adequate reliability.It may therefore be employed in the evaluation of educational programs geared towards adolescents exposed to mercury, in areas of gold production.
's χ 2 test was utilized to verify the equivalence between the proportions of positive responses in the test and retest.Statistical analysis was conducted utilizing the 9.0 version of the SPSS Program.A favorable report was obtained from the Research Ethics Committee of the Instituto de Estudos em Saúde Coletiva [Institute of Studies in Collective Health] of the Federal University of Rio de Janeiro before the study instrument was applied and all research subjects and their guardians signed Terms of Informed Consent before answering the questionnaire.

Table .
Test-retest reliability of questions related to the work environment among adolescent students.Poconé, Central-west, 2007.