Evaluation of informative materials on leishmaniasis distributed in Brazil : criteria and basis for the production and improvement of health education materials

Based on categories related to structure, content, language, and illustrations, the present study provides an evaluation of the quality of educational materials on leishmaniasis available to health services in Brazil. The 18 publications evaluated consisted of four handbooks, four guided studies, four booklets, and six leaflets. Of the total publications assessed, nine were produced by the Brazilian National Health Foundation (FUNASA), five by State and Municipal Health Departments jointly with FUNASA, and one by the Pan-American Health Organization. The evaluations were also performed by three professionals: a physician specialized in leishmaniasis, a parasitologist, and an information/communications expert. The publications failed to specify key items such as target public, objective, and bibliography. The illustrations, especially in the booklets and leaflets, failed to clarify the text, portrayed biased concepts, and omitted credits and scale. According to this study, informative materials on leishmaniasis distributed in Brazil present major limitations which jeopardize the quality of information they contain.


Introduction
Leishmaniasis is a serious public health problem in Brazil.American tegumentary leishmaniasis (ATL) and American visceral leishmaniasis (AVL) are diagnosed in almost all regions of the country (Waldman et al., 1999).Over the last ten years (1990)(1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000), despite control measures, there has been a sharp increase in the number of cases reported in urban areas.The main factors involved in the change in the epidemiological profile of the disease are probably the ecological characteristics in each region, the virulence of Leishmania parasites, and vector domiciliation (Oumeish, 1999).In addition, migration of non-immune individuals from rural areas to poor urban suburbs has been a key contributing factor to the expansion of AVL in cities like Natal, Fortaleza, João Pessoa, São Luis, and Salvador (Desjeux, 2001).In Greater Metropolitan Belo Horizonte, with 4,293,433 inhabitants, 95%-100% of whom living in the urban area (IBGE, 2002), autochthonous cases of ATL have been reported in 32 (89%) of the municipalities (Luz et al., 2001).The number of AVL cases has increased steadily since the first reported case in 1989, to 94 cases in the year 2000.In addition, AVL case fatality in Greater Metropolitan Belo Horizonte is high; some 10% of fatal cases can be associated with the lack of qualified health professionals for early diagnosis and prompt treatment.In 19 (61.3%) of the municipalities in Greater Metropolitan Belo Horizonte, no referral services were available for diagnosis of the infection.Twelve (39%) of the municipalities had health services for leishmaniasis evaluation, but only eight (67%) had basic specific diagnostic tests available (Luz et al., 2001).Measures to improve health services and training of health professionals are therefore essential.Moreover, it is important to have high-quality informative materials available, including handbooks, booklets, leaflets, and posters, which are potentially valuable tools that can contribute to routine health services and control programs with community participation.Nevertheless, in order to be effective, such materials should be preceded by scientific investigation and produced within well-defined criteria in order to foster knowledge and cognitive potential within the reality of the target population.The use of illustrations does not necessarily enhance the materials' readability or ensure understanding of their message (Lefèvre, 1981).In addition, informative materials on diseases in Brazil can tend to copy each other, reproducing the same errors for decades, as shown by one group of authors in the case of schistosomiasis (Schall & Diniz, 2001).Although educational materials are frequently produced and used in Brazil as part of disease control programs, little is known about the efficacy of such information.Recently, with evidence of serious errors and inadequacies in several such publications, some public institutions and researchers have begun to evaluate and require improvements in them.An example is the Ministry of Education and Sports, which is taking steps to evaluate the publications listed in the National Textbooks Plan (PNLD, 2002), identifying parameters to assess the quality of the textbooks from the first through the eighth grades.Criteria were set to potentially evaluate quality in terms of correct content and pertinence of the text, in addition to identifying and eliminating biased concepts, using general parameters combined with specific ones depending on the area of knowledge.
In the present study we also develop parameters to perform an initial analysis of the quality of informative materials on leishmaniasis currently used in Brazil.The study thus adopts an exploratory design combining quantitative and qualitative approaches to assess educational materials on leishmaniasis available to health services in the country as an effort to create standards to achieve greater efficiency in this kind of information.

Material and methods
Informative materials on leishmaniasis were requested by letter from the regional offices of the Brazilian National Health Foundation (FU-NASA) and State and Municipal Health Departments.Additionally, all existing materials were consulted at the library of FUNASA and the Brazilian Health Ministry.The material was grouped in the following categories: booklets, leaflets, guided studies, and handbooks.The research was conducted in two phases.In the first phase we established four sets of parameters, structure, content, language, and illustrations, based on the general criteria previously described by the Brazilian Ministry of Education and Sports (PNLD, 2002).An open-ended questionnaire was prepared (Table 1), and evaluation of all the materials was performed using the questionnaire guide.We observed the following for each parameter:  (Ferreira, 1986:917).The second phase focused on an evaluation by three professionals: a physician specialized in leishmaniasis, a parasitologist specialized in vector and parasite research, and an expert in information and communications.The opinions by the physician and parasitologist were considered the "gold standard" on the disease.We selected an expert in information and communications to assess the material as to language, forms of communication, clearness, layout, and presence/absence of biased ideas.
The samples (n = 18) were evaluated individually according to the four sets of parameters: structure, content, language, and illustrations.The evaluators were also asked to submit extra comments if they felt necessary.The title, authorship, and origin of the materials are listed in Table 2.

Quantitative analysis: overall characteristics
The 18 publications (Table 3) were classified as four handbooks, four guided studies, four booklets, and six leaflets.Of the total publications assessed, nine (50%) were produced by FUNASA, five (28%) by State or Municipal Health Departments, three (17%) by Health Departments jointly with FUNASA, and one (5%) by the Pan-American Health Organization (PAHO).Dates of publication were recorded in the handbooks and guided studies and ranged from 1990 to 1999.Table 3 shows the frequency of the target public, bibliography, scale, and credit for illustrations in the materials evaluated.The target public was specified in one handbook (25%) and three guided studies (75%).However, this information was lacking in the booklets and leaflets.Mean number of pages was 36 in handbooks and 68 in guided studies.The leaflets contained a single page, and the booklets ranged from five to 14 pages.Bibliographies were included in two out of four of the handbooks and guided studies (50%) and none of the booklets or leaflets.Illustrations were drawn to scale in one handbook and one guided study (25%), as well as in two booklets (50%).However, none of the leaflets featured drawings to scale.Credits for illustrations were found in one booklet.
Table 1 Questionnaire for evaluation of informative materials on leishmaniasis.

Qualitative assessment based on defined categories
A description of the analysis will be presented below by category of material.Table 4 shows a summary of the assessments made by the evaluators:

• Handbooks
The size of the text and quality of printing were considered appropriate and sufficient in quality in 100% of the handbooks according to the parasitologist and information/communications expert, as compared to 75% according to the physician.The information was generally adequate in this kind of material, despite some grammatical errors.Although most of the handbooks failed to specify the target public, the most important disease concepts were presented in a clear and objective way according to the physician and the parasitologist.Interestingly, however, the information/communications experts, who is not an expert on leishmaniasis, found two handbooks difficult to understand because of a poorly structured text in one and unclear wording in the other.According to the physician, important aspects such as side effects of pentavalent antimonials, the first line of treatment for leishmaniasis, were not approached with the necessary emphasis.Three handbooks also placed too much emphasis on the disease "vector" theme as com-pared to diagnosis and treatment.In general the illustrations were considered good, although one handbook had few, out-of-context illustrations.Some illustrations lacked captions and/or headings.One issue shows a male sandfly to illustrate the vector, although only females act as vectors.None of the handbooks included a glossary of technical terms.

• Guided studies
The size of the text and printing quality were considered appropriate and good in 100% of the samples according to the parasitologist and the information/communications expert and in 75% according to the physician.The author's names were listed in 100% of these publications, and the target public was defined in three of the four guided studies.In general, the information was considered appropriate for the target public, although important concepts, especially pertaining to diagnosis and treatment, were not approached properly.For example, in two guided studies, which were otherwise well-designed according to one of the specialists, there was two much poetry and lyrics and too little information on the disease itself.Only one of the four guided studies had adequate bibliographic references, in two the references were incomplete or outdated, and one had no references at all.None of the guided studies included a technical glossary.

• Booklets
Although the booklets failed to specify the target public, the language was considered to be aimed at schoolchildren.The size of the text was considered appropriate in 75% of the samples and quality of printing was good in 50%.
Depending on which expert evaluated the materials, the information was considered adequate in one (25%) to three (75%) of the booklets.The language was also considered unclear, difficult to understand, and grammatically incorrect.The illustrations included biased concepts such as diseased dogs portrayed as werewolves, sandflies as blood dealers, and the disease leishmaniasis as inherited from Africa.The main incorrect concept above is that the vector is cited as a mosquito rather than a sandfly.Contrary to information provided incorrectly in some of the booklets, only female sandflies can act as vectors, since they require a blood meal for egg development (WHO, 1990).In addition, the vector does not lodge on the reservoir.In general, it only remains on the reservoir long enough to feed.The transmutation of humans into werewolves is an incorrect and inappropriate metaphor (Figure 1).Such devices vulgarize the information and fail to enhance and disseminate correct and appropriate knowledge.b) Concerning the disease: "... another type of kala azar, the so-called tegumentary form, causes wounds in the mouth and destroys the nose, deforming the individual" (in "Beware of the Dog: Kala Azar, a Beastly Disease!" Figure 2 shows an incorrect illustration of the tegumentary clinical manifestation of the disease. With regard to the disease, the tegumentary form is not another type of kala azar.The two are different presentations of Leishmania spp.infection, and a small proportion (3-5%) of patients with the tegumentary form develops the mucosal disease, affecting the nasal, oral, and pharyngeal membranes (WHO, 1998).
Furthermore, several of the publications portray the readers as the ones to blame for the disease, as shown in the following example: "Just stop and think.If you don't tear down the forest or change the habits of the cangalhinha mosquito, and it you keep your dog healthy and vaccinated against rabies, kala azar and rabies won't get you.Why?", in the booklet Seu Cachorrinho, Amigo ou Inimigo?["Your Dog, Friend or Foe?"].

• Leaflets
The six leaflets failed to mention authors, target public, or bibliography.One leaflet considered the dog "the main villain" in the story, which all the evaluators considered a biased notion.The concepts were considered clear and objective in three of the six leaflets.One of the six lacked any illustrations, and in the other five the illustrations were considered good and pertinent in three and unsuitable in two.The size of the text was considered appropriate in 66% of the samples, and printing quality was considered good in all six.

Discussion
Health education activities have been considered an essential tool for any disease control program (WHO, 1990).Critical studies in this area of health are generally justified and necessary, especially since most educational materials are distributed by public health institutions.In Brazil, although educational materials are included in disease control programs, little is known about the quality or efficiency of the information they contain.According to a prospective study in Maranhão, a State in Northeast Brazil with the country's second largest AVL prevalence (FUNASA, 2002), knowledge was scanty concerning AVL among the rural and peripheral urban populations (Gama et al., 1998).In relation to ATL, knowledge was also deficient in five areas studied in the same State, especially concerning prevention and treatment (Moreira et al., 2002).For an educational message to be understood, it must meet minimum requirements, including the use of adequate vocabulary.It should also consist of comprehensible, well-constructed, and well-worded sentences.Appropriate language is essential to orient the influence that the universe of beliefs and values exerts on the relationship between information and behavior in any disease.Evaluation of the effectiveness of community mobilization for schistosomiasis control has shown that information provided to informants is often reinter-preted in light of existing beliefs (Uchoa et al., 2000).This paper does not analyze what readers actually understood, but what was offered to them for their comprehension.Participation by a panel of specialists including a physician, a parasitologist, and an expert in information/ communications with different backgrounds and distinct points of view on the subject provided richer and more complete assessments of the materials.Considering that the primary purpose of a pedagogical message is comprehension, such messages must contain a system of signals that is as clear as possible (Lefèvre, 1981).Fawdry (1994) developed key points that essential for information leaflets aimed at patients, including clear and simple language, appropriate and adequate information that can be understood by the majority of readers, and references to additional reading for other sources of help.Nevertheless, the 18 publications analyzed in our study failed to specify important items such as the target reader public, objectives, or bibliography, besides lacking appropriate illustrations.In an overall analysis, lack of specification of the target population and or objectives of the materials hindered the assessment, because it was often difficult to determine the content's relevancy.Definition of the target public is necessary in order to adapt the wording.An accurate, up-to-date bibliography is also essential because it offers readers the opportunity to study the subject further.Our research also indicated that although almost all of the publications used were produced for the Brazilian Ministry of Health, through the FUNASA, such publications failed to comply with any kind of standard format.This shows that even the country's top official health agency lacks well-defined criteria for such publications.Illustrations, potentially important to the text to assist comprehension and enrich the reading process, must have an appropriate purpose.They should also be clear and easy to understand, to help stimulate thinking and curiosity.None of these features were found in the materials we evaluated.Rather, numerous misstatements were made in an attempt to simplify the information.Material aimed at the population at large frequently refers to the leishmaniasis vector insect as a "mosquito" rather than a sandfly.Although both are insects, they have important differences in their life cycles (for example, taking place in water for mosquitoes and in organic matter for sandflies) (Forattini, 1973).Such characteristics are essential to determine control measures.It is sometimes necessary to reduce the complexity of the information in order to reach more people.However, this cannot be done by distorting the meaning, which leads to mistakes, helps create unreal knowledge, and even commits metaphorical abuse (Mohr, 1994).Many of the illustrations failed to clarify the text.In addition, many of the illustrations lacked credits.The drawings were rarely made to scale, failing to provide a true idea of the actual size of the objects.It is widely accepted that the lack of some kind of scale can lead receivers of the message to an incorrect understanding of the information.For example, Mohr (1994) mentions the presence of drawings in didactic materials that portrayed mosquitoes and Schistosoma sp.larvae the size of rats and sardines, respectively.The same author points out that when it is impossible to provide a measurement, it may be feasible to draw such objects next to more common items, like the head of a pin.Drawings of larger objects should contain measurements or some other type of reference.Two of the booklets evaluated demonstrated concern for providing a more realistic perspective of the object by using a magnifying glass.However, this procedure was considered inappropriate because the use of an icon, in this instance a magnifying glass, without any further explanation does not assure comprehension of the object's real size.Lefèvre (1981) points out that specific icons should belong to the reader's cultural universe, and that the message must be clear as to what that icon actually represents.The same author also mentions that the introduction of images into educational messages does not ensure readability or comprehension of their meaning.Another important point concerning illustrations is that the message they transmit must be critically evaluated.By no means should they express, induce, or reinforce prejudices or stereotypes of any kind.Generally speaking, the publications evaluated contained incorrect or biased information.For example, dogs turn into werewolves when bitten by sandflies, which in turn are referred to as "blood dealers" or "bandits".These clearly demonstrate an excessive appeal to people's fear, besides containing distorted concepts such as sandflies portrayed with "human" traits (a good-versus-evil, "beware-of-the-sandfly" approach).Critical analysis and quality criteria should foster the preparation of appropriate health information materials, enhancing the nature of such publications in order to reach their goal (Schall & Diniz, 2001).The above-mentioned educational materials on leishmaniasis that are distributed in Brazil are limited by their inappropriate language and conceptual mistakes, clearly jeopardizing the information they contain.In general, the handbooks and guided studies were more adequate and informative for the target groups than the booklets and leaflets.The problems identified in the booklets probably express their authors' difficulty in properly communicating to the reader public, besides a misunderstanding of the appropriate resources to reach the target group.It is important that public health agencies pay attention to the production of health education materials, particularly because although leishmaniasis is endemic to 88 countries in the world, 90% of the mucocutaneous cases are found in Brazil, Bolivia, and Peru alone (WHO, 1998).For the visceral form, the same proportion is found in only five countries: Brazil, Bangladesh, India, Nepal, and Sudan.Informative materials must be approached as an important resource that can contribute to the construction of transformative knowledge, capable of synergism with other health measures to change this situation.

1.
Concerning the structure/format, what is your opinion of: a) Size (dimensions) of material?b) Is the size of text (number of pages) adequate?c) Quality of paper?d) Quality of illustrations? e) Does the material cite authors and collaborators?f) Does the material define the target public?2. Concerning the content: a) Is the information correct?Please identify any distorted or incorrect concepts in the material.b) Is the information appropriate to the target population?c) Is the information presented in a context that is suitable to the target population?d) Does the text include a lack of or too many definitions?e) Are bibliographic references appropriate and up-to-date in relation to date of publication?f) Are there important definitions or facts that were not approached?g) Have definitions of greater or lesser importance been given due emphasis?3. Concerning language: a) Is the language comprehensible and suitable to the target population?b) Have all important concepts been approached clearly and objectively?c) Have you noticed any biased or prejudiced ideas concerning the information in the text?If so, please indicate page and paragraph and state your opinion.

4.
Concerning illustrations (pictures, drawings, tables, maps, diagrams):a) The Portuguese-language dictionary Aurélio defines illustrations as "images and figures of various kinds used to clarify and/or arrange text in books, pamphlets, or periodicals".Do illustrations in the material follow this definition?b) Is the visual layout attractive and well-organized?If not, please explain as in item 3c.c) What is your opinion of the quality, pertinence, and number of illustrations?5.What is your final opinion after reading? 6.What suggestions would you give in order to improve the material?7. Please add any other comments you find relevant and that have not been included in this questionnaire.Cad.Saúde Pública, Rio de Janeiro, 19(2):561-569, mar-abr, 2003

Figure 1
Figure 1Dog that turns into a werewolf when bitten by sandflies, which are referred to as "blood dealers" (in: Kala Azar: A Dog's Disease).

Figure 2
Figure 2 Inadequate illustration of tegumentary clinical manifestation: mucocutaneous form on left and cutaneous form on right (in: Beware of Dog: Kala Azar, a Beastly Disease!).

Table 2
Category, title, institution, and year of publication of handbooks, guided studies, leaflets, and booklets [original titles in Portuguese translated into English].
MS = Brazilian Ministry of Health; FUNASA = Brazilian National Health Foundation; PAHO = Pan-American Health Organization; WHO = World Health Organization; SHS = State Health Department; MHS = Municipal Health Department.

Table 3
Target group, bibliography, and scale and authorship of illustrations for educational materials on leishmaniasis (n = 18) distributed in Brazil.