SciELO - Scientific Electronic Library Online

 
vol.46 issue2The individual with leg ulcer and structured nursing care intervention: a systematic literature reviewWork-related illness in nursing: an integrative review author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Revista da Escola de Enfermagem da USP

Print version ISSN 0080-6234

Rev. esc. enferm. USP vol.46 no.2 São Paulo Apr. 2012

http://dx.doi.org/10.1590/S0080-62342012000200030 

ARTIGO DE REVISÃO

 

The use of validated questionnaires to measure adherence to arterial hypertension treatments: an integrative review

 

Utilización de cuestionarios validados para mensurar la adhesión al tratamiento de la hipertensión arterial: una revisión integral

 

 

José Wicto Pereira BorgesI; Thereza Maria Magalhães MoreiraII; Malvina Thaís Pacheco RodriguesIII; Célida Juliana de OliveiraIV

IRegistered nurse. Specialist in Clinical Nursing, Master's student of Clinical Healthcare of the Ceará State University. Member of the Epidemiology Group, Chronicity and Nursing Care. CAPES Scholarship. Fortaleza, CE, Brazil. wictoborges@yahoo.com.br
IIRegistered nurse. PhD in Nursing of the Ceará Federal University. Professor of the Post-graduation Program of the Ceará State Univesty. Leader of the Epidemiology Group, Chronicity and Nursing Care. CNPq Researcher Fortaleza, CE, Brazil. tmoreira@uece.br
IIIRegistered nurse. MSc in Education. PhD Student in Public Health of the Ceará State Univesty. Professor of the Agriculture College of the Piauí Federal University. Member of the Epidemiology Group, Chronicity and Nursing Care of the Ceará State Univesty. CAPES Scholarship. Fortaleza, CE, Brazil. malvinat@gmail.com
IVRegistered nurse. MSc in Clinical Healthcare.  PhD in Nursing of the Ceará State University. Adjunct Professor of the Cariri Regional University Fortaleza, CE, Brazil. celidajuliana@yahoo.com.br

Correspondence addressed

 

 


ABSTRACT

The objective of this study was to analyze the scientific production in collective health regarding which validated questionnaires are being used to evaluate adherence to hypertension treatments. This integrative review was performed on the SciELO, MEDLINE and LILACS databases. Nine studies were selected, which used seven questionnaires: Moriski-Green Test, Cuestionário de Valoración de Adherencia, Cuestionário MBG, Questionário QAM-Q, Haynes Test, Escala de Conductas em Salud, and Hill-Bone compliance Scale. We concluded that the evaluation of hypertension treatment adherence is still an field open of research, due to the fact that the different instruments used have limitations, and there is no ideal method yet identified.

Descriptors: Hypertension; Therapeutics; Medication adherence; Patient compliance; Questionnaires


RESUMEN

Este estudio tuvo como objetivo analizar en la producción científica de salud colectiva cuáles cuestionarios validados están siendo utilizados para evaluar la adhesión al tratamiento de la hipertensión. Revisión integral realizada en las bases de datos SciELO, MEDLINE y LILACS. Fueron seleccionados nueve estudios que utilizaron siete cuestionarios: Test de Moriski-Green, Cuestionario de Valoración de Adherencia, Cuestionario MBG, Cuestionario QAM-Q, Test de Haynes, Escala de Conductas en Salud y Hill-Bone Compliance Scale. Concluimos en que la evaluación de la adhesión al tratamiento de la hipertensión aún es campo abierto para investigación, los diferentes instrumentos utilizados tienen sus limitaciones y no existe un método ideal.

Descriptores: Hipertensión; Terapéutica; Complimiento de la medicación; Cooperación del paciente; Cuestionarios


 

 

INTRODUCTION

Currently, it is recognized that the generic impact of illness on people can not be described in its entirety by objective measurements of health(1). The challenge of objectively capturing subjective attributes of different topics related to the health-disease-care process is driving the creation and adaptation of instruments, which, once validated, allow proximity with reality in order to support the most effective interventions(2). The search for information from the measurements of subjective properties has motivated many researchers to develop models that can estimate these properties(3). In the context of arterial hypertension, a chronic disease that requires pharmacological and non-pharmacological treatment for life, the major challenge makes itself known: the adherence to the therapy as a subjective construct to be measured.

Adherence to hypertension treatment is understood as the degree of coincidence between the behavior of the user and the recommendation of the healthcare professional in view of the treatment regimen(4). It is expected that this will result in the control of arterial pressure, the reduction in the incidence or delay in the occurrence of complications and an improvement in the quality of life(5). Although various strategies exist to evaluate the adherence to hypertension treatment, there is no consensus on a gold standard(6). The methods can be direct, such as the dosage of the principle active/metabolite of the drug or indirect, such as counting the tablets and user reports(7). Some authors have developed generic questionnaires to measure treatment adherence in chronic diseases, which can also used for people with hypertension(8). The use of validated instruments to measure the adherence construct allows for precision regarding this subjective data in the quotidian care of people with hypertension. Validity refers to the aspect of the measure, which is consistent with the measured property of the objects, showing the representation of this construct in the practice(9). Thus, when validated instruments are used to measure adherence, reliable results will be made use of that can be used by healthcare managers and teams as support for the establishment of improvements in the services, culminating in an increase in the adherence. The objective of this study was to analyze, in the scientific production of collective health, which validated questionnaires are being used to evaluate the adherence to hypertension treatment.

 

METHOD

This study was conducted through an integrative literature review, which is based on the analysis of material, emphasizing the organization and interpretation in the care to the object of study(10). This integrative review carefully followed six steps: selection of the guiding question; definition of the characteristics for the preliminary studies of the sample; selection of the studies that comprised the sample of the review; analysis of the findings of the articles included in the review; interpretation of the results and report of the review, which provided a critical examination of the findings.

Analysis of the selected studies was performed, guided by the research question: of the questionnaires that have passed through the validation process, which are being used for the evaluation of the adherence to arterial hypertension treatment? A search was carried out in the electronic databases SciELO, MEDLINE and LILACS. The period defined for the article search was from 2000 to 2009. The following descriptors were used: hypertension, adherence, compliance, test, epidemiologic studies, epidemiology and measurement, according to the DeCS health terminology of the Virtual Health Library To systematize the searches Boolean operators were used in the following way: (Hypertension and adherence and epidemiology/Hypertension and compliance and epidemiology) in the subsequent searches, this pattern was used consecutively changing the last descriptor, systematically considering all the descriptors. The following inclusion criteria were adopted: articles that focused analytically on the adherence to arterial hypertension treatment; available in their entirety; in Portuguese, Spanish or English; published between 2000 and 2009; studies with empirical research data. The exclusion criteria adopted were: to be a prior communication, a literature review or a theoretical review.

The data collection phase occurred from March to May 2010 in the previously mentioned electronic databases. As a result of the search 249 potential articles were identified for the study, with reading of the abstracts carried out as a screening for relevance and properties that conformed to the aim of the study. After the reading of the abstracts, 33 articles, available in full, were pre-selected, reread and selected again. Of these, the following were excluded: five articles about people with hypertension and other diseases, six that addressed adherence without measuring it, three that addressed adherence of the professionals to the clinical guidelines for hypertension management and two theoretical articles. Thus, a total of 17 studies were selected, among which nine used validated questionnaires to analyze the adherence to hypertension treatment, which comprised the study sample. The variables selected for analysis were: type of study, study site, subjects, sample, year of performance, data collection, adherence to treatment evaluation instruments, prevalence of the adherence to treatment. To ensure the validity of the review, the selected studies were critically analyzed in detail, seeking explanations for the different or conflicting results(10-11).

 

ANALYSIS AND DISCUSSION OF THE RESULTS

Regarding the characterization of the studies, the articles presented different characteristics in relation to the countries in which they were performed, to the subject and to the methodological design. These characteristics are shown in Chart 1.

Box 1 shows that the studies which evaluate the adherence to hypertension treatment through validated questionnaires have been increasing over the decade examined, with an increase in the frequency in the years 2008 and 2009. A likely explanation for this growth is the interest of the healthcare professionals in understanding the phenomenon of adherence/non-adherence, which has a great impact on the morbidity and mortality of people suffering from chronic health problems(12). It is also evident that the issue of the evaluation of adherence to hypertension treatment, considering the entire burden that this morbidity places on society, requires the efforts of specialists from various countries in the search for the elucidation of the prevalence of the adherence. Among the selected studies, seven were performed in the Americas, with: three in Brazil, one in Mexico, one in Cuba, one in Colombia and one in Chile; and two in Europe: one in Spain and one in the United Kingdom. One review showed that various countries have been developing studies in search of an index of adherence to pharmacological treatment of arterial hypertension(13).

Regarding the study designs, the theme was studied in depth, since the studies shows observational and analytical designs aiming to know the area of adherence to arterial hypertension treatment and to correlate it with some variables, in search of subsidies that address the problem. The studies were distributed into: one case control, one cohort, one survey, two methodological studies, one cross-sectional population-based, two exploratory-descriptive and one correlational descriptive, according to the definitions of the studies themselves. The study design refers to the general plan of the researcher in order to respond to the research questions or their hypotheses(14). In the articles studied, considering the randomized samples, two studies used experimental research designs.

In relation to the subjects involved in the studies, there was variation in sample sizes, which can be explained by the diversity of sites where the studies were conducted and the various study designs. Although eight studies considered people with arterial hypertension as the research subjects, only one selected the sample as people who have some cardiovascular risk factor, included in this review because all sample subjects were hypertensive. Therefore, concerning the inclusion criteria of the samples, there is: one study that considered uncontrolled hypertensive patients, two studies were with elderly hypertensive patients, one with hypertensive resistant patients, and four with hypertensive patients enrolled in Primary Healthcare Units.

Concerning the adherence to arterial hypertension treatment evaluation instruments, there was the use of validated questionnaires concerning the methodological instruments that evaluate user behavior in addition to managing empirical data that is obtained by simple observation of the quotidian or habitual behavior of the user. Table 1 lists the seven instruments highlighted in this review.

 

 

The Morisky-Green test was the most used, being present in four studies. It is a measure constructed in 1986 and constitutes the instrument most widely used to measure adherence to the use of medication. The fundamental theory of this measure comprehends that the inappropriate use of medication occurs in one or all of the following ways: forgetfulness, carelessness, interrupting the medication when feeling better or interrupting the medication when feeling worse. It is easy to measure and validated, with a relatively small number of comprehensible questions, which provide a verification of the attitude of the user concerning taking medication(15). It is a qualitative instrument and the questions are atemporal(16). It has been validated in English, Spanish and Portuguese.

The evaluation of adherence to pharmacological and non-pharmacological treatment questionnaire was used in one study(17). The questionnaire was used in its original version by the authors, who went through the process of content validation by expert nurses in Colombia with reference to the indicators and the semantic scale of the NOC (Nursing Outcomes Classification) Taxonomy. It consists of 14 Likert type items with five response alternatives where one is equal to never, being the lowest value and five equal to always, being the highest value. The scale is available in Spanish.

The development and validation of adherence to treatment evaluation questionnaires was the subject of two studies(6,18). The MBG (Martin-Bayarre-Grau) Questionnaire, developed at the National School of Public Health of Cuba, was constructed from the definition of therapeutic adherence, developed by the authors: active and voluntary actions of the user to adopt behavior related to compliance with the treatment arranged in agreement with the physician(18).

The MBG Questionnaire is self-administered, short, easy to apply and presents convenient response possibilities for the user. It was validated only in Spanish, a process that took place in Havana, Cuba. It has 12 items in the form of statements, with a Likert-type response scale of five possible answers ranging from never to always. To calculate the score obtained by each user, it is considered that the value 0 corresponds to never, 1 to almost never, 2 sometimes, 3 almost always and 4 always, with a total of 48 points attainable. The authors also standardized the values ​​considering the scores obtained: total adherence those that obtain 38 to 48 points; partial adherence 18 to 37 points, and non-adherence 0 to 17 points, so that it can be quantified quickly to determine three levels of adherence: total, partial and non-adherence. The questionnaire presented a Cronbach's alpha internal consistency of 0.889 and the content validation was performed by experts(18).

The other questionnaire developed was the Questionnaire for Adherence to Medication - Qualiaids (QAM-Q) developed to address the act (if the individual takes and how they take their medication), the process (how the individual takes the medication within a period of seven days, if they skip doses, if they take it erratically, if they have "holidays") and the result of adherence (in this case, if their arterial pressure was controlled) with AIDS patients on antiretroviral therapy(6).

The QAM-Q has three measures of non-adherence: 1) Proportion of doses consumed - continuous measurement of the adherence act: number of tablets consumed multiplied by the number of times, divided by the number of pills prescribed multiplied by the number of times; 2) Process of taking the medication - ordinal measure of the adherence process: frequency of occurrence of abandonment (has not taken any dose of any medication in the last seven days), holidays (the user does not take any medication on that day), taking erratically (stops taking medication on different days and at variable times), or half-adherence (takes one medication correctly and another incorrectly), and finally, 3) referred outcome - dichotomous measurement of the adherence result: report of the last arterial pressure measurement, stating whether it was normal or altered(6). The aforementioned researchers applied the QAM-Q with hypertensive patients and constructed a composite measure, in which the presence of one of these conditions was sufficient to classify the respondent as non-adherent: either not taking the correct quantity (80%-120% of the prescribed doses), or not taking it in the correct way (no "holidays", "taking erratically", abandonment or "half-adherence"), or reports that their blood pressure was altered. The measurements of accuracy to detect non-adherence showed a sensitivity of 62.5% and a specificity of 85.7%, the area under the ROC curve of 74.1% and a positive predictive value of 90.9%. The validation study for users with hypertension occurred in Blumenau, Santa Catarina, Brazil and is found only in Portuguese(6,16).

Another instrument for evaluating non-adherence was the Haynes-Sackett test(6). In this test the user makes a self-report of their treatment adherence by means of a question: Are those who have adherence equal to or greater than 80% considered compliant?(19).

The Health Behavior Scale was used in a study in Chile(20). The authors used a modified scale to characterize the treatment adherence. The sum of the scores of the responses (1=never, 2=sometimes, 3=always) produce a possible total score of 12 to 36 points. Four subscales inquire about diet, physical activity, the taking of medication and whether they can control stressful situations in each of the contexts developed in the daily life (home, recreation and social activities). The original scale has a Likert format of five alternative responses that allow a maximum of 100 points to be achieved and has a fifth subscale that inquires about smoking and comprises the workplace. Despite the changes made by the authors, the scale still obtained acceptable scores, with the Cronbach's alpha internal consistency of between 0.70 and 0.85. It has only been validated in Spanish

The Hill-Bone compliance to high blood pressure therapy scale (Hill-Bone compliance scale) was used in one study(21), and evaluates user behavior in three important behavioral domains in the treatment of elevated arterial pressure: 1) reducing sodium intake, 2) number of consultations and 3) taking the medication. This scale is comprised of 14 items in three subscales. Each item is a four-point Likert type scale and was developed and validated by the School of Nursing at Johns Hopkins University, Baltimore, USA(22). The scale was validated in English.

Regarding the hypertension treatment adherence scores, it is difficult to detect the adherence and, even more so, to quantify it. Although the adherence is often described as a dichotomous variable adherence x non-adherence, it can vary over a continuum of zero to more than 100% in users who use more than the medication prescribed by a physician(13).

Figure 1 shows the adherence prevalence values found from the application of the validated questionnaires. These scores, although presented together, have the limitation that they can not be compared in a linear relationship, primarily due to the differences between the questionnaires, because they measure different constructs related to adherence and according to the sample characteristics.

 

 

In Spain, a group of researchers applied the Morisky-Green Test with a sample of 4,000 people over 60 years of age and found a prevalence of 53% of good therapeutic behavior(23). The authors stratified the results according to gender and no statistically significant difference was found, with a adherence prevalence of 52.3% in the men and 53.9% in the women. In a reference hospital in Rio de Janeiro, Brazil, 51% adherence was found in a sample of resistant hypertension patients, using the same instrument(24). According to the authors, the Morisky-Green Test did not present good performance in this population, when used alone, requiring other methods in association to evaluate the resistant hypertension patients(24).

In southern Brazil an adherence prevalence of 56.6% was found with the Morisky-Green Test(6). This result was used as the gold standard combined with other methods (adherence from the report of the physician and adherence from the report of the user), in the evaluation of a questionnaire that was under construction. One of the limitations of this finding is the small sample (n=46), which affects the interpretation of this result.

In São Paulo, Brazil, the application the same test with a sample of 68 hypertensive patients of a hospital obtained an adherence prevalence of 13.07%(25). This low adherence may be associated with a selection bias because the sample consisted of hypertensive patients with uncontrolled blood pressure, where the inclusion criterion was having arterial pressure greater than 140x90mmHg.

The use of the Hill-Bone Scale resulted in 41.7% adherence in a sample of 66 hypertensive patients of various ethnicities in Birmingham, United Kingdom(21). This scale was used in a modified version to evaluate user behavior and attitude toward the taking of the medication. The original scale is American and the adaptation used has not been through the validation process, which can compromise the reliability of this prevalence found.

Considering the use of the QAM Questionnaire, its application resulted in an adherence prevalence of 47.8% in southern Brazil. The study points out some limitations: the small number of cases, the performance in one context and with only one pathology, which may limit the generalization of the results. However, the QAM-Q seems to be capable of obtaining similar or better results when combined with other objective and subjective methods of measuring adherence. Another method used in the same study, the Haynes Test, which measures medication adherence in the last 30 days, obtained an adherence prevalence of 8.7%(6). This low adherence may be associated with memory bias, which might negatively influence the method.

In Chile, the application of the Health Behavior Scale(20) showed an adherence of 22.7% in a sample of 211 hypertensive patients over 64 years of age. The study warns that this result may be biased due to the mental state of the individuals, or the use of drugs that alter the central nervous system. In Colombia, the use of the Evaluation questionnaire of pharmacological and non-pharmacological treatment adherence behavior showed an adherence of 59.6% in a sample of 201 people, the authors emphasize that the sample users presented great difficulty adhering to the non-pharmacological treatment

 

CONCLUSION

This review aims to contribute to addressing a theme relevant to Brazilian and global public health, which is the measurement of hypertension treatment adherence. Although there are different instruments that can be used to measure hypertension treatment adherence, there is not an instrument that is adequate for all the studies where the results can be compared. Therefore, research is needed to develop and validate questionnaires directed toward the hypertensive user in order to improve the quality of life of these users and to reduce the demand on the healthcare services.

 

REFERENCES

1. Prieto L, Badia X. Cuestionarios de salud: concepto y metodología. Aten Primaria. 2000;28(3):201-9.         [ Links ]

2. Graziano KU, Padilha KG. Construction and adaptation of management and health care measures in nursing [editorial]. Rev Esc Enferm USP  [Internet]. 2009  [cited  2010  Sept  12];43(n.espe): 989-9. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0080-62342009000500001&lng=pt&nrm=iso&tlng=en        [ Links ]

3. Araújo EAC, Andrade DF, Bortolotti SLV. Item response theory. Rev Esc Enferm USP [Internet]. 2009  [cited 2010 Sept  12] ;43(n.espe):1000-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/en_a03v43ns.pdf         [ Links ]

4. World Health Organization (WHO). Adherence to long term therapies: evidence for action. Geneva; 2003.         [ Links ]

5. Souza WA. Avaliação da adesão ao tratamento e dos resultados clínicos e humanísticos na investigação da hipertensão arterial [tese doutorado]. Campinas: Universidade Estadual de Campinas; 2008.         [ Links ]

6. Santa-Helena ET, Nemes MIB, Eluf-Neto J. Desenvolvimento e validação de questionário multidimensional para medir não-adesão ao tratamento com medicamentos. Rev Saúde Pública. 2008;42(4):764-7.         [ Links ]

7. Gusmão JL, Ginani GF, Silva GV, Ortega KC, Mion Jr D. Adesão ao tratamento em hipertensão arterial sistólica isolada. Rev Bras Hipertens. 2009;16(1):38-43.         [ Links ]

8. Borges JWP. Métodos de avaliação da adesão ao tratamento da hipertensão arterial: uma revisão integrativa da literatura [monografia]. Fortaleza: Universidade Estadual do Ceará; 2010.         [ Links ]

9. Pasquali L. Psychometrics. Rev Esc Enferm USP  [Internet]. 2009 [cited 2010  Apr 12];43(n.esp): 992-9. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/en_a02v43ns.pdf        [ Links ]

10. Mendes KS, Silveira RCCP, Galvão MC. Revisão integrativa: método de pesquisa para a incorporação de evidências na saúde e na enfermagem. Texto Contexto Enferm. 2008;17(4):758-64.         [ Links ]

11. Souza MT, Silva MD, Carvalho RC. Revisão integrativa: o que é e como fazer. Einstein. 2010;8(1):102-6.         [ Links ]

12. Reiners AAO, Azevedo RCS, Vieira MA, Arruda ALG. Produção bibliográfica sobre adesão/não-adesão de pessoas ao tratamento de saúde. Ciênc Saúde Coletiva. 2008; 13 Supl 2:2299-306.         [ Links ]

13. Barbosa RGB, Lima NKC. Índices de adesão ao tratamento anti-hipertensivo no Brasil e mundo. Rev Bras Hipertens. 2006;13(1):35-8.         [ Links ]

14. Polit DF, Beck CT, Hungler BP. Fundamentos de pesquisa em enfermagem: métodos, avaliação e utilização. 5ª ed. Porto Alegre: Artmed; 2004.         [ Links ]

15. Ungari AQ. Adesão ao tratamento farmacológico de usuários hipertensos seguidos nos núcleos de saúde da família do município de Ribeirão Preto, SP [dissertação]. Ribeirão Preto: Faculdade de Medicina, Universidade de São Paulo; 2007.         [ Links ]

16. Santa-Helena ET. Adesão ao tratamento farmacológico de usuários com hipertensão arterial em unidades de saúde da família em Blumenau, SC [tese doutorado]. São Paulo: Faculdade de Medicina, Universidade de São Paulo; 2007.         [ Links ]

17. Velandia-Arias A, Rivera-Álvarez LN. Agencia de autocuidado y adherencia al tratamiento en personas con factores de riesgo cardiovascular. Rev Saúde Pública. 2009;11(4):538-48.         [ Links ]

18. Alfonso LM, Vea HDB, Ábalo JAG. Validación del cuestionario MBG (Martín-Bayarre-Grau) para evaluar la adherencia terapéutica en hipertensión arterial. Rev Cub Salud Pública [Internet]. 2008 [citado 2010 set. 14];34(1). Disponível em: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-34662008000100012&lng=es&nrm=iso&tlng=es        [ Links ]

19. Melchiors AC. Hipertensão arterial: análise dos fatores relacionados com o controle pressórico e a qualidade de vida [dissertação]. Curitiba: Universidade Federal do Paraná; 2008.         [ Links ]

20. Mendoza-Parra S, Merino JM, Barriga OA. Identificación de factores de predicción del incumplimiento terapéutico en adultos mayores hipertensos de una comunidad del sur de Chile. Rev Panam Salud Pública. 2009;25(2):105-12.         [ Links ]

21. Gohar F, Greenfield SM, Beevers DG, Lip GYH, Jolly K. Self-care and adherence to medication: a survey in the hypertension outpatient clinic. BMC Complement Altern Med. 2008;8:4.         [ Links ]

22. Kim MT, Hill MN, Bone LR, Levine DM. Development and testing of the Hill-Bone compliance to High Blood Pressure Therapy Scale. Prog Cardiovasc Nurs. 2000;15(3):90-6.         [ Links ]

23. Tuesca-Molina R, Guallar-Castillón P, Banegas-Banegas JR, Graciani-Pérez RA. Determinantes del cumplimiento terapêutico en personas mayores de 60 años en España. Gac Sanit. 2006;20(3):220-7.         [ Links ]

24. Bloch KV, Melo NA, Nogueira AR. Prevalência da adesão ao tratamento anti-hipertensivo em hipertensos resistentes e validação de três métodos indiretos de avaliação da adesão. Cad Saúde Pública. 2008;24(12):2979-84.         [ Links ]

25. Dosse C, Cesarino CB, Martin JFV, Castedo MCA. Fatores associados a não adesão dos usuários ao tratamento de hipertensão arterial. Rev Latino Am Enferm. 2009;17(2):201-6.         [ Links ]

 

 

Correspondence addressed to:
José Wicto Pereira Borges
Rua Irmã Bazet, 537 – Apto. 304A - Montese
CEP 60420-670 – Fortaleza, SP, Brasil

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License