The effect of psychotherapy provided as part of a pulmonary rehabilitation program for the treatment of patients with chronic obstructive pulmonary disease

Abstracts

OBJETIVO: Avaliar o efeito da psicoterapia sobre os níveis de ansiedade, depressão, qualidade de vida e capacidade de exercício em portadores de doença pulmonar obstrutiva crônica, submetidos a um programa de reabilitação pulmonar. MÉTODOS: Ensaio clínico prospectivo, controlado e cego realizado com 49 portadores de doença pulmonar obstrutiva crônica randomizados em três grupos: programa de reabilitação pulmonar completo (grupo 1), sem exercícios físicos (grupo 2) e sem psicoterapia (grupo 3). Foram utilizados para avaliação: inventários de ansiedade e de depressão de Beck, questionário respiratório de Saint George e o produto da distância caminhada pelo peso. RESULTADOS: Os grupos 1 e 2 demonstraram melhora absoluta estatisticamente significativa na capacidade de realizar exercícios (p = 0,007, p = 0,008 e p = 0,06, respectivamente). Os grupos 1 e 2 demonstraram redução significativa dos níveis de ansiedade e depressão, e melhora no índice de qualidade de vida: grupo 1 - p = 0,0000, p < 0,0003 e p = 0,0007; e grupo 2 - p = 0,0001, p = 0,0014 e p = 0,002, respectivamente. O grupo 3 demonstrou redução no nível de ansiedade (p = 0,03), mas não no nível de depressão e no índice de qualidade de vida, significativamente. CONCLUSÃO: Sessões de psicoterapia num programa de reabilitação pulmonar auxiliaram portadores de doença pulmonar obstrutiva crônica no manejo das limitações impostas pela doença, através da redução dos níveis de sintomas comportamentais, especialmente depressão, influindo na melhora da capacidade de realizar exercícios físicos e na qualidade de vida relacionada à saúde.

Doença pulmonar obstrutiva crônica; Doença pulmonar obstrutiva crônica; Ansiedade; Depressão; Qualidade de vida; Terapia por exercícios

ORIGINAL ARTICLES

The effect of psychotherapy provided as part of a pulmonary rehabilitation program for the treatment of patients with chronic obstructive pulmonary disease^*

Dagoberto Vanoni de Godoy^I; Rossane Frizzo de Godoy^II; Benno Becker Júnior^III; Paula Fernanda Vaccari^IV; Maurício Michelli^V; Paulo José Zimermann Teixeira^VI; Bruno Carlos Palombini^VI

^IPulmonologist at the University of Caxias do Sul (UCS), Caxias do Sul, Rio Grande do Sul, Brazil

^IIPsychologist at the University of Caxias do Sul (UCS), Caxias do Sul, Rio Grande do Sul, Brazil

^IIIPsychologist at the University of Cordoba - Spain

^IVPhysical Therapist at the University of Caxias do Sul (UCS), Caxias do Sul, Rio Grande do Sul, Brazil

^VEducator at the University of Caxias do Sul (UCS), Caxias do Sul, Rio Grande do Sul, Brazil

^VIPulmonologist at the Universidade Federal do Rio Grande do Sul (UFRGS, Federal University of Rio Grande do Sul), Porto Alegre, Rio Grande do Sul, Brazil

^{Correspondence to}

ABSTRACT

OBJECTIVE: To assess the effect of psychotherapy on levels of anxiety and depression, as well as on quality of life and exercise capacity in patients with chronic obstructive pulmonary disease enrolled in a pulmonary rehabilitation program.

METHODS: A randomized, controlled, blind clinical trial was conducted involving 49 chronic obstructive pulmonary disease patients. Patients were randomized into three groups: those submitted to the complete pulmonary rehabilitation program, which included psychotherapy and an exercise regimen (group 1); those submitted to the program minus physical exercise (group 2); and those submitted to the program minus psychotherapy (group 3). The three groups underwent a 12-week treatment program. All patients were evaluated at baseline and at completion of the pulmonary rehabilitation program through four instruments: The Beck Anxiety Inventory, Beck Depression Inventory and St. George's Respiratory Questionnaire were applied. The distance walked-weight product was also calculated.

RESULTS: Statistically significant absolute improvements in exercise capacity were found for groups 1 and 2, although not for group 3 (p = 0.007, p = 0.008 and p = 0.06, respectively). In groups 1 and 2, levels of anxiety and depressions were also significantly reduced (group 1: p = 0.0000 and p < 0.0003; group 2: p = 0.0001 and p = 0.0014), and quality of life was significantly improved (p = 0.0007 and p = 0.002, respectively). Anxiety levels were also reduced in group 3 (p = 0.03), although levels of depression were not, and quality of life was unaffected.

CONCLUSION: Psychotherapy sessions provided as part of a pulmonary rehabilitation program assist patients with chronic obstructive pulmonary disease in coping with disease-related limitations by reducing behavioral symptoms, especially depression, thereby influencing exercise capacity and health-related quality of life.

Keywords: Pulmonary disease, chronic obstructive/rehabilitation; Pulmonary disease, chronic obstructive/psychology; Anxiety; Depression; Quality of Life; Exercise therapy

INTRODUCTION

Pulmonary rehabilitation may be defined as an art of medical practice focused on the stabilization or reversion of the physiological and psychological effects of the pathogenesis of pulmonary diseases. The aim is to restore the highest degree of performance capacity that is compatible with the pulmonary function and general life situation of patients with lung disease. Rather than a therapeutic approach restricted to medical practice, pulmonary rehabilitation should be considered an interdisciplinary program that takes into account the establishment of a precise diagnosis, pharmacological treatment, physical therapy, physical reconditioning, psychological support and education. Pulmonary rehabilitation programs (PRPs) should be adapted to the needs of each individual patient. Some authors consider PRPs the standard therapeutic approach for many patients with severe respiratory disease, particularly for those with chronic obstructive pulmonary disease (COPD).^(1-3) However, to date, the mechanism responsible for the clinical improvement of patients with lung disease submitted to PRPs has not been fully elucidated. The increased motivation, together with desensitization to dyspnea, the learning of techniques that may facilitate daily life activities, and the training of respiratory and skeletal muscles, contribute, to varying degrees (as yet not well defined), to the well-being of the patient.^(4-5)

The objective of this study was to analyze the effects of psychotherapy sessions on the severity of the behavioral symptoms anxiety and depression, on quality of life and on exercise tolerance in patients with COPD participating in a PRP.

METHODS

A prospective, controlled, randomized, blind (patients) clinical trial was conducted at the University of Caxias do Sul Central Outpatient Clinic and Vila Olímpica.

The study involved adults of both genders with COPD, staged as moderate and severe, corresponding to stages and as defined by the Global Initiative for Chronic Obstructive Lung Disease Scientific Committee.⁽⁶⁾ The sample consisted of consecutive patients treated in the outpatient clinic of the University of Caxias do Sul Pulmonology and Thoracic Surgery Department. All patients were referred to the clinic by pulmonologists or thoracic surgeons. Initially, 58 patients were referred to the PRP, but 6 of those patients declined to participate in the study. Of the remaining 52 patients, 3 were excluded during the first month of activities - 2 due to lesions in the spinal column and 1 due to severe respiratory infection that required long-term hospitalization. Therefore, the final study sample comprised 49 consecutive patients randomized into three groups (G1, G2 and G3).

The medical evaluation consisted of anamnesis, physical examination and postbronchodilation spirometry (Spirodoc; Medical International Research, Rome, Italy), as well as frontal and lateral view chest X-rays.

The measurement instruments described herein were applied by a psychologist blinded as to the group to which any given patient belonged.

The Beck Anxiety Inventory (BAI) consists of a list of 21 symptoms with four alternatives each. The alternatives are listed in ascending order by level of anxiety. In Brazil, the BAI was validated with the following classification: 0 to 9 = minimal; 10 to 16 = mild; 17 to 29 = moderate; and 30 to 63 = severe. Anxiety is considered clinically significant when classified as mild or higher.^(7-8) The Beck Depression Inventory (BDI) consists of 21 categories of symptoms and activities, with four alternatives each. The alternatives are listed in ascending order by level of depression. The BDI was validated in Brazil with the following classification: 0 to 11 = minimum; 12 to 19 = mild; 20 to 35 = moderate; and 36 to 63 = severe. Depression is considered clinically significant when classified as mild or higher.^(7-8) The Saint George's Respiratory Questionnaire (SGRQ), also validated in Brazil, evaluates three domains: symptoms; limitations in daily life activities; and impact of the disease on the individual. Each domain has a maximum possible score, and differences of greater than 10% reflect a life quality alteration in that domain. Alterations equal to or greater than 4% after an intervention indicate a significant change in patient quality of life.⁽⁹⁾ The distance walked-weight product is obtained by multiplying the distance walked by patient body weight. This parameter presents good correlation with the anaerobic threshold and with maximal oxygen uptake in patients with COPD.⁽¹⁰⁾ We used the distance walked in six minutes.

The study protocol was approved by the Ethics in Research Committee of the University of Caxias do Sul.

The G1 patients received the complete PRP. The participants in this group performed various activities as listed below. There were two weekly sessions of physical exercises, including upper-body, lower-body and flexibility exercises, as well as aerobic exercises using a treadmill. The intensity of the exercise was modulated based on patient signs and symptoms. The principal objective was to exercise patients at 75% to 85% of maximum heart rate. The exercises were suspended when the heart rate exceeded the limits presented above or when the patient presented any of the following: cardiac arrhythmia; arterial blood pressure above 180/110 mmHg; chest pain; blurred vision; pallor; cold sweats; hemoglobin oxygen saturation below 85%; and deterioration of motor coordination or level of consciousness. This activity was supervised by the physical education instructor.

Individual psychotherapy sessions were held on a weekly basis and were conducted by a psychologist. The psychotherapy sessions addressed the psychological needs of the patients, including difficulties in social life, in marital life, at work and with their health. To that end, cognitive-behavioral therapy and logotherapy techniques were used.^(11-13) The maintenance of anxiety control was performed through cognitive-behavioral techniques. For example, facing a situation that is a dyspnea trigger, the patient was taught to apply the following sequence of actions: "stop"; "calm down"; "breathe"; "notice the reduction of anxiety"; and "take control of the situation". In the logotherapy, also known as "therapy through meaning", patients were asked to reevaluate their quality of life, seeking alternatives to cope with the painful situations caused by the disease.

A monthly group educational session to discuss important COPD topics was conducted by the pulmonologist. The objective of the educational sessions was to enable patients to incorporate into their daily lives the knowledge imparted during the PRP activities.

Meetings with the physical therapist to work on respiratory rehabilitation were held twice weekly. During the physical therapy sessions, diaphragmatic breathing techniques were taught, as well as the use of an anchor point and pursed-lip breathing. Another goal was to remove excess respiratory secretions through postural drainage and coughing, as well as chest percussion and vibration.

The G2 patients received psychotherapy (identical to that given to the G1 patients) but did not participate in the physical exercise sessions.

The G3 patients participated in physical exercise sessions (identical to those attended by the G1 patients) but did not receive psychotherapy.

The activities of the various groups were scheduled at different times in order to ensure that the patients in each group were blinded as to the activities of those in the other groups.

At the end of the twelve weeks of PRP, as a post-test, the patients went through the same phases used in the initial evaluation, being evaluated by the same professionals using the same methodology.

The quantitative data are expressed as means and standard deviations. Data related to categorical variables are expressed as percentages. In the comparison of the groups by the type of variable analyzed, we used the following statistical tests: Student's t-test, ANOVA and chi-square test. The level of significance adopted in the study was = 0.05. The data were processed and analyzed using the programs SPSS for Windows, version 6.0, and Epi Info, version 6.04.

RESULTS

The 49 patients were studied from October 1999 to December 2001 and were divided as follows: G1: 19 patients; G2: 16 patients; and G3: 14 patients.

The comparisons among the groups regarding spirometric variables, COPD severity and body mass index are shown in Table 1. There was a predominance of males (73%) and Caucasians (93%). All of the patients were over 50 years old, 69% were married, 8% were illiterate, and 71% had completed elementary school. A total of 55% of the patients had an income of over three times the minimum wage. All were using medication, principally inhaled beta-2-agonists (90%), ipratropium (67%), oral or inhaled corticosteroids (57%) and xanthines (51%). Home oxygen therapy was used by 8% of the patients, antidepressants by 10% and anxiolytics by 6%. The most common concomitant diseases were: systemic arterial hypertension (20%), ischemic cardiopathy (18%) and diabetes mellitus (16%). At the beginning of the PRP, 5 (10%) of the patients were still smokers. None of the patients presented severe nutritional disturbances. Mean body mass index and mean serum electrolytes related to respiratory muscle function were normal. In the spirometric measurements, mean forced vital capacity was 66% of predicted, and mean forced expiratory volume in one second was 34% of predicted. On average, the patients were not polycythemic or hypercapnic, although most presented hypoxemia (hemoglobin oxygen saturation of 91%).

The groups were similar in the initial evaluation regarding BAI, BDI, SGRQ, distance walked-weight product and percentage of maximum heart rate attained.

The BAI and BDI scores indicated that the mean impact on the behavioral symptoms anxiety and depression was from mild to moderate. The SGRQ scores indicated that patient quality of life was seriously affected in all three domains: symptoms; limitations in daily life activities; and impact of the disease. The patients in all three groups exercised within a similar range of intensity, measured by the percentage of maximum heart rate attained (between 70% and 80%), with equal adherence to the PRP activities.

At the end of the PRP, G1 and G2 patients demonstrated statistically significant reductions in their levels of anxiety and depression, as well as statistically significant improvement in quality of life and exercise tolerance. The final mean BAI and BDI scores indicated that anxiety and depression were minimal. All of the SGRQ domains were at least 4% lower at the end of the PRP. Among G3 patients, the only statistically significant post-PRP reduction was that seen in levels of anxiety. There was no statistically significant improvement in quality of life or in the distance walked-weight product.

The mean final BAI and BDI scores indicated that anxiety and depression were minimal. None of the SGRQ domains suffered a mean reduction of less than 4% by the end of the PRP. The data regarding BAI, BDI, SGRQ, distance walked-weight product and percentage of maximum heart rate attained are presented in Table 2.

In an ANOVA of the three groups, including the quantitative covariables distance walked-weight product, BAI and BDI, increased exercise tolerance was not found to have any impact on levels of anxiety and depression.

There were no statistical differences among the three groups regarding the confounding variables requiring hospitalization during the PRP (p = 0.8) and COPD exacerbation (p = 0.2).

DISCUSSION

The analysis of the evolution of the patients belonging to PRP groups that were partially distinct demonstrated that the inclusion of psychotherapy sessions in the PRP was beneficial for those patients. A reduction in the levels of the behavioral symptoms anxiety and depression was obtained, as well as an improvement in the quality-of-life index.

Reviews of the literature have demonstrated a notable prevalence of anxiety and depression in individuals with moderate or severe COPD. Clinically significant levels of anxiety are found in 40% to 96% of such patients,^(14-16) and depression is seen in 51% to 74% of cases.^(17-18) It is possible that depression and anxiety are being underdiagnosed and undertreated in the population of patients with COPD.⁽¹⁹⁾

In the present study, a post-PRP reduction in anxiety was observed in all of the groups analyzed. However, this reduction was significantly greater in patients receiving psychotherapy (G1 and G2 patients) than in those not receiving such therapy (G3 patients). After the PRP, depression was reduced in G1 and G2 patients, whereas it was increased in G3 patients. These results suggest the need to include psychological support in PRPs in order to better understand and motivate patients. In a study of 55 individuals with COPD, it was reported that symptoms of anxiety and depression were reduced after a single session of cognitive intervention and relaxation technique training.⁽²⁰⁾ The authors of another study showed increased exercise tolerance, increased cognitive performance (greater verbal fluency) and reduced anxiety in a subgroup of PRP patients who were trained using education, exercise and stress management techniques.⁽²¹⁾ In a cases series study involving 95 patients with COPD treated in a PRP that included meetings with a clinical psychologist, anxiety and depression were reduced by 81% and 82%, respectively, by the end of the program, and a control analysis carried out three months later showed that these levels remained stable.⁽²²⁾

Patients with COPD are often segregated from society, most receiving fragmented and incomplete treatment, which results in a low quality of life for the patient and family. Those PRPs that last from six to ten weeks have been shown to have a favorable affect on the quality of life of patients with COPD.^(23-25) A marked increase in quality of life was observed in a ten-week PRP involving 30 patients with COPD.⁽²⁴⁾ In the present study, baseline quality of life was similar in all three groups. Statistically significant favorable responses in the levels of quality of life, measured through the use of the SGRQ, were obtained in all three groups, although the improvement in G3 was significantly inferior to those seen in G1 and G2.

Patients with moderate to severe COPD typically experience exercise intolerance, which may result in a severe reduction in functional mobility. The mechanisms and the magnitude of post-PRP improvement depend on the clinical profile, the degree of increase in aerobic capacity, the extent to which skeletal and respiratory muscles recover, how advanced the exercise techniques employed are and the degree to which dyspnea is reduced, as well on patient motivation. In the comparison between the baseline and post-PRP values, the patients in all three groups presented clinical improvement in exercise tolerance. The G3 patients presented a clinically significant, albeit less than statistically significant, increase in exercise tolerance. It should be noted that the improved physical performance seen in G2 patients might be attributable to the psychotherapy, in which was designed to prepare patients to better cope with the pain imposed by the disease. Following this same line of thinking, the fact that the increase in exercise tolerance was found to be less pronounced among G3 patients might have been related to the higher levels of depression seen in those patients.

Physical activity may be an excellent alternative means of unloading or releasing tensions, emotions and frustrations accumulated as a result of the pressures of modern life. However, the therapeutic impact of exercise on levels of anxiety and depression has not been well defined.^(26-32) In the sample studied, no relation was found between improved physical performance and lower levels of anxiety and depression.

In conclusion, the inclusion of psychotherapy sessions in a PRP designed to treat patients with moderate to severe COPD reduced levels of the behavioral symptoms anxiety and depression, improved the quality-of-life indices and have a favorable effect on exercise tolerance. Further studies are needed in order to determine the duration of the beneficial effects that psychotherapy during a PRP has on anxiety and depression in this patient population.

REFERENCES

Submitted: 24 June 2004. Accepted, after review: 6 May 2005

Sponsor: Grant of Scientific Productivity, President's Office of Graduate Studies and Research of the University of Caxias do Sul, Brazil.

• 1. Pulmonary rehabilitation: joint ACCP/AACVPR evidence-based guidelines. AACP/AACVPR Pulmonary rehabilitation Guidelines Panel. American College of Chest Physicians. American Association of Cardiovascular and Pulmonary Rehabilitation. Chest. 1997;112(5):1363-96.
• 2. Celli BR. Pulmonary rehabilitation for patients with advanced lung disease. Clin Chest Med. 1997;18(3):521-34.
• 3. Fahy B. Pulmonary rehabilitation: nuts and bolts. Clin Pulm Med. 2000;7:319-25.
• 4. Lacasse Y, Brosseau L, Milne S, Martin S, Wong E, Guyatt GH, et al. Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2002;(3):CD003793.
• 5. Sinn DD, McAllister FA, Man SF, Anthonisen NR. Contemporary management of chronic obstructive pulmonary disease: scientific review. JAMA. 2003;290 (17):2301-12.
• 6. GOLD Global Initiative for Chronic Obstructive Lung Disease [home page on the Internet]. Global Initiative for Chronic Obstructive Lung Disease: 2000 [updated 2002 Aug 28; cited 2005 Jan 9]. Available from: www.goldcopd.com
• 7. Cunha JA. Generalidades sobre a versão brasileira do BDI,BAI,BHS e BSI. In: 7ş Encontro Nacional sobre testes psicológicos; 1ş Congresso Ibero-Americano de Avaliação Psicológica. Anais. Porto Alegre, 1997. p.135-9.
• 8. Cunha JA. Estudos dos pontos de corte do BDI e BAI na versão em português [pôster]. In: 8ş Congresso Nacional de Avaliação Psicológica. Anais. Porto Alegre; 1999.
• 9. Souza TC, Jardim JR, Jones P. Validação do questionário do Hospital Saint George na doença respiratória (SGRQ) em pacientes portadores de doença pulmonar obstrutiva crônica no Brasil. J Pneumol. 2000;26(2):119-28.
• 10. Chuang ML, Lin IF, Wasserman K. The body-walking distance product as related to lung function, anaerobic threshold and peak VO₂ in COPD patients. Respir Med. 2001; 95(7):618-26.
• 11. Beck AT, Rush AJ, Shaw BF, Emery GE. Cognitive therapy of depression. New York: Guilford Press; 1979.
• 12. Beck AT, Emery GE, Greenberg RL. Anxiety disorders and phobias: a cognitive perspective. New York: Basic Book, 1985.
• 13. Frankl VE. Psychotherapy and existentialism: selected papers on logotherapy New York: Simon & Schuster; 1985.
• 14. White RJ, Rudkin ST, Ashley J, Stevens VA, Burrows S, Pounsford JC, et al. Outpatient pulmonary rehabilitation in severe chronic obstructive pulmonary disease. J R Coll Physicians Lond. 1997;31(5):541-5.
• 15. Agle DP, Baum GL. Psychological aspects of chronic obstructive pulmonary disease. Med Clin North Am. 1977;61(4):749-58.
• 16. Godoy RF, Becker Jr B, Godoy DV, Vaccari P, Michelli M. El efecto del ejercicio sobre los niveles de ansiedad, depression y autoconcepto de pacientes com dolencia pulmonar obstructiva crónica. Rev Psicol Deporte. 2001;10(2):267-78.
• 17. Ries A, Kaplan RM, Limberg TM, Prewitt LM. Effects of pulmonary rehabilitation on physiologic and psychological outcomes in patients with chronic obstructive pulmonary disease. Ann Intern Med. 1995;122(11):823-32.
• 18. Emery C, Leatherman N, Burker EJ, MacIntyre NR. Psychological outcomes of a pulmonary rehabilitation program. Chest. 1991;100(3):613-7.
• 19. Kim HF, Kunik ME, Molinari VA, Hillman SL, Lalani S, Orengo CA, et al. Functional impairment in COPD patients: the impact of anxiety and depression. Psychosomatics. 2000;41(6):465-71.
• 20. Kunik ME, Braun U, Stanley MA, Wristers K, Molinari V, Stoebner D, et al. One session cognitive and behavioural therapy for elderly patients with chronic obstructive pulmonary disease. Psychol Med. 2001;31(4):717-23.
• 21. Emery CF, Schein RL, Hauck ER, MacIntyre NR. Psychological and cognitive outcomes of a randomized trial of exercise among patients with chronic obstructive pulmonary disease. Health Psychol. 1998;17(3):232-40.
• 22. Whiters NJ, Rudkin SH, White RJ. Anxiety and depression in severe chronic obstructive pulmonary disease: the effects of pulmonary rehabilitation. J Cardiopulm Rehabil. 1999;19(6):362-5.
• 23. Singh SJ, Smith DL, Hyland ME, Morgan MD. A short outpatient pulmonary rehabilitation programme: immediate and longer term effects on exercise performance and quality of life. Respir Med. 1998;92(9):1146-54.
• 24. Lisboa C, Villafranca AC, Caiozzi AG, Berrocal C, Leiva A, Pinochet R, et al. [Quality of life in patients with chronic pulmonary obstructive disease and the impact of physical training]. Rev Med Chile. 2001;129(4):359-66. Spanish.
• 25. Benzo R, Flume PA, Turner D, Tempest M. Effect of pulmonary rehabilitation on quality in patients with COPD: the use of SF-36 summary scores as outcomes measures. J Cardiopulm Rehabil. 2000;20(4):231-4.
• 26. Lobstein DD, Mosbacher BJ, Ismail AH. Depression as a powerful discriminator between physically active and sedentary middle-aged men. J Psychosom Res. 1983;27(1):69-76.
• 27. Francis KT, Carter R. Psychological characteristics of joggers. J Sports Med Phys Fitness. 1982;22(3):386-91.
• 28. McCann L, Holmes DS. Influence of aerobic exercise on depression. J Pers Soc Psychol. 1984;46(5):1142-7.
• 29. Wykoff W. The psychological effects of exercise on non-clinical population of adult women. Occup Ther Ment Health. 1993;12(3):69-106.
• 30. Bennett J, Carmack M, Gardner VJ. Effect of a program of physical exercise on depression in older adults. Physiol Educ. 1982;39(1):21-4.
• 31. Steptoe A, Edwards S, Moses J, Mathews A. The effects of exercise training on mood and perceived coping ability in anxious adults from the general population. J Psychosom Res. 1989;33(5):537-47.
• 32. Morgan WP, Roberts VA, Brand FR, Feinerman AD. Psychological effects of chronic physical activity. Med Sci Sports. 1970;2(4):213-7.

*

Study carried out at the University of Caxias do Sul (UCS), Caxias do Sul, Rio Grande do Sul, Brazil

Publication Dates

• Publication in this collection
  07 Mar 2006
• Date of issue
  Dec 2005

History

• Accepted
  06 May 2005
• Received
  24 June 2005