SciELO - Scientific Electronic Library Online

 
vol.47 issue2Oral taste recognition in health volunteersInsulin resistance index (HOMA-IR) in the differentiation of patients with non-alcoholic fatty liver disease and healthy individuals author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

  • English (pdf)
  • Article in xml format
  • How to cite this article
  • SciELO Analytics
  • Curriculum ScienTI
  • Automatic translation

Indicators

Related links

Share


Arquivos de Gastroenterologia

Print version ISSN 0004-2803

Arq. Gastroenterol. vol.47 no.2 São Paulo Apr./June 2010

https://doi.org/10.1590/S0004-28032010000200008 

ORIGINAL ARTICLE ARTIGO ORIGINAL

 

Cost-effectiveness analysis on spinal anesthesia versus local anesthesia plus sedation for loop colostomy closure

 

Análise de custo-efetividade da raquianestesia versus anestesia local associada à sedação para o fechamento de colostomia em alça

 

 

Filinto Aníbal Alagia VazI; Rone Antônio Alves AbreuII; Patrícia Coelho de SoárezIII; Manlio Basílio SperanziniI; Luís Cesar FernandesIV; Delcio MatosIV

IDigestive System Surgery Mandaqui, Hospital Complex, São Paulo, SP
IIDepartment of Surgery, Instituto Tocantinense Presidente Antonio Carlos - ITPAC - Araguaína, TO
IIISão Paulo Center for Health Economics, Universidade Federal de São Paulo, Escola Paulista de Medicina - UNIFESP-EPM
IVDiscipline of Gastric Surgery, UNIFESP-EPM, São Paulo, SP, Brazil

Correspondence

 

 


ABSTRACT

CONTEXT: Studies in the area of health economics are still poorly explored and it is known that the cost savings in this area is becoming more necessary, provided that strict criteria.
OBJECTIVE: To perform a cost-effectiveness analysis of spinal anesthesia versus local anesthesia plus sedation for loop colostomy closure.
METHODS: This was a randomized clinical trial with 50 patients undergoing loop colostomy closure either under spinal anesthesia (n = 25) or under local anesthesia plus sedation (n = 25). The duration of the operation, time spent in the post-anesthesia recovery room, pain, postoperative complications, length of hospital stay, laboratory and imaging examinations and need for rehospitalization and reoperation were analyzed. The direct medical costs were analyzed. A decision tree model was constructed. The outcome measures were mean cost and cost per local and systemic postoperative complications avoided. Incremental cost-effectiveness ratios were presented.
RESULTS: Duration of operation: 146 ± 111.5 min. vs 105 ± 23.6 min. (P = 0.012); mean time spent in post-anesthesia recovery room: 145 ± 110.8 min. vs 36.8 ± 34.6 min. (P<0.001). Immediate postoperative pain was lower with local anesthesia plus sedation (P<0.05). Local and systemic complications were fewer with local anesthesia plus sedation (P = 0.209). Hospitalization + rehospitalization: 4.5 ± 4.1 days vs 2.9 ± 2.2 days (P<0.0001); mean spending per patient: R$ 5,038.05 vs 2,665.57 (P<0.001). Incremental cost-effectiveness ratio: R$ -474.78, indicating that the strategy with local anesthesia plus sedation is cost saving.
CONCLUSION: In the present investigation, loop colostomy closure under local anesthesia plus sedation was effective and appeared to be a dominant strategy, compared with the same surgical procedure under spinal anesthesia.

Headings: Colostomy. Anesthesia, local. Anesthesia, spinal. Concious sedation. Costs and cost analysis.


RESUMO

OBJETIVO: Análise de custo-efetividade entre fechamento de colostomia e m alça sob raquianestesia versus anestesia local associada à sedação.
MÉTODOS: Ensaio clínico randomizado com 50 pacientes para fechamento de colostomia em alça, sob raquianestesia (n = 25) e sob anestesia local com sedação (n = 25). Avaliaram-se tempo operatório e de sala de recuperação pós-anestésica, dor, complicações pós-operatórias, tempo de internação, exames laboratoriais e de imagens, reoperações e reinternações. Foi feita análise de custos diretos médicos. A medida de desfecho foi: complicações pós-operatórias locais e sistêmicas. Aplicaram-se razão incremental e árvore de decisão.
RESULTADOS: Tempo operatório (146 ± 111,5 min vs 105 ± 23,6 min; P = 0,012), tempo médio de sala de recuperação pós-anestésica (145 ± 110,8 min vs 36,8 ± 34,6 min, P<0,001). Dor no pós-operatório imediato em favor da anestesia local (P<0,05). Complicações pós-operatórias locais e sistêmicas (P = 0,209) em favor da anestesia local. Internação + reinternações (4,5 ± 4,1 dias vs 2,9 ± 2,2 dias; P<0,0001), valor médio gasto por paciente (R$ 5.038,05 vs R$ 2.665,57; P<0,001). Razão de custo-efetividade: -R$ 474,78, indicando que a estratégia é dominante.
CONCLUSÃO: Na presente investigação o fechamento de colostomia em alça sob anestesia local associada à sedação foi eficaz e apresentou boa relação de custo-efetividade em relação ao mesmo procedimento cirúrgico sob raquianestesia.

Descritores: Colostomia. Anestesia local. Raquianestesia. Sedação consciente. Custos e análises de custo.


 

 

INTRODUCTION

The use of colostomies started to spread from the time of the Second World War onwards. Surgery for colostomy reversal is not free from complications and is performed using a variety of anesthetic techniques(1)

Two studies on loop colostomy closure under local anesthesia plus sedation have been conducted(1, 6). It was concluded that this method is effective and safe, but there was no mention of the costs.

Rationalization of healthcare costs is technically challenging and has the aim of achieving an optimized relationship between financial resources and results(4). Studies on the costs of disease began in the 1960s with the work by Dorothy Rice(21), and these costs were defined as direct, indirect or intangible(18). Direct costs result from interventions; indirect costs relate to productivity losses, mortality or incapacitation; and intangible costs relate to psychological overload(10, 14).

A full economic evaluation analysis in healthcare can use one of the following methodologies: cost-minimization analysis, cost-effectiveness analysis, cost-utility analysis or cost-benefit analysis. Cost-effectiveness analysis has been most commonly used in the evaluation of competing strategies in the healthcare sector. In this, costs and consequences are expressed in costs per unit of health outcome(18).

Based on the results from the studies on loop colostomy closure under local anesthesia plus sedation, and with the lack of cost analysis studies and the increasingly evident importance of such studies, a cost-effectiveness analysis of local anesthesia plus sedation versus spinal anesthesia for loop colostomy closure was proposed.

 

METHODS

A cost-effectiveness analysis was carried out on the data from a randomized clinical trial on 50 patients in whom a loop colostomy was reconstructed either under spinal anesthesia (n = 25) or under local anesthesia plus sedation (n = 25) at the "Mandaqui Hospital" Complex, São Paulo, SP, Brazil between June 2005 and June 2006. The study was coordinated by the post-graduated course of the Discipline of Surgical Gastroenterology of the Department of Surgery, Federal University of Sao Paulo - Escola Paulista de Medicina (UNIFESP-EPM). This study was approved by the Research Ethics Committees of the institutions involved. All the patients read and signed the informed consent statement.

Patients were excluded if they presented hypersensitivity to local anesthetics, psychiatric disorders, obesity (BMI >30 kg/m2), age less than 16 years, paracolostomy hernias, coagulopathy and use of anticoagulants.

The cases were randomized by computer and the breaking of the secrecy of allocation was only done when the patient was already in the operating room. The groups thus formed were considered to be homogenous (Table 1). The patients underwent a preoperative clinical evaluation and all of them underwent colonoscopy, mechanical intestinal preparation and antibiotic prophylaxis.

 

 

The group that underwent local anesthesia received midazolam endovenously (0.03 mg/kg), 10 minutes before the operation. For the local anesthesia plus sedation, 2% lidocaine and 0.5% bupivacaine were used. The surgery was monitored by a single anesthesiologist and the operative technique was standardized. The postoperative follow-up lasted for 30 days.

Study variables

The main objective was to conduct a cost-effectiveness analysis on colostomy closure under spinal anesthesia compared with the same procedure under local anesthesia plus sedation. The presence of local and systemic postoperative complications was selected as the outcome measure for conducting the economic analysis. The following preoperative factors were evaluated: pre-anesthetic sedation, time spent in the operating room, local anesthetic, materials used, conversion or supplementation of the anesthetic technique and pain (analyzed using a visual analog scale, VAS). The following postoperative factors were evaluated: time spent in the post-anesthesia recovery room, analgesics used, solutions for intravenous hydration, reintroduction of diet, local and systemic postoperative complications, length of hospital stay, rehospitalization and postoperative examinations (laboratory and imaging).

Cost analysis

This was performed from the perspective of the healthcare system. The direct medical costs were taken into consideration: costs of hospitalization, operating room, post-anesthesia recovery room, complementary treatment and products. Costs were presented in Reais (Brazilian currency) of 2005. Like in other studies(5, 7, 11, 12, 15, 19, 20, 25), the intangible costs were not included.

Analysis of direct medical costs

Because the preoperative procedures were inherent to both techniques, their costs were not evaluated. A spreadsheet for fund transfers from the public network of the State of São Paulo to hospitals administrated by social healthcare organizations relating to the first quarter of 2005 was used for the cost analysis.

Analysis performed and cost indicators for the study

The cost calculations for the two anesthetic techniques were performed using the macrocosting technique.

Total cost of the colostomies using spinal anesthesia

Total amount spent by the public health system for colostomy closure under spinal anesthesia.

Mean cost per colostomy using spinal anesthesia

Mean amount spent by the public health system for colostomy closure under spinal anesthesia:

Direct (medical) costs ÷ total number of patients treated using spinal anesthesia.

Total cost of the colostomies using local anesthesia plus sedation

Total amount spent by the public health system for colostomy closure under local anesthesia plus sedation.

Mean cost per colostomy using local anesthesia plus sedation:

Mean amount spent by the public health system for colostomy closure under local anesthesia plus sedation:

Direct (medical) costs ÷ total number of patients treated using local anesthesia plus sedation.

Cost-effectiveness analysis

A decision tree was developed using the Excel software (2000 edition; Microsoft) and the TreeAge Pro Suite 2007 (TreeAge Software Inc., Williamstown, MA), to estimate the costs and benefits associated with the two techniques (Figure 1).

The patients' outcomes were listed according to the method used.

The incremental cost-effectiveness ratio (ICER) was calculated as the difference in costs between local anesthesia plus sedation and spinal anesthesia, divided by the difference in health outcomes between the use of local plus sedation and spinal anesthesia. The cost difference was divided by the number of complications avoided.

Statistical analysis

Student's t test(16) and the Mann-Whitney test(22) were applied to the cost analysis on the study data. The statistical significance level for rejection of the nullity hypothesis was set at 0.05 or 5% (α <0.05) and values considered significant were highlighted.

 

RESULTS

While observing the "intervention-to-treat" principle, 25 patients were analyzed in the group that received spinal anesthesia and 25 in the group that received local anesthesia plus sedation. The analysis on the sample, mean duration of the operation, local and systemic intercurrences, intraoperative pain and conversion or supplementation of the anesthetic technique is shown in Tables 1 and 2.

 

 

Both groups had one case of local complication (4.0%), consisting of wound hematoma. Systemic complications occurred in 28.0% (7/25) of the patients in the group using spinal anesthesia: a) four patients (16%) with complications from the anesthetic technique, of whom three had post-puncture headache and one presented urine retention; b) three patients (12%) with complications from the surgical procedure, of whom two evolved with intestinal subocclusion and one presented dehiscence of the anastomosis. There were complications in two patients (8.0%) in the group using local anesthesia plus sedation, who both presented intestinal subocclusion. There was no mortality and no statistical difference (Table 3).

 

 

There was a statistical significant difference with regard to the use of the operating room, post-anesthesia recovery room and ward (Table 4)

The patients who required laboratory tests and imaging examinations were listed and the costs were summed (Table 5).

The total amounts spent using the two techniques are listed in Table 6.

Comparison of the cost-effectiveness of the two anesthetic strategies

Loop colostomy closure under local anesthesia plus sedation was a cost saving strategy. It provided lower costs than the spinal anesthesia technique with better effectiveness (Table 7).

 

DISCUSSION

Patients who were under 16 years old were excluded from this study. Nevertheless, the sample in this study had a relatively low mean age, and this may correlate with the high proportion of cases originating from trauma, i.e. victims of urban violence. The Mann-Whitney test found that the characteristics of the two groups analyzed were homogeneous (Table 1), thus demonstrating the efficiency of the randomization. The stomas were closed late, after 12.5 to 13 months on average, because some patients had been operated in other hospitals

Clinical analysis

The mean duration of surgery with P = 0.034 in favor of the group under local anesthetic may be explained by the fact that the initial infiltration of 50% of the total volume caused intumescence in the tissue, thus favoring dissection. Wong et al.(27) concluded that operations lasting more than 120 minutes would increase the morbidity.

In procedures involving local anesthesia plus sedation, it is important to evaluate the intraoperative pain(2). The levels found among this sample were low in both groups, thus demonstrating that in this respect, the two methods were equivalent and equally effective (Table 2).

Complications were adopted as the primary outcome measurement for incremental ratios and were an important question. Such complications are feared and, without careful analysis when comparing these items, it is impossible to carry out cost analysis in an ethical manner. Although the postoperative complication rate in the group under spinal anesthesia was greater, it was not significantly different (P = 0.209). Laboratory tests and imaging examinations were performed according to the complications presented. Since the group of patients who underwent surgery under spinal anesthesia had greater numbers of complications, they consequently generated a higher demand for such tests. Patients without postoperative complications did not routinely undergo these tests. Although the difference between the groups in this respect did not reach statistical significance (P = 0.284 and P = 0.312), respectively, this caused a difference in costs.

There was no mortality. The results from the present sample can be considered superior to those obtained by Cantele et al.(6), who had 3 cases of dehiscence of anastomoses, 2 of infection of the operative wound and 1 of intestinal obstruction, among 14 cases operated. Abreu et al.(1) observed a complication rate of 9.6%.

The hospital discharge is important in cost analysis. The clinical parameters defined for hospital discharge have to be respected, and this was rigorously followed in the present study. When the rehospitalization data were included, the overall length of hospital stay was higher in the group under spinal anesthesia, to 4.5 days, versus 2.9 days for the group under local anesthesia plus sedation (P<0.001).

Using local anesthesia, Cantele et al.(6) reported that the mean hospital stay after the operation was 9 days, while Abreu et al.(1) observed that it was 4 days.

Economic analysis

Health-related economic analysis for any intervention, whether it is clinical or surgical, only has meaning if this intervention is effective(17, 23). This concept needs to form the basis for all studies of this nature. In the present study, every care was taken to prove the efficacy of the method and the cost-effectiveness economic analysis was proposed from this starting point.

Economics has always had a difficult relationship with the field of healthcare. The latter has concentrated on individualistic ethics, according to which, "health has no price". Despite these mismatches, economics and healthcare have gradually become interlinked, and systematic study and research applied to operational strategies within the healthcare sector have given rise to Health Economics. Fund shortages in this sector have made such studies inevitable.

A randomized clinical trial was the basis for this cost-effectiveness analysis. A study of this nature, with good methodology, is the basis for carrying out cost-effectiveness economic analysis with an impact on the field of health economics. Scientific evidence about the beneficial effect of the technology is essential for economic evaluations of cost-effectiveness type(9, 11).

Drummond et al.(9) identified the key elements and the methodological characteristics that should be found in well-executed economic evaluations. This cost-effectiveness analysis presented most of them.

The present clinical trial contained certain exclusion criteria. Nonetheless, it was judged that these were not so all-embracing as to characterize the study group as a sample that would be unrepresentative of the population.

With regard to effectiveness, it needs to be borne in mind that the clinical trials from which estimates of the benefits from technology are often made are almost always carried out under ideal or near-ideal conditions of use. Thus, cost-effectiveness analysis often measures cost-efficacy(9). This is not a serious problem if the alternatives compared involve technologies or procedures that are relatively simple. There are examples of other studies that have proceeded in this manner, with good results and apt papers(3, 8, 12, 13, 19, 20, 24).

Direct costs result from interventions, and the present study was developed with this focus. Five basic variables were used in relation to economics: the expenditure on the operating room, post-anesthesia recovery room, clinical-surgical ward, laboratory tests and imaging examinations.

To calculate these amounts separately, the basis was the spreadsheet of fund transfers to hospitals within the public network of the State of São Paulo that are administrated by social healthcare organizations, relating to the first quarter of 2005. The arithmetic mean for the 14 establishments included was used.

The costs of laboratory tests and imaging examinations performed after the operation were calculated for both groups. In cases with good evolution, such examinations are not routinely performed. If they are performed, this is an indication that the case is not evolving satisfactorily.

After calculating the expenditure relating to the two anesthetic methods, explanatory tables of absolute values and statistical analyses were produced. At this moment, some idea of possible differences could already be discerned. Since this was a cost-effectiveness analysis, incremental ratios needed to be calculated. These were able to demonstrate the potential cost-effectiveness of the procedure carried out under local anesthesia plus sedation, compared with the same procedure under spinal anesthesia. A decision tree was also developed: this would indicate not only the likelihood of success for each alternative, but also the interventions corresponding to each alternative. The latter would have the potential to change the state of health, thereby triggering new costs(9, 26).

The results achieved in the present study clearly favored the procedure of colostomy closure under local anesthesia plus sedation. It is believed that the main factors leading to this result were the questions of shorter post-anesthesia recovery room use and duration of hospitalization. The first of these can be explained by the fact that the patients operated under local anesthesia stayed in that sector for much shorter times, and some of them were even sent directly to the ward. The post-anesthesia recovery room is a high-cost hospital sector, given all the supplies and equipment that are used there. With regard to the total duration of hospitalization, this is associated with the fact that, after the procedure under local anesthetic, patients are able to start walking again at an earlier stage and intestinal transit is reestablished more rapidly, thereby shortening the length of hospital stay.

Well-designed randomization and breaking the secrecy of allocation at the ideal moment certainly help to reduce the possible bias in the study. The presence of postoperative complications is expected with surgical treatment and cannot be predicted. They could have occurred and did occur in both groups.

With regard to the source for the calculations of the costs relating to the procedures, the table of fund transfers of the government of the State of São Paulo is an official document that guides fund transfers to State hospitals that are managed by social healthcare organizations. This is a current trend for public healthcare establishments. This source of cost calculation was chosen because this study was conducted among patients in public hospitals of the State of São Paulo.

There are few scientific studies in the literature comparing the two anesthetic methods used for this type of procedure(1). Such studies combined with cost analysis are even rarer, and thus the present study has a relatively novel "status".

 

CONCLUSION

Loop colostomy closure under local anesthesia plus sedation was effective and appeared to be a dominant strategy when compared with the same surgical procedure under spinal anesthesia.

The importance of health economics is evident. Unlike developed nations, which have been using it for some time, in this country, this is still just beginning.

This study opens up the prospect of further coverage of this topic, complementing it with data on the impact of this treatment.

 

REFERENCES

1. Abreu RAA, Speranzini MB, Fernandes LC, Matos D. Avaliação da praticabilidade do fechamento de colostomias em alça sob anestesia local associada à sedação. Acta Cir Bras. 2006;21:275-8.         [ Links ]

2. Aldrete JA. The post-anesthesia recovery score revisited. J. Clin Anesth. 1995;7:89-91.         [ Links ]

3. Andujar PR, Canário M, Grulloón G. Herniorrafia inguinal ambulatoria bajo anestesia local versus herniorrafia em admisión bajo anestesia regional. Acta Méd Domin. 1992;14:84-9.         [ Links ]

4. Araújo DJ. As peculariedades do mercado dos serviços de saúde. Rev Adm Pública. 1977;229-38.         [ Links ]

5. Callesen T, Beck K, Kehlet H. The feasibility, safety and cost of infiltration anaesthesia for hernia repair. Anaesthesia. 1998;53:31-5.         [ Links ]

6. Cantele H, Méndez A, Leyba J. Colostomy closure using local anesthesia. Surg Today. 2001;31:678-80.         [ Links ]

7. Cothren CC, Moore EE, Johnson JL, Moore JB, Ciesla DJ, Burch JM. Can we afford to do laparoscopic appendectomy in an academic hospital? Am J Surg. 2005;190:950-4.         [ Links ]

8. Craig MS, Tseng MD. Cost-effectiveness of gastric bypass for severe obesity. Am J Med. 2002;113:491-8.         [ Links ]

9. Drummond MF, O'Brien BJ, Stoddart GI, Sculpher MJ, Torrance GW. Methods for the economic evaluation of helth care programmes. 2nd ed. Oxford: University Press; 1997. p.13.         [ Links ]

10. Ferraz MB. Como aproveitar os recursos disponíveis. Cremesp; 1996. [Disponível em: www.cpes.org.br/como.pdf]         [ Links ].

11. Finkelstein EA, Brown DR. A cost-benefit simulation model of coverage for bariatric surgery among full-time employees. Am J Manag Care. 2005;11:641-6.         [ Links ]

12. Forssblad M, Jacobson E, Weidenhielm L. Knee arthroscopy with different anesthesia methods: a comparison of efficacy and cost. Knee Surg Sports Traumatol Arthrosc. 2004;12:344-9.         [ Links ]

13. Henriques AC, Horta SH, Pezzolo S, Waisberg J, Boratto SF, Helal S, Gomes M, Speranzini MB. Procedimentos cirúrgicos em pacientes proctológicos selecionados sob anestesia local. Estudo de 150 casos. Arq Gastroenterol. 2000;37:158-61.         [ Links ]

14. Maetzel A. Costs of illness and the burden of disease [editorial]. J. Rheumatol. 1997;24:3-5.         [ Links ]

15. Martin II RCG, Vitale CG, Reed ND, Larson GM, Edwards MJ, McMasters KM. Cost comparison of endoscopic stenting vs surgical treatment for unresectable cholangiocarcinoma. Surg Endosc. 2002;16:667-70.         [ Links ]

16. Neter J, Kutuer MH, Naclitseim CJ, Wasserman W. Applied linear statistic models. 4th.ed. Boston: Irwin; 1996.         [ Links ]

17. Oliveira MA, Muniz MT, Santos LA, Faresin SM, Fernandes ALG. Custo-efetividade de programa de educação para adultos asmáticos atendidos em hospital escola de instituição pública. J Pneumol. 2002;28:71-6.         [ Links ]

18. Pereira J, Mateus C, Amaral J.M. Custos da obesidade em Portugal. Revista Associação Portuguesa Economia da Saúde. 1999;11:66-105.         [ Links ]

19. Quirk DM, Rattner DW, Del Castilho CF, Warshaw AL, Brugge WR. The use of endoscopic ultrasonography to reduce the cost of treating ampullary tumors. Gastrointest Endosc. 1997;46:334-7.         [ Links ]

20. Ramos JR, Pinho M, Valory E, Nascimento ML, Sousa PC. Cirurgia ambulatorial em colo-proctologia. Rev Bras Coloproctol. 1988;8:11-3.         [ Links ]

21. Rice, DP. Estimating the cost of illness. Am J Public Health Nations Health. 1967;57:424-40.         [ Links ]

22. Siegel S. Estatística não-paramétrica (para as ciências do comportamento). São Paulo: McGraw-Hill; 1975.         [ Links ]

23. Smith DJ. Organizational theory and the hospital. J Nurs Adm. 1997;4:103.         [ Links ]

24. Sobrado CW, Nahas SC, Marques CF, Gama AH. Cirurgia ambulatorial sob anestesia local em proctologia: experiência e análise do resultado de 503 operações. Rev Bras Coloproctol. 2001;21:228-33.         [ Links ]

25. Stocche RM, Garcia LV, Klamt JG, dos Reis MP, Gil DR, Mesquita KLM. Influence of intravenous clonidine in the cost of sevoflurane anesthesia for outpatient middle ear procedures. Rev Bras Anestesiol. 2004;54:91-8.         [ Links ]

26. Torrance GW, Siegel JE, Luce BR. Framing and designing the cost-effectiveness analysis. In: Gold MR, Siegel JE, Russel LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996. p.54-81.         [ Links ]

27. Wong RW, Rappaport WD, Witzke DB, Putnam CW, Hunter GC. Factors influencing the safety colostomy closure in the elderly. J Surg Research. 1994;57:289-92.         [ Links ]

 

 

Correspondence:
Dr. Rone Antônio Alves de Abreu
Rua Cel. Fleury, 1930 - Setor Carajás
77800-000 - Araguaína, TO, Brazil
E-mail: roneabreu@uol.com.br

Received 29/3/2009.
Accepted 14/10/2009.

 

 

Funding: Heath Department of the State of São Paulo, SP, Brazil

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License