Surgery cancelling at a teaching hospital: implications for cost management

Abstracts

Este estudo discute a problemática do cancelamento de cirurgias sob a perspectiva econômico-financeira. Foi realizado na Unidade de Centro Cirúrgico de um hospital de ensino, com o objetivo de identificar e analisar os custos diretos (recursos humanos, medicamentos e materiais) e custo de oportunidade gerados pelo cancelamento de cirurgias eletivas. Os dados foram coletados durantes três meses consecutivos, utilizando-se documentos institucionais e formulário elaborado pelas pesquisadoras. Apenas 58 (23,3%) das 249 cirurgias programadas canceladas representaram custos para a instituição. O custo direto total dos cancelamentos foi de R$ 1.713,66 (custo médio por paciente de R$ 29,54), assim distribuídos: despesas com materiais de consumo R$ 333,05; processo de esterilização R$ 201,22; medicamentos R$ 149,77 e recursos humanos R$ 1.029,62. O custo com recursos humanos representou o maior percentual em relação ao custo total (60,1%). Observou-se que a maior parte dos cancelamentos eram potencialmente evitáveis. Planejamento administrativo, redesenho dos processos de trabalho, medidas educativas de pessoal e avaliação clínica prévia constitue em estratégias recomendadas para minimização da ocorrência.

hospitais de ensino; enfermagem; custos hospitalares; custos diretos de serviços

Este artículo discute la problemática del cancelamiento de cirugías bajo una perspectiva económico-financiera. Fue llevado a cabo en la Unidad del Centro Quirúrgico de un hospital-escuela con objeto de identificar y analizar los costos directos (recursos humanos, recursos materiales y medicamentos) e indirectos ocasionados por el cancelamiento de cirugías no urgentes. Los datos fueron recogidos durante tres meses consecutivos mediante documentos institucionales y un cuestionario elaborado por las investigadoras. Solamente 58 (23,3%) de las 249 operaciones previstas y que fueron canceladas resultaron en costos para la institución. El costo directo total de los cancelamientos fue R$ 1.713,66 (costo medio por paciente de R$ 29,54), repartidos así: gastos con materiales de consumo R$ 333,05 y proceso de esterilización R$ 201,22, medicamentos R$ 149,77 y recursos humanos R$ 1.029,62. El costo de los recursos humanos repesentó el mayor porcentaje en relación al costo total (60,1 %). Se constató que la mayor parte de los cancelamientos podría haber sido evitada. Planificación administrativa, rediseño de los procesos de trabajo, medidas educativas del personal y evaluaçión clínica previa constituyen estrategias recomendadas para reducción de los casos de cancelamiento.

hospitales escuela; enfermería; costos hospitalares; costos directos de servicios

ORIGINAL ARTICLE

Surgery cancelling at a teaching hospital: implications for cost management¹

Márcia Galan Perroca^I; Marli de Carvalho Jericó^II; Solange Diná Facundin^III

^INurse, PhD in Nursing, Faculty

^IINurse, Doctoral student, Faculty, e-mail: [e-mail]

^IIINurse, Faculty. Medical School of São José do Rio Preto, Brazil

ABSTRACT

This study discusses the problem of surgery cancellation on the economic-financial perspective. It was carried out in the Surgical Center Unit of a school hospital with the objective to identify and analyze the direct costs (human resources, medications and materials) and the opportunity costs that result from the cancellation of elective surgeries. Data were collected during three consecutive months through institutional documents and a form elaborated by the researchers. Only 58 (23.3%) of the 249 cancelled scheduled surgeries represented costs for the institution. The cancellations direct total cost was R$ 1.713.66 (average cost per patient R$ 29.54); distributed as follows: expenses with consumption materials R$ 333.05; sterilization process R$201.22; medications R$149.77 and human resources R$1,029.62. The human resources costs represented the greatest percentile in relation to the total cost (60.40%). It was observed that most of the cancellations could be partially avoided. Planning on management; redesigning work processes, training the staff and making early clinical evaluation can be strategies to minimize this occurrence.

Descriptors: hospitals teaching/organization and administration; nursing; hospital costs; direct service costs

INTRODUCTION

The cancelling of scheduled surgical procedures has been studied not only in Brazil^(1-2), but also in other countries like Australia⁽³⁾, Ireland⁽⁴⁾, Mexico⁽⁵⁾, the United States⁽⁶⁾ and the United Kingdom⁽⁷⁾ . Like Brazilian research, these studies have indicated high frequency of cancelling, due to organizational problems in health institutions, including lack of beds^(3-4,7), scheduling errors, communication errors^(3,5) and other administrative problems.

Although several authors acknowledge that the impact of surgery cancelling raises the operational and financial costs of the Surgical Center Unit and reduces the efficiency of the service, few studies have analyzed the theme from the economic-financial aspect, especially in terms of direct costs (human resources, medication and material) and cost of opportunity. As the surgical movement has been appointed as an intervenient factor in productivity and hospital quality rates⁽⁸⁾, the maximum use of surgical capacity is one of the main measures for the efficiency of financial resource use. A study performed at a teaching hospital⁽⁹⁾ demonstrated that surgical patients represent 24% of total hospitalizations and contribute to 43% of revenues.

Surgery cancelling is an error resulting from non compliance with the unit's administrative planning requirements. It corresponds to one of the four components of quality cost, classified in costs of internal and external errors, costs of prevention and analysis. To achieve excellence, the hospital must continually be committed to the problem solving capacity, quality and low costs of medical procedures. Therefore, the elimination of waste is needed, as well as the ability to improve the hospital process (diagnosis, treatment, hospitalization, management support) through adequate automation, more information and decrease of the patient's hospital stay⁽¹⁰⁾.

Waste can be considered as any and all resources used, beyond the necessary, to execute a product or service (consumption material, supplies, human effort, energy, technology, among others). It is extra expenditure added to the product or service's regular costs without any kind of improvement to the client⁽¹¹⁾. When work processes are inadequate, the cost of products or services increases. Consequently, the institutions incur financial losses due to rework, absorbing the time that would be used for another activity.

The repercussions of surgery cancelling negatively affect not only the client, who experiences a broken bond of trust in relation to the institution, but also the nursing team (work operation, time and material resource consumption, diminished care quality) and the hospital itself⁽¹⁾. The cancelling of the surgical procedure increases operational and financial costs, causing losses to the institution. The financial loss is caused by the deficient process and can be evidenced by the reservation of the surgery room and loss of opportunity to schedule another patient, underused surgery rooms, longer hospitalizations (and risk of hospital infection) and, consequently, increase in the bed price/day and diminished bed availability. Other sources of loss are the waste of sterilized material, rework of the personnel involved in the preparation of the surgery room and in the sterilization process⁽²⁾.

The cost of opportunity can be defined as the value of a resource in its best alternative use⁽¹²⁾. It represents the value that is not gained due to the decision of investing the resource in another alternative, to the detriment of others. Different parameters have been used to address cost of opportunity, such as the measurement of profit, income, among others. Since each decision involves a different cost of opportunity, the concept can be associated with different attributes⁽¹³⁾. The implementation of the concept guides the manager's decision in the use of a certain resource during the whole decision process, i.e. in the phase preceding the decision, as an element for evaluating the performance of the manager responsible for the action and also to evaluate the results of the decision, after its implementation⁽¹⁴⁾. The focus on economic measurement by cost of opportunity is a relevant instrument of feedback for planning and control⁽¹⁵⁾. Thus, this study aims to support nursing in the decision making process based on quality and opportunity cost information.

OBJECTIVES

- Identify and analyze the distribution of direct costs related to human resources, consumed supplies (consumption and reprocessed materials) and medications, generated by the cancelling of elective surgeries at the Surgical Center Unit of a teaching hospital.

- Verify the direct cost related to the time spent by different professional categories.

- Identify the cost of opportunity at the surgical center.

METHODOLOGY

This is an exploratory and descriptive study, carried out at the Surgical Center of a Teaching Hospital of extra capacity in a city in the interior of São Paulo, Brazil. This hospital is a referral center and delivers hospital and outpatient care in several medical specialties, totaling an average of 2,500 hospitalizations/mo. and 1,600 large, medium and small-size surgeries/mo. The surgical program covers the period from 7am to 7pm, daily, from Monday to Friday and on Saturday mornings. Night periods, weekends and holidays are reserved for emergencies. The study population is composed of all elective surgeries in the period from September to November 2004 at the study hospital, cancelled after the preparation of the surgery room (SR) and during the surgery itself. This criterion was used because these types of cancelling imply costs (considered in this study).

The map with the monthly surgical schedule was used to verify the occurrence of surgery cancelling. For the identification of causes, a structured form was elaborated, containing 4 groups of data: 1 - Demographic Characteristics (age, gender, health insurance, surgery room and time of surgery); 2 - Circumstance in which the surgeries were cancelled (before and after the preparation of the surgery room and during the surgical procedure); 3 - Professional Categories involved in the arrangement of the SR, time spent and activities developed; 4 - Material and equipment used in the arrangement of the SR which were not included in the debt bill of the Surgical Center Unit (SCU), such as clothing (sheets, gowns) and instruments.

Data collection started after formal authorization was obtained from the institution and from the head nurse at the SCU, and after orientation of the nurses who worked in the morning and afternoon shifts about how to fill out the instrument to be used. Approval by the Research Ethics Committee was not necessary because this research did not involve human beings. The researchers collected the forms every day and data were complemented.

An electronic worksheet was elaborated in Microsoft Excel for data treatment. This contained information on the client's identification, health plan, hospitalization unit, surgery, date and reason for cancelling. The sheet was divided in four parts: materials, fees, medication and human resources. The cost identification was based on the BRASINDICE, factory price from April 2005 and AMB table (Brazilian Medical Association). Data regarding salaries were provided by the Personnel Department. To survey costs related to the time spent on human resources, the minimum remuneration of the professional categories involved and social charges (FGTS, PIS, vacation and 13^th salary at the proportion of 1/12) were taken into account. The taxes totaled 21.67% of added cost. To estimate the cost of opportunity, the surgeries were classified according to anesthetic size and the SR and Post Anesthesia Care Unit (PACU) fees from the AMB table were used.

RESULTS

In total, 249 scheduled surgeries were cancelled at the Surgical Center unit during the study period. From these, 191 (76.7%) occurred before the preparation and arrangement of the surgery room, 54 (21.7%) after its preparation and only four of them (1.6%) during the anesthetic-surgical procedure. Cancellations before the SR preparation were excluded from the analysis since they did not represent additional costs. Thus, the sample was composed of the 58 (23.3%) remaining surgeries.

Among the patients who had their surgeries cancelled, 30 were men and 28 women, with an average age of 43.1 ± 24.2 years (ranging from 6 months to 80 years old). Most patients (82.7%) used the Single Health System (SUS). As can be observed in Table 1, the most frequent medical specialties in the cancelled surgical procedures with costs for the institution were: general surgery (20.7%), followed by otolaryngological surgery (19%) and orthopedic surgery (13.8%). The main causes for cancelling were related to the patient (53.4%), such as unfavorable clinical condition (arterial hypertension and respiratory diseases, among others) - 29.3% and absence (17.2%). Problems related to the unit's organization (24.2%) also caused a significant share of cancelling, mainly because of the lack of beds at the Intensive Care Unit (13.8%). Lack of documentation and exams, emergency surgeries and contamination of the SR were responsible for a smaller part of cancellations, respectively, 3.5%, 3.5% and 1.7%. In only 8 of the 58 surgeries that were cancelled (13.8%), the main causes of cancelling were related to the allocation of human resources, such as, extended surgical time (5.1%), changes in medical conduct (3.5%) and lack of surgical team professionals (anesthesiologist and surgeon) - 3.5%. The causes related to the allocation of human resources and equipment represented only 8.6% of the total causes of cancelling.

The direct costs related to human resources and supplies (medication, consumption and reprocessed material) totaled R$ 1,713.66 (average cost per patient R$ 29.54), while R$ 1,169.08 (68.2%) were related to cancellations during the preparation of the surgery room and R$ 544.58 (31.8%) during the surgical procedure (Table 2).

Nursing (nursing technicians and aids) was the category that totaled the highest number of minutes worked (2,255) corresponding to 46.7% of the work load during the preparation of the SR and 54.9% during the anesthetic-surgical procedure. The time of this category was distributed among the functions of the scout nurse, help to the anesthesiologist and surgical instrumentation. At the study institution, the nursing professional has a management function and is responsible for supervising the surgery rooms. Because it is an indirect cost, this professional's time was not considered. The findings showed that the scout nurse spends more time working both in the preparation of the SR (30.5%) and during the surgical procedure (24.1%). The highest cost for the institution is related to the surgeon (R$ 446.40) and the anesthesiologist (R$ 286.75) (Table 3).

Due to the high number of items among the different materials, in this study, the three most representative items of each material were selected. The most used consumption materials that caused higher spending for the institution during SR preparation were medullary puncture needles, malleable peripheral venous puncture catheter and electrode; during the surgical procedure, the materials were medullary puncture needles, surgical compresses and malleable peripheral venous puncture catheters. In terms of medication, the materials were: anesthetic eye drops, lidocaine, isobaric bupivacaine and oxygen (SR preparation phase) and propofol, lidocaine and suxamethonium (during the surgical procedure).

The material reprocessed by the Sterile Processing Department (SPD) used in the cancelled surgeries was classified in instruments and surgery clothing. The total cost of instrumental reprocessing was R$59.32 (the most expensive items were the small surgery box, septoplasty box and glossectomy). Surgery clothing reprocessing cost amounted to R$ 141.90 for the institution; the general laparotomy package incurred in a higher cost. To survey SR and PACU costs, the cancelled surgeries were classified according to anesthetic size, which varies from 0 to 7. The most frequent surgeries were of size 2. However, surgeries that incurred a higher cost of opportunity were of size 5, responsible for 22.1% (R$ 2,388.08), followed by size 2 with 19% (R$ 2,050.80) and smaller size (size 0) with 0.7% (R$ 78.96). The cost of opportunity at the Surgical Center Unit totaled R$ 10,782.40.

DISCUSSION

Fifty-eight of all surgeries that were cancelled entailed costs. The main causes were related to the patient (53.4%), such as unfavorable clinical condition (hypertension and respiratory diseases among others) - 29.3% and absence (17.2%). Studies performed in Mexico⁽⁵⁾ and Australia⁽³⁾ also showed alterations in patients' clinical conditions as sources of costs, with 40% and 17.1% respectively. Pre-surgery clinical assessment is appointed as an important factor to be considered for reducing the rates of cancelled surgical procedures^(6,16).

Regarding the patient's absence from the scheduled surgery, research in teaching hospitals have indicated values of 41% (cataract surgery)⁽¹⁷⁾ and 54.3% (several specialties)⁽¹⁾, higher than what was found in this study (17.2%). The reasons for the patient's absence have been associated to the institutional condition (unawareness and alteration of the date of surgery, hospitalization difficulty, lack of vacancy and lack of pre-surgical exam); to the clinical condition, social condition (occupational problems) and also personal condition⁽¹⁾.

Surgery cancelling due to organizational problems at the unit (24.2%) is mainly related to the lack of beds at the Intensive Care Unit and also for hospitalization (13.8%). In a study performed in Ireland⁽⁴⁾, the lack of beds represented 31% of surgery cancellations, against 18.1% in Australia⁽³⁾. The findings of this research revealed that cancellations due to human resource factors represented only 13.8% of the total, in which extended surgery time predominated (5.1%). Values of 18.7% for previous surgeries that exceeded estimated time have been reported in literature⁽³⁾, as well as the observation that surgeons who frequently underestimate the time necessary for the surgery have also presented a significantly higher number of cancellations than those who do not.

Measures to minimize cancellations are an important attribution for the head nurse at the surgical center. It is estimated that around 60% of elective procedure cancellations is potentially avoidable and could be prevented by using quality improvement techniques. The cancellations should be seen as adverse events and be routinely monitored in the clinical systems of hospital incidents, since they are the main cause of inefficient time use at the SR and waste of resources⁽³⁾. These authors consider the following types of cancellation as potentially avoidable: previous surgeries that exceeded the estimated time, scheduling errors, administrative causes, equipment and transportation problems, poor communication, errors in adequate patient preparation and non-available surgeon.

The United Kingdom's experience⁽⁷⁾ in using auditing as an intervention process has presented promising results. The first audit found a surgery cancellation rate of 16.1%, while the reasons related to the hospital (mainly lack of beds) represented 42% of total cancellations, clinical reasons 34% (especially lack of anesthetic and surgical conditions) and patient-related cancellations 21% (absence). The second audit was performed 15 months after the measures were implemented (improvement in bed distribution, clinical assessment before scheduling and improvement in communication with the patient by discussing the surgical date and his(er) convenience and pre-surgical orientation). The cancellation rate was reduced by 42.9%.

The results of this study showed that surgery cancelling presented a total direct cost of R$ 1,713.66, a small amount if we consider the size of the study institution. It also revealed that the cost of human resources represented the highest percentage of total cost (60.1%). The direct cost generated by the cancelling of elective surgeries indicated that errors occurred in the internal environment of the Surgical Center Unit and that, consequently, the administrative planning requirements were not met. It is a loss for the hospital, since internal errors do not result in procedures that will eventually generate revenues.

Since most of the cancellations were potentially avoidable, these findings evidence concrete possibilities of reducing the level of surgical cancellations by analyzing causes that generate the problem. We argue that changing only one factor, the main reason for the cancelling, probably will not exert a positive effect if other factors are not changed simultaneously⁽³⁾. For the hospital to reduce the rate of cancellations, each problem needs to be solved in the process, starting with surgery scheduling, effective distribution SCU and notification to the patient. In the quality approach, the costs must be composed mainly by preventive costs (emphasizing the education of several categories with respect to unnecessary expenditures) to the detriment of evaluation or correction costs⁽¹⁸⁾.

The economic amount that composes the cost of opportunity of the cancelled surgeries under study was R$ 10,782.40. The difference between the SR usage capacity and its effective use determines the economic value that could be obtained and represents the relative economic loss in relation to the opportunity cost of not using the surgery room. This cost is related to management inefficiency, because it is an economic consumption and identifies an inefficacy, since income that would contribute to the result if the surgery had been performed was not generated.

Considering that economic profit is obtained by the income minus all costs involved in its achievement, the measurement of the results, for the sake of management performance assessment, should take into account the cost of opportunity⁽¹⁴⁾. However, it is observed that it is difficult to perceive this cost in the study institution for two reasons. The first is related to a high demand for surgeries, with a consequent waiting list. On the one hand, this prevents the inefficient process from being reflected in the idle capacity of the SR and PACU and, on the other side, it hinders the judgment of what is a right or wrong decision due to the expected result. The second is related to the common habit of letting patients with non-scheduled surgeries, fast for a potential surgery.

Thus, the institutional structure transfers the inefficiency of the process to the patient (usually an SUS user), who ends up carrying the relative costs of the surgery cancellation (waiting time, fasting, emotional costs). Patients with other health insurances than the Single Health System (SUS), however, are not willing to deal with poorly elaborated processes or inefficient managers. These characteristics of the institution, such as high demand for surgeries and keeping non-scheduled patients fasting can go unnoticed and uncorrected for a long time because they have been accepted as a natural part of daily work.

The implementation of the opportunity cost concept in measuring the cost of a good is the method that best reflects a manager's efficacy in the management of the resources used⁽¹⁵⁾. The focus on economic measurement by opportunity cost is a relevant instrument of feedback for planning and control⁽¹⁶⁾. Measurement is the first stage that leads to control and, eventually, to the improvement of a process⁽¹⁹⁾.

CONCLUSION

This study highlighted the microeconomic aspects of surgery cancelling, including direct costs and cost of opportunity, and allowed for a situational diagnosis of the Surgical Center Unit. Based on this information, nurses can use strategies in their management work to minimize this occurrence. Administrative planning, redesigned work processes, educative measures and previous clinical assessment are recommended strategies.

Although the results reflect the experience of a public teaching hospital and, therefore, may not be representative of other hospitals, the authors hope that these findings can help hospitals in the development of strategies to reduce the cancelling of scheduled surgeries and its impact on cost management.

Recebido em: 21.2.2006

Aprovado em: 14.6.2007

• 1. Paschoal MLH. Taxa de suspensão de cirurgia em um hospital universitário e os motivos de absenteísmo do paciente à cirurgia programada. [dissertação]. São Paulo (SP): Escola de Enfermagem/USP; 2002.
• 2. Cavalcanti JB, Pagliuca LMF, Almeida PC. Cancelamento de cirurgias programadas em um hospital-escola: um estudo exploratório. Rev Latino-am Enfermagem 2000 julho-agosto; 8(4):59-65.
• 3. Schofield WN, Rubin GL, Piza M, Lai YY, Sindhusake D, Fearnside MR, Klineberg PL. Cancellatoin of operations on the day of intended surgery at a major Australian referral hospital. Med J Aust 2005 June 20; 182(12):612-5.
• 4. Robb WB, O'Sullivan MJ, Brannigan AE, Bouchier-Hayes DJ. Are elective surgical operations cancelled due to increasing medical admissions? Ir J Med Sci 2004 July-September; 173(3):129-32.
• 5. Aguirre-Cordova JF, Chavez-Vazquez G, Huitron-Aguillar GA, Cortes-Jimenez N. Why is surgery cancelled? Causes, implications, and bibliographic antecedents. Gac Med Mex 2003 November-December; 139(6):545-51.
• 6. Tait AR, Voepel-Lewis T, Munro HM, Gutstein HB, Reynolds PI. Cancellation of pediatric surgery: economic and emotional implications for patients and their family. J Clin Anesth 1997 May; 9(3):213-9.
• 7. Abdellaoui A, Addison A. A study of cancelled operations in an orthopaedics department. Clin Gov Bull 2005 March; 5(6):6-9.
• 8. Gatto MAF, Jouclas VMG. Otimizando o uso da SO. Rev SOBECC 1998 janeiro-março; 3(1):23-8.
• 9. Silva SH da. Controle da qualidade assistencial: implementação de um modelo. [tese]. São Paulo (SP): Escola de Enfermagem/USP; 1994.
• 10. Robles A Junior. Custos da qualidade: uma estratégia para a competição global. São Paulo (SP): Atlas; 1994.
• 11. Souza D de LE. CCQ - Fazendo acontecer. Belo Horizonte: Christiano Ottoni/Escola de Engenharia/UFMG; 1996.
• 12. Pereira GBS, Baraúna MLPS de. Custo de oportunidade sob o enfoque do modelo gestão econômica. In: 9ş Congresso Brasileiro de Custos; 2002. Outubro 13-15; São Paulo, São Paulo. [citado 2004 setembro 21]. Disponível em: URL: http://www.abccustos.org.br/texto/viewpublic?ID_TEXTO= 1443
• 13. Silva AS, Reis EA dos, Leão LCG. Custo de oportunidade. In: 4ş Congresso Brasileiro de Custos; 1997. Novembro-Dezembro 28-01; Belo Horizonte, Minas Gerais. [citado 2004 setembro 21]. Disponível em: URL: http://www.abccustos.org.br/texto/viewpublic?ID_TEXTO= 205
• 14. Nascimento AM, Souza MA de. Custos de oportunidade: evolução e mensuração. In: 10ş Congresso Brasileiro de Custos; 2003. Outubro 15-17; Guarapari, Espírito Santo. [citado 2004 setembro 21]. Disponível em: URL: http://www.abccustos.org.br/texto/viewpublic?ID_TEXTO= 1729
• 15. Catelli A. Custos de oportunidade na gestão da cadeia de valor. In: 9ş Congresso Brasileiro de Custos; 2002. Outubro 13-15; São Paulo, São Paulo. [citado 2004 setembro 21]. Disponível em: URL: http://www.abccustos.org.br/texto/viewpublic?ID_TEXTO= 1421
• 16. Arieta CEL, Taiar A, Kara-José N. Utilização e causas de suspensão de intervenções cirúrgicas oculares em Centro Cirúrgico ambulatorial universitário. Rev Assoc Med Bras 1995; 41(3): 233-5.
• 17. Lira RPC, Nascimento MA, Temporini ER, Kara-José N, Arieta CEL. Suspensão de cirurgias de catarata e suas causas. Rev Saúde Pública 2001; 35(5): 487-9.
• 18. Pinho RCS, Pessoa MNM, Peter MGA, Cochrane TMC, Peter FA. Custos da qualidade na atividade de auditoria. In: 10ş Congresso Brasileiro de Custos; 2003. Outubro 15-17; Guarapari, Espírito Santo. [citado 2004 setembro 21]. Disponível em: URL: http://www.abccustos.org.br/texto/viewpublic?ID_TEXTO= 1683
• 19. Maia JRC, Nascimento M do, Kielwagen KE, Costa FA. A gestão dos custos da qualidade otimizando o processo de garantia da qualidade. In: 8ş Congresso Brasileiro de Custos; 2003. Outubro 3-5; São Leopoldo, Rio Grande do Sul. [citado 2004 setembro 21]. Disponível em: URL: http://www.abccustos.org.br/texto/viewpublic?ID_TEXTO= 2109

1

Extracted from Research Project

Publication Dates

• Publication in this collection
  17 Dec 2007
• Date of issue
  Oct 2007

History

• Accepted
  14 June 2007
• Received
  21 Feb 2006