Acessibilidade / Reportar erro

Nursing outcomes content validation according to Nursing Outcomes Classification (NOC) for clinical, surgical and critical patients

Abstracts

The objective of this study was to validate the Nursing Outcomes (NO) from the Nursing Outcomes Classification (NOC) for the two Nursing Diagnoses (ND) most frequent in hospitalized surgical, clinical and critical patients. The content validation of the REs was performed adapting the Fehring Model. The sample consisted of 12 expert nurses. The instrument for data collection consisted of the NOs proposed by NOC for the two NDs in the study, its definition and a five-point Likert scale. The data were analyzed using descriptive statistics. The NOs that obtained averages of 0.80 or higher were validated. The ND Risk for Infection was the most frequent, being validated eight (38.1%) of 21 NOs proposed by the NOC. The ND Self-Care Deficit: Bathing/Hygiene was the second most frequent and five (14.28%) out of 35 NOs were validated.

Nursing Diagnosis; Validation Studies; Nursing; Nursing Process; Nursing Assessment


Este estudo teve como objetivo validar os Resultados de Enfermagem (RE) da Classificação dos Resultados de Enfermagem (NOC) para os dois Diagnósticos de Enfermagem (DE) frequentemente identificados em pacientes internados em unidades clínicas, cirúrgicas e de terapia intensiva. A validação de conteúdo dos REs foi realizada adaptando-se o modelo de Fehring. A amostra constitui-se de 12 enfermeiros peritos. O instrumento de coleta de dados, na segunda etapa, se constituiu dos REs, propostos pela NOC, para os dois DEs em estudo, sua definição e uma escala Likert de cinco pontos, para os peritos pontuarem. Os dados foram analisados por estatística descritiva. Foram, ainda, validados os REs que obtiveram médias iguais ou maiores que 0,80. Risco de infecção foi o DE mais frequente, sendo validados oito (38,1%) dos 21 REs propostos pela NOC. Déficit no autocuidado: banho/higiene foi o segundo DE mais frequente e cinco (14,28%) dos 35 REs foram validados.

Diagnóstico de Enfermagem; Estudos de Validação; Enfermagem; Processos de Enfermagem; Avaliação em Enfermagem


Este estudio tuvo como objetivo validar los Resultados de Enfermería (RE) de la Clasificación de los Resultados de Enfermería (NOC) para los dos Diagnósticos de Enfermería (DE) frecuentemente identificados en pacientes internados en unidades clínicas, quirúrgicas y de terapia intensiva. La validación de contenido de los REs fue realizada adaptando el modelo de Fehring. La muestra fue constituida por 12 enfermeros expertos. El instrumento de recolección de datos en la segunda etapa se constituyó de los REs propuestos por la NOC para los dos DEs en estudio, su definición y una escala Likert de cinco puntos para que los expertos evaluaran. Los datos fueron analizados por estadística descriptiva. Fueron validados los REs que obtuvieron promedios iguales o mayores a 0,80. Riesgo de Infección fue el DE más frecuente, siendo validados ocho (38,1%) de los 21 REs propuestos por la NOC. Déficit en el Autocuidado: Baño/Higiene fue el segundo DE más frecuente y cinco (14,28%) de los 35 REs fueron validados.

Diagnóstico de Enfermería; Estudios de Validación; Enfermería; Procesos de Enfermería; Evaluación en Enfermería


ORIGINAL ARTICLES

Nursing outcomes content validation according to Nursing Outcomes Classification (NOC) for clinical, surgical and critical patients1 Corresponding Author: Miriam de Abreu Almeida Universidade Federal do Rio Grande do Sul Escola de Enfermagem Departamento de Enfermagem Médico-Cirúrgica Rua São Manoel, 963 Bairro Rio Branco CEP: 90620-110 Porto Alegre, RS, Brasil E-mail: miriam.abreu2@gmail.com.br

Deborah Hein SeganfredoI; Miriam de Abreu AlmeidaII

IRN, M.Sc. in Nursing, Escola de Enfermagem, Universidade Federal do Rio Grande do Sul, RS, Brazil. E-mail: debhseg@gmail.com

IIRN, Ph.D. in Education, Adjunct Professor, Escola de Enfermagem, Universidade Federal do Rio Grande do Sul, RS, Brazil. E-mail: miriam.abreu2@gmail.com

Corresponding Author Corresponding Author: Miriam de Abreu Almeida Universidade Federal do Rio Grande do Sul Escola de Enfermagem Departamento de Enfermagem Médico-Cirúrgica Rua São Manoel, 963 Bairro Rio Branco CEP: 90620-110 Porto Alegre, RS, Brasil E-mail: miriam.abreu2@gmail.com.br

ABSTRACT

The objective of this study was to validate the Nursing Outcomes (NO) from the Nursing Outcomes Classification (NOC) for the two Nursing Diagnoses (ND) most frequent in hospitalized surgical, clinical and critical patients. The content validation of the REs was performed adapting the Fehring Model. The sample consisted of 12 expert nurses. The instrument for data collection consisted of the NOs proposed by NOC for the two NDs in the study, its definition and a five-point Likert scale. The data were analyzed using descriptive statistics. The NOs that obtained averages of 0.80 or higher were validated. The ND Risk for Infection was the most frequent, being validated eight (38.1%) of 21 NOs proposed by the NOC. The ND Self-Care Deficit: Bathing/Hygiene was the second most frequent and five (14.28%) out of 35 NOs were validated.

Descriptors: Nursing Diagnosis; Validation Studies; Nursing; Nursing Process; Nursing Assessment.

Introduction

The last two decades have been focused on the determination of patient outcomes in response to health actions, seeking to identify the individual and collective effects of the delivered services(1). In view of nurses' growing needs to describe and measure practice outcomes, terminologies have been created, with the Nursing Outcomes Classification (NOC), which started in 1991, as the most developed and used one. The team that developed the NOC, including experienced researchers from the University of Iowa, accomplished an extensive literature review to identify patient indicators and outcomes influenced by nursing actions, grouped and refined by expert nurses from different specialties(2). The Nursing Outcomes (NOs) from the Iowa Outcomes Project have been constantly tested to check their validity and reliability in different specialties(3).

The NOC complements the other two classifications, the North American Nursing Diagnosis Association International - NANDA-I, which groups Nursing Diagnoses (ND) and the Nursing Intervention Classification - NIC, which groups nursing interventions and activities. These three terminologies complement one another and can be used in computer systems to apply the Nursing Process (NP)(4). This method can be considered a deliberate intellectual activity to help nurses with decision making, with a focus on achieving expected outcomes(5-8).

The leading researchers of NIC and NOC studies have developed connections between the three classifications (NANDA-I/NIC/NOC)(4).

The first NOC publication, issued in 1997, contained 190 outcomes. The second edition, from 2000, already covered 260 outcomes and the third, published in 2004, was expanded to 330 outcomes. The fourth edition, from 2008, has not been translated to Portuguese yet and presents 385 outcomes, grouped in 31 classes and seven domains(9-10).

The NOC includes NO that describe patients' status, behaviors, reactions and feelings in response to the delivered care. A five-point Likert scale accompanies each NO to assess the listed indicators. Fourteen different five-point Likert scales exist to assess the range of outcomes that are part of the classification. The scales permit measurements at any point in a continuum, with the fifth point reflecting the patient's most desired condition regarding the outcome. This makes it easier to identify changes in the patient's status through different scores over time. Thus, the use of the NOC permits monitoring improvement, worsening or stagnation in the patient's status during a care period(10).

This research was developed to gain deeper knowledge on the NOC and select nursing outcomes to assess interventions put in practice for specific patient groups, based on the most frequent nursing diagnoses. The researchers expect to obtain support to complement the steps of the computerized Nursing Process at the institution where the study was accomplished, besides helping other health institutions that deliver care to patients with similar characteristics, thus qualifying nursing care.

The goal of this study was to validate the NOC's NO for the two NP most frequent in hospitalized surgical, clinical and critical patients, based on the connection between NOC and NANDA-I.

Method

This is a content validation study. Content validation involves a systematic analysis of contents by expert nurses, selected based on a scoring system(9,11-12). NO validation studies are still incipient in Brazil, which justifies the choice to adapt Fehring's content validation method, a researcher on ND validation(11).

The research was developed at the Clinical (SEM), Surgical (SEC) and Intensive Care (SETI) Nursing Services of a university hospital.

The study involved two phases. First, the information contained in the database of the institution's nursing prescription computer system was considered as the population, related to past patient hospitalizations at the Services under analysis. The sample comprised information on the two most frequent ND in hospitalized adult patients. The researcher collected the data. In the first phase, information was surveyed for six alternate months, between July 2007 and June 2008. Data were analyzed through descriptive statistics, with frequencies (f) and percentages (%) of the investigated ND.

The second phase consisted of the content validation of the suggested and additional associated NO proposed in the fourth NOC edition, still in English, for the two ND(10,13). In this phase, the sample comprised the nurses who complied with the following inclusion criteria: participating/having participated in research and refresher activities on the NP at the institution for at least four months during the last five years; academic-scientific production in the NP and Nursing Classifications area; professional experience of at least two years; work at the institution for at least one year, using the NP; experience of at least one year with surgical, clinical and critical patients during the last five years and agreement to participate in the research through the signing of the Informed Consent Term (ICT). The exclusion criterion was defined as: not returning the second-phase research instrument within 30 days after it was handed over.

First, to select expert nurses, a survey was carried out at the institution's Nursing Diagnosis Work Group (GTDE) to identify possible professionals that complied with the inclusion criteria. In total, 15 nurses were indicated, 13 of whom complied with the expert prerequisites. Twelve of them returned the instrument properly filled out within the deadline set, together with the signed ICT.

The validation instrument for the second phase was a table with seven columns, comprising the following parts: first column: outcomes proposed in NOC and their definitions; second to sixth column - five-point Likert scale (1= not important; 2= not very important; 3= moderately important; 4= very important; 5= extremely important) to measure the importance of each outcome for the ND, and the seventh column offered room for the experts to write down suggestions, criticism and observations.

Second-phase data were analyzed through descriptive statistics and inserted in Microsoft Excel 2007. Weighted arithmetical averages were calculated for the scores the experts attributed to each outcome, considering the following values: 1 = 0; 2 = 0.25; 3 = 0.50; 4 = 0.75; 5 = 1(11). The NO proposed for the two most frequent ND during patient hospitalizations at the SEM, SEC and SETI were validated according to the chapter in the fourth NOC edition published on the connections with the NANDA-I ND and the NOC NO. Fehring proposes categorizing the NO as critical, with a weighted arithmetical average of 0.80 or higher, and as supplementary when averages range between 0.79 and 0.50. NO with averages below 0.50 are discarded(11,14). In this study, however, the cut-off point was set at 0.80 for the NO, eliminating categories. The cut-off point selected in this research phase is justified by the 80% inter-rater agreement level Fehring suggests to categorize the critical indicators(11). The NOC recommendation to choose only NO that are truly relevant in the care context where they will be applied is also highlighted(10). The goal of an 80% inter-rater agreement level is to provide the set of NOC NO greater consistency, solidity and applicability for use in the computer system. Approval for the project was obtained from the Research Ethics Committee at the place of study.

Results

The two most frequent ND in patient hospitalizations at the SEC, SEM and SETI, in the first study phase, are shown in Table 1.

In the second study phase, the expert group was constituted with 12 nurses. Their characteristics are shown in Table 2.

In the fourth edition of NOC, not yet translated to Portuguese, the chapter on connections proposes 24 NO for the ND Risk for Infection, all of which are considered suggested, as all ND regarding risk in the NOC only present connections for suggested NOs(10). Three of the proposed NO were excluded, because they did not regard adult surgical, clinical or critical patients attended at the HCPA. There are: Burn healing, Burn recovery and Infection Severity: newborn. Next, the scores are shown of the NO proposed by NOC for the ND Risk for Infection.

Table 4

Next, the scores of the NO proposed by NOC are presented for the ND Self-Care Deficit: Bathing/Hygiene.

Discussion

Regarding the characteristics of the expert sample, the difficulty to define inclusion criteria in validation studies is well known. Not only is there no consensus in literature on specific criteria, but there is also a barrier regarding nurses' education and specific professional training(15). The expert sample showed to be qualified though, evidencing commitment to academic and research activities, as five hold a Master's degree and one is taking a Master's program. They have also published academic papers on Nursing Care Systemization. Eleven of the experts participated in the institution's GTDE, which can be appointed as another qualifying factor for these nurses. The goal of the group is to update and discuss the NP method, focusing on the ND phase, put in practice in the institution's computer system. The activities developed in the GTDE offer professional growth opportunities to nurses at the institution, teachers and students, and also permitted a rich and integrated partnership, as theory and practice are combined(16).

The ND Risk for Infection, defined as being "at increased risk for being invaded by pathogenic organisms"(17); was the most frequent at the three nursing services analyzed. This finding is in line with the results of other studies involving surgical, clinical and critical patients(18-20). The NO Risk Control: Infectious Process and Pre-Procedure Readiness were included in the fourth NOC edition. The experts validated the NO Risk Control: Infectious Process, while Pre-Procedure Readiness was discarded.

The ND Risk for Infection can be identified as the most present in hospitalized patients due to several factors involved in the hospitalization process, demanding a preventive attitude that should guide nursing care actions, in view of its interface with other diagnoses(21). Risk for Infection is associated, among others, with treatment-related factors (surgery, presence of invasive accesses, medication therapy). Besides invasive procedures and insufficient primary defense, provoked by the surgical trauma, various other factors influence the incidence of surgical wound infection, including the patient's preoperative clinical conditions, the technical conditions the surgery was accomplished in and preoperative hospital stay(18). Having breaks in skin continuity due to invasive procedures, surgical wounds, scarifications due to compression or chafes, being bedridden and with an impaired immune status were identified in a study as risk factors for the ND Risk for Infection(22). Patients' impaired mobility is another risk factor for this ND because it enhances the development of pulmonary edema, favoring infections and atelectasis(22). Infection prevention and control demand technical and behavioral measures, influencing health quality and the consequent reduction of efforts, problems, complications and resources(23).

The NO Knowledge: Infection Control was the NO that scored highest in this study. The experts may have considered that patients' knowledge on the prevention and identification of signs and symptoms, among other infection-related information, can influence its incidence and also limit the development of infectious processes, to the extent that patients themselves can identify the signs and symptoms.

The ND Self-Care Deficit: Bathing/Hygiene, defined as "impaired ability to produce or complete bathing/hygiene activities for oneself"(17); was the second most frequent ND at the services under analysis. Published studies ratify this study finding, identifying the ND Self-Care Deficit: Bathing/Hygiene among the most frequent in patients with clinical, surgical and critical problems(19,21). For this ND, the NOC proposes 35 NO, four (11.42%) of which are classified as suggested and 31 (88.58%) as additional associated.

Among the four suggested outcomes from the fourth NOC edition, the experts validated three: Self-Care: Bathing, Self-Care: Hygiene and Self-Care: Activities of Daily Living (ADL). The NO Ostomy Self-Care, defined as "personal activities to maintain the ostomy for elimination"(9); considered as suggested in the third and fourth NOC editions, was discarded.

In the third NOC edition, besides the four NO included as suggested in the fourth edition for the ND Self-Care Deficit: Bathing/Hygiene, the NO Self-Care: Oral Hygiene is also proposed as suggested. In the fourth edition, this outcome was classified as additional associated. In this study, however, the experts validated the NO Self-Care: Oral Hygiene, defined as "ability to take care of the own mouth and teeth". This finding demonstrates that oral hygiene is valued by nurses and sustained by literature, as periodontal disease has been associated with different illnesses, including bacterial pneumonia, chronic obstructive pulmonary disease (COPD), diabetes mellitus, cardiovascular illnesses, rheumatoid arthritis and premature labor, which can result from the colonization of the oral cavity and oropharynx by potential respiratory pathogens. The complications deriving from the lack or inadequacy of the oral hygiene procedure can extend hospital stay by 6.8 to 30 days, demanding mechanic control of bacterial plaque through brushing and dental floss use(24). That is the context the nursing team is inserted in, delivering care according to the patient's dependence level.

The experts may have validated the NO Self-Care: Activities of Daily Living (ADL) and Pain Level because these outcomes interfere directly in bathing activities. The nurses probably considered Pain Level important because it interferes directly in care management by the nursing team. The greater the patients' pain, the more dependent they will probably be on nursing care for bathing. In many cases, more than a symptom, pain is the disease itself, and its control is the goal of treatment. Its experience results in biological and psychosocial alterations and suffering. Sleep, movements and walking are impaired(25); interfering directly in body hygiene maintenance activities.

In line with the importance of these findings, studies appoint that the effective application of the nursing process in all of its phases, including the validation phase of nursing care outcomes, leads to better health care quality and stimulates the construction of theoretical and scientific knowledge based on best clinical practices(26-28).

Conclusion

In view of the study aims, the ND Risk for Infection was the most frequent at the three nursing services under analysis. Out of 21 NO the NOC proposed for this ND, the experts validated eight (38.1%). The ND Self-Care Deficit: Bathing/Hygiene was the second most frequent among the studied patients. Out of 35 NO proposed for the ND, five (14.28%) were validated.

Thus, based on the data analysis produced in this research, it is concluded that the use of the NOC classification, although still recent in Brazil, represents a viable alternative to assess and identify the best nursing care practices.

The use of nursing classifications has shown improvements and significant advances not only in documentation quality, but also in nursing practices. It is known that establishing the ND alone is insufficient to clarify patient needs. To achieve desired and more adequate outcomes, interventions need to be listed and outcomes to be achieved need to be set.

As a recommendation for future research, similar studies with other expert samples are appointed, and also focusing on patients from other age ranges and in different scenarios. For the same nursing diagnoses, other nursing outcomes could be considered a priority in view of other client groups.

Received: Mar. 14th 2010

Accepted: Nov. 5th 2010

1 Paper extracted from Master's Dissertation "Validação de resultados de enfermagem segundo a nursing outcomes classification noc na prática clínica de enfermagem em um hospital universitário" presented to Programa de Mestrado em Enfermagem, Escola de Enfermagem, Universidade Federal do Rio Grande do Sul, RS, Brazil. Supported by, Fundo de Incentivo à Pesquisa e Eventos, Hospital de Clínicas de Porto Alegre (FIPE/HCPA), RS, Brazil.

  • 1. Behrenbeck JG, Timm JA, Griebenow LK, Demmer KA. Nursing-sensitive outcome reliability testing in a tertiary care setting. Int J Nurs Terminol Classif. 2005 Jan-Mar;16(1):14-20.
  • 2. Head BJ, Maas M, Johnson M. Validity and community-health-nursing sensitivity of six outcomes for community health nursing with older clients. Public Health Nurs. 2003 Sep-Oct;20(5):385-98.
  • 3. Macnee C, Edwards J, Kaplan A, Reed S, Bradford S, Walls J, et al. Evaluation of NOC standardized outcome of "health seeking behavior" in nurse-managed clinics. J Nurs Care Qual. 2006 Jul-Sep;21(3):242-7.
  • 4. Johnson M. NANDA, NOC and NIC Linkages. 2 ed. St. Louis: Mosby; 2006.
  • 5. Alfaro-Lefevre R. Aplicação do Processo de enfermagem: um guia passo a passo. 4 ed. Porto Alegre: Artes Médicas; 2000.
  • 6. Carpenito L. Diagnósticos de enfermagem: aplicação à prática clínica. 8 ed. Porto Alegre: Artes Médicas; 2002.
  • 7. Backes DS, Koerich MS, Nascimento KC, Erdmann AL. Nursing care systematization as a multidimensional and interactive phenomenon. Rev. Latino-Am. Enfermagem. 2008 Nov-Dec;16(6):979-85.
  • 8. Amante LN, Rossetto AP, Schneider DG. Nursing care systematization at the intensive care unit (ICU) based on Wanda Horta's theory. Rev Esc Enferm USP. 2009 Mar;43(1):54-64.
  • 9. Moorhead S, Johnson M, Maas M. Classificação dos Resultados de Enfermagem NOC. 3 ed. Porto Alegre: Artmed; 2008.
  • 10. Moorhead S, Johnson M, Maas M. Nursing Outcomes Classification NOC. 4 ed. New York: Elsevier; 2008.
  • 11. Fehring RJ. Methods to validate nursing diagnoses. Heart Lung. 1987 Nov;16(6 Pt 1):625-9.
  • 12. Carvalho EC, Mello AS, Napoleão AA, Bachion MM, Dalri MCB, Canini SRMS. Validação de diagnósticos de enfermagem: reflexão sobre dificuldades enfrentadas por pesquisadores. Rev Eletr Enferm. 2008;10(1):235-40. [acesso 14 jan 2010]. Disponível em: HTTP://www.fen.ufg.br/revista/v10/n1/v10n1a22.thm
  • 13. Benedet S, Bub M. Manual de Diagnósticos de Enfermagem. 2ed. Florianópolis: Bernúncia; 2001.
  • 14. Head BJ, Aquilino ML, Johnson M, Reed D, Maas M, Moorhead S. Content validity and nursing sensitivity of community-level outcomes from the Nursing Outcomes Classification (NOC). J Nurs Scholarsh. 2004;36(3):251-9.
  • 15. Galdeano LE, Rossi L. Validação de Conteúdo Diagnóstico: Critérios para seleção de expertos. Ciênc Cuidado Saúde. 2006;5(1):60-6.
  • 16. Almeida MA, Franzen E, Vieira RW, Araújo VG, Laurent MCR. Estudos Clínicos sobre Processo e Diagnóstico de Enfermagem em um Hospital Universitário - Relato de Experiência. Rev HCPA. 2007;27(2):4.
  • 17
    North American Nursing Diagnosis Association. Diagnósticos de enfermagem da NANDA: definições e classificação, 2009-2011. Porto alegre: Artmed; 2009.
  • 18. Galdeano L, Rossi L, Santos C, Dantas R. Nursing diagnosis in the perioperative period of cardiac surgery. Rev Esc Enferm USP. 2006 Mar;40(1):26-33.
  • 19. Volpato MP, Cruz DALM. Diagnósticos de Enfermagem de pacientes internadas em unidade médico-cirúrgica. Acta Paul Enferm. 2007;20(2):119-24.
  • 20. Silva F, Viana M, Volpato M. Nursing diagnoses in patients admitted by the orthopedic clinic in a surgical unit. Rev Gauch Enferm. 2008 Dec;29(4):565-72.
  • 21. Cafer CR, Barros ALBL, Lucena AF, Mahl MLS, Michel JLM. Diagnósticos de enfermagem e proposta de intervenções para pacientes com lesão medular. Acta Paul Enferm. 2005;18(4):347-53.
  • 22. Pasini D, Alvim I, Kanda L, Mendes RSP, Cruz DALM. Diagnósticos de enfermagem de pacientes em unidades de terapia intensiva Rev Esc Enferm USP. 1996;30(3):501-18.
  • 23. Barreto RASS, Rocha LO, Souza ACS, Tipple AFV, Suzuki K, Bisinoto SA. Higienização das mãos: a adesão entre os profissionais de Enfermagem da sala de recuperação pós-anestésica. Rev Eletr Enferm. 2009;11(2):334-40. [acesso 14 jan 2010]. Disponível em: hrrp://WWW.fen.ufg.br/revista/v11/n2/v11n2a14.htm
  • 24. Martins GC, Santos NO, Gomes ET. Higiene Oral: Atuação da Equipe de Enfermagem em Paciente com Déficit no Autocuidado. Rev Enferm Integrada. 2009;2(1):8.
  • 25. Rigotti MA, Ferreira AM. Intervenções de enfermagem ao paciente com dor. Arq Ciênc Saúde. 2005;12(1):50-4.
  • 26. Cerullo JAS, Cruz DALM. Raciocínio clínico e pensamento crítico. Rev. Latino-Am. Enfermagem. 2010;18(1):124-9.
  • 27. Almeida MA, Pergher AK, Canto DF. Validação do mapeamento de cuidados prescritos para pacientes ortopédicos à classificação das intervenções de enfermagem. Rev. Latino-Am. Enfermagem. 2010;18(1):116-23.
  • 28. Pokorski S, Moraes MA, Chiarelli R, Costanzi AP, Rabelo ER. Processo de enfermagem: da literatura à prática. O quê de fato nós estamos fazendo? Rev. Latino-Am. Enfermagem. 2009;17(3):302-7.
  • Corresponding Author:
    Miriam de Abreu Almeida
    Universidade Federal do Rio Grande do Sul
    Escola de Enfermagem
    Departamento de Enfermagem Médico-Cirúrgica
    Rua São Manoel, 963
    Bairro Rio Branco
    CEP: 90620-110 Porto Alegre, RS, Brasil
    E-mail:
  • Publication Dates

    • Publication in this collection
      11 Mar 2011
    • Date of issue
      Feb 2011

    History

    • Accepted
      05 Nov 2010
    • Received
      14 Mar 2010
    Escola de Enfermagem de Ribeirão Preto / Universidade de São Paulo Av. Bandeirantes, 3900, 14040-902 Ribeirão Preto SP Brazil, Tel.: +55 (16) 3315-3451 / 3315-4407 - Ribeirão Preto - SP - Brazil
    E-mail: rlae@eerp.usp.br