Acessibilidade / Reportar erro

Measurement of the area of venous ulcers using two software programs1 1 Paper extracted from Master's Thesis "Measurement of venous ulcers area using the softwares Autocad(r) and ImageTool: reproducibility of methods", presented to Universidade Federal de Santa Maria, Santa Maria, RS, Brazil. Supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) and Programa de Iniciação Científica (PROIC) do Hospital Universitário de Santa Maria.

ABSTRACT

Objective:

to compare the measurement area of venous ulcers using AutoCAD(r) and Image Tool software.

Method:

this was an assessment of reproducibility tests conducted in a angiology clinic of a university hospital. Data were collected from 21 patients with venous ulcers, in the period from March to July of 2015, using a collection form and photograph of wounds. Five nurses (evaluators) of the hospital skin wound study group participated. The wounds were measured using both software programs. Data were analyzed using intraclass correlation coefficient, concordance correlation coefficient and Bland-Altman analysis. The study met the ethical aspects in accordance with current legislation.

Results:

the size of ulcers varied widely, however, without significant difference between the measurements; an excellent intraclass and concordance correlation was found between both software programs, which seem to be more accurate when measuring a wound area >10 cm².

Conclusion:

the use of both software programs is appropriate for measurement of venous ulcers, appearing to be more accurate when used to measure a wound area > 10 cm².

Descriptors:
Nursing; Varicose Ulcer; Weights and Measures; Software Validation

RESUMO

Objetivo:

comparar a mensuração de área de úlceras venosas por meio dos softwares AutoCAD(r) e Image Tool.

Método:

trata-se de um estudo de avaliação de reprodutibilidade de testes, realizado em um ambulatório de angiologia de um hospital universitário. Os dados foram coletados de 21 pacientes com úlceras venosas, no período de março a julho de 2015, por meio de formulário de coleta e fotografia das feridas. Cinco enfermeiros (avaliadores) do Grupo de Estudos de Lesões de Pele do hospital participaram da pesquisa. As feridas foram mensuradas em ambos os softwares. Os dados foram analisados por meio do Coeficiente de correlação intraclasse, Coeficiente de correlação de concordância e procedimento de Bland e Altman. A pesquisa respeitou os aspectos éticos de acordo com a legislação vigente.

Resultados:

os tamanhos das úlceras apresentaram grande amplitude, porém, sem diferença significativa entre as mensurações, existe excelente correlação intraclasse e de concordância entre os softwares, os quais parecem ser mais precisos na mensuração de feridas com área > 10 cm².

Conclusão:

o uso de ambos os softwares é indicado para a mensuração de úlceras venosas, parecendo ser mais precisos quando utilizados para mensurar feridas com área > 10 cm².

Descritores:
Enfermagem; Úlcera Varicosa; Pesos e Medidas; Validação de Programas de Computador

RESUMEN

Objetivo:

comparar la medida del área de úlceras venosas por medio de los softwares AutoCAD(r) e Image Tool.

Método:

se trata de un estudio de evaluación de reproducibilidad de pruebas, realizado en un ambulatorio de angiología de un hospital universitario. Los datos fueron recolectados de 21 pacientes con úlceras venosas, en el período de marzo a julio de 2015, por medio de formulario de recolección y fotografías de las heridas. Cinco enfermeros (evaluadores) del Grupo de Estudios de Lesiones de Piel del hospital participaron de la investigación. Las heridas fueron medidas en ambos softwares. Los datos fueron analizados por medio de: Coeficiente de correlación intraclase, Coeficiente de correlación de concordancia y Procedimiento de Bland y Altman. La investigación respetó los aspectos éticos de acuerdo con la legislación vigente.

Resultados:

los tamaños de las úlceras presentaron gran amplitud, sin embargo, sin diferencia significativa entre las medidas; existe excelente correlación intraclase y de concordancia entre los softwares, los que parecen ser más precisos en medidas de heridas con área > 10 cm².

Conclusión:

el uso de ambos softwares es indicado para medir úlceras venosas, pareciendo ser más precisos cuando utilizados para medir heridas con área > 10 cm².

Descriptores:
Enfermería; Úlcera Varicosa; Pesos y Medidas; Validación de Programas de Computación

Introduction

Venous ulcers are among the chronic conditions that affect the population and require differentiated management of nursing care. The prevalence is estimated at 0.5 to 0.8%, with an incidence between two and five new cases per thousand per year11. Verdú J, Marinello J, Armans E, Carreño P, March JR, Soldevilla J, coordenadores. Conferencia nacional de consenso sobre úlceras de la extremidad inferior. Documento de consenso C.O.N.U.E.I. Barcelona : Edi Kamed SL; 2009..

Venous ulcers significantly affect the quality of life of individuals, with repercussions at work, in social relationships, and limitations in their daily routines22. Silva DV, Budó MLD, Schimith MD, Ecco L, Costa IKF, Torres GV. Experiences constructed in the process of living with a venous ulcer. Cogitare Enferm. [Internet]. 2015. [Access Jun 6, 2016]5;20(1):13-9. Available from: http://revistas.ufpr.br/cogitare/article/view/37784/24830
http://revistas.ufpr.br/cogitare/article...
. Furthermore, they cost 900-1000 EUR when requiring three to six months for healing11. Verdú J, Marinello J, Armans E, Carreño P, March JR, Soldevilla J, coordenadores. Conferencia nacional de consenso sobre úlceras de la extremidad inferior. Documento de consenso C.O.N.U.E.I. Barcelona : Edi Kamed SL; 2009..

Considering this scenario, the nurse has an important role in the evaluation of these patients22. Silva DV, Budó MLD, Schimith MD, Ecco L, Costa IKF, Torres GV. Experiences constructed in the process of living with a venous ulcer. Cogitare Enferm. [Internet]. 2015. [Access Jun 6, 2016]5;20(1):13-9. Available from: http://revistas.ufpr.br/cogitare/article/view/37784/24830
http://revistas.ufpr.br/cogitare/article...
-33. Jesus PBR, Brandão ES, Silva CRL. Nursing care to clients with venous ulcers an integrative review of the literature. Rev Pesqui Cuid Fundam. (Online). [Internet]. 2015 [Access Jun 6, 2016]; 7(2):2639-48. Available from: http://www.seer.unirio.br/index.php/cuidadofundamental/article/view/2176/pdf_1560
http://www.seer.unirio.br/index.php/cuid...
and should use available technologies to conduct this process. Thus, the measurement is an objective way to evaluate the wound and identify the progress of healing.

In addition to the importance of the subject being studied, a gap remains in the knowledge produced44. Eberhardt TD, Kessler M, Soares RSA, Lima SBS, Fonseca GGP, Rodrigues LR. Mensuração de feridas: revisão da literatura. Ciência & Saúde. [Internet]. 2015 [Acesso 6 jun 2016];8(2):79-84. Disponível em: http://revistaseletronicas.pucrs.br/ojs/index.php/faenfi/article/view/19947/13476
http://revistaseletronicas.pucrs.br/ojs/...
. Among the software available for measurement, two programs can be used: AutoCAD(r) software and the Image Tool. The first is a program commonly used by engineers in topography for physical area calculation55. Reis CLD, Cavalcante JM, Rocha Jr EP, Neves RS, Santana LA, Guadagnin RV, et al. Mensuração de área de úlceras por pressão por meio dos softwares Motic e do AutoCAD(r). Rev Bras Enferm. [Internet]. 2012 [Acesso 6 jun 2016];65(2):304-8. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000200016
http://www.scielo.br/scielo.php?script=s...
and the second was developed by the University of Texas Health Sciences Center at San Antonio, and is a free software used as a tool to obtain objective and reliable measures to the real size of the lesion66. Sousa ATO, Vasconcelos JMB, Soares MJGO. Software Image Tool 3.0 as an instrument for measuring wounds. Rev Enferm UFPE On Line. [Internet]. 2012 [Access Jun 6, 2016];6(10):2569-73. Available from: http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/download/3078/4533
http://www.revista.ufpe.br/revistaenferm...
.

Based on the above, the objective was to compare the measurement of the area of venous ulcers using AutoCAD(r) and Image Tool software.

Method

This was a reproducibility study of evaluation tests. The study was conducted in a angiology clinic of a university hospital in southern Brazil, which treats patients with venous ulcers. The patients were selected according to the following inclusion criteria: age over 18 years, intact cognitive and verbal skills, presenting venous ulcers covering one side of the lower limb.

Five nurses who were participants in the skin wound study group of the hospital where the research was conducted, called evaluators, were responsible for data collection, which occurred from March to July of 2015. All nurses were trained in the process of obtaining the photographs and wound measurement.

In the collection period, among 48 patients with venous ulcers, 21 met the inclusion criteria, totaling 36 venous ulcers, 72 photographs and 144 measurements in each software program. The researcher and one of the evaluators, who was present at the time of ulcer dressing change, photographed the venous ulcers. Both photographs were taken during the same care period. Next, the information was transferred to the researcher's notebook. The evaluator and the researcher performed the measurements of both photographs. This process was performed with all evaluators, always in pairs (researcher and one evaluator), until reaching the maximum number of individuals of the study population.

The data collection procedure occurred according to the following protocols.

-Protocol for obtaining photographs of wounds: after cleaning the wound with 0.9% saline solution, a surgical compress was placed under the lower limb with the ulcer, in order to give a white background color for the picture; a black square with three centimeters printed in on A4 sheet was placed close to the ulcer, to be considered as a reference object. The photograph was taken with a Fujifilm Camera FinePix S14 megapixels (f/6.4, ISO 400, enabled macro function, automatic white balance, high sharpness, and flash disabled). The camera was perpendicular to the wound (90°), and 50 cm away from the wound, bringing it closer to or farther away from the wound, as necessary.

-Protocol for wound area measurement using the AutoCAD(r) software (software 1): based on the manual from the Federal University of Santa Catarina77. Universidade Federal de Santa Catarina. Centro Tecnológico. Departamento de Engenharia Civil. Programa de Educação Tutorial da Engenharia Civil PET/ECV. Curso Básico de AutoCAD 2015: Desenho em 2D para Engenharia Civil. Florianópolis: UFSC; 2015.. The 2015 version was used, with educational license for student registration Nº 900-5013697. To obtain the area of a venous ulcer, in cm², the following formula was used:

-Protocol for wound area measurement using the software Image tool (software 2): based on the information described in the article, Software Image Tool 3.0 an instrument for measuring the wounds, published in 201266. Sousa ATO, Vasconcelos JMB, Soares MJGO. Software Image Tool 3.0 as an instrument for measuring wounds. Rev Enferm UFPE On Line. [Internet]. 2012 [Access Jun 6, 2016];6(10):2569-73. Available from: http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/download/3078/4533
http://www.revista.ufpe.br/revistaenferm...
.

Data were uploaded into Microsoft Office Excel(r), using duplicate, independent entry in order to correct any typographical errors. For the statistical treatment of the area of the venous ulcers, mean and standard error were used. In order to identify the normality of the data distribution, the Shapiro-Wilk test was applied; due to lack of data normality, the Wilcoxon test was used in order to identify differences between the measurements, and to verify the degree of systematic differences between measurements in pairs (researcher and evaluator).

The distribution of the differences between the measures and the mean difference between them was analyzed88. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45:255-68.. The reproducibility was measured using the intraclass correlation coefficient (ICC) and concordance correlation coefficient (CCC) of Lin88. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45:255-68.. The correlation was considered low for values < 0.40, moderate for values between 0.40 and 0.75, and excellent for values > 0.75.

The Bland and Altman procedure99. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135-60. was performed only on normally distributed data, after performing the logarithmic transformation. For analysis purposes, a significance level of 5% and confidence interval (CI) of 95% were used Analyses were performed using the R statistical program.

The ethical principles were based on the National Council of Health Resolution No. 466 of December 12, 2012. The project was submitted to the Research Ethics Committee and was approved, protocol No. 932838 and (CAAE) No. 40250814.6.0000.5346. The research was conducted after the Terms of Free and Informed Consent Form was signed by participants.

Results

The study participants (n = 21) had a mean age of 60.9 years, and nine (42.9%) were aged 64-72 years; the majority were male (66.7%). The number of venous ulcers ranged from one to five per patient, with a median of 1.8, totaling 36 venous ulcers (Figure 1).

Figure 1
Diagram of the study participants selection and data collection procedure, Santa Maria, RS, Brazil, 2015

The areas measured by software 1 ranged from 0.2 to 71.0 cm², with a mean of 14.4 ± 1.4. The areas measured by software 2 ranged from 0.4 to 89.1, with a mean of 14.9 ± 1.5. The dispersion of the measures are demonstrated in Figure 2, and the mean observed difference of 1.6 ± 0.2. No difference (p = 0.80) was identified between the measurements performed by the two software programs. There were excellent ICC [ρ = 0.98; 95% (0.98-0.99); p < 0.05] and CCC [0.95; IC 95% (0.92-0.97) values.

Figure 2
a) Scatterplot of measured areas in AutoCAD(r) (s1) and Image Tool (s2) software; b) Scatter plot of the areas ≤ 10 cm² measured in AutoCAD(r) (s1) and Image Tool (s2); c) Chart <dispersion of areas> 10 cm² measured in AutoCAD(r) (s1) and Image Tool (s2); d) Scatter plot of the areas measured by the researcher in AutoCAD(r) (s1) and Image Tool (s2); e) Scatter plot of the areas measured by the evaluators in AutoCAD(r) (s1) and Image Tool (s2), Santa Maria, RS, Brazil, in 2015.

No difference was found between the measurements of venous ulcers ≤ 10 cm² (p=0.64) with the two software programs, with an area > 10 cm² (p=0.92) between the measurements of the researcher (p=0.80) and evaluators (p = 0.90).

The ICC and CCC values were excellent for all comparisons, as shown in Table 1.

Table 1
Intraclass correlation coefficient and concordance correlation coefficient of venous ulcers measured by AutoCAD(r) and Image Tool software. Santa Maria, RS, Brazil, 2015

The Bland-Altman plot (Figure 3) shows the correlation between the measurements in both software programs. This analysis was performed only for the two categories that showed normal distribution after log transformation (ulcers ≤ 10 cm² and> 10 cm²).

Figure 3
a) Bland-Altman plot for the difference between means and areas >10 cm², measured in AutoCAD(r) (s1) and Image Tool (s2); b) Bland- Altman plot for the difference between means and areas ≤10 cm², measured in AutoCAD(r) (s1) and Image Tool (s2), Santa Maria, RS, Brazil, 2015

For ulcers with an area >10 cm², the upper concordance limit (UCL) was 1.26 cm², and the lower concordance limit (LCL) was 0.74 cm², with a measure outside these limits. At this time, for the wounds with an area ≤10 cm², a UCL of 1.8 cm² was found and a LCL of 0.02 cm², and a variety of measures outside these limits were displayed.

Discussion

The sizes of ulcers varied widely; however, there was no statistically significant difference between the measurements. Other studies that characterized patients with venous ulcers in outpatient care also found wide range of wound sizes1010. Sant'ana SMSC, Bachion MM, Santos QR, Nunes CAB, Malaquias SG, Oliveira BGRB. Úlceras venosas: caracterização clínica e tratamento em usuários atendidos em rede ambulatorial. Rev Bras Enferm. [Internet]. 2012 [Acesso 6 jun 2016];65(4):637-44. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000400013
http://www.scielo.br/scielo.php?script=s...
-1111. Malaquias SG, Bachion MM, Sant'Ana SMSC, Dallarmi CCB, Lino Junior RS, Ferreira PS. People with vascular ulcers in outpatient nursing care: a study of sociodemographic and clinical variables. Rev Esc Enferm USP. [Internet]. 2012 [Acesso 6 jun 2016];46(2):302-10. Disponível em: http://www.scielo.br/scielo.php?pid=S0080-62342012000200006&script=sci_arttext&tlng=en
http://www.scielo.br/scielo.php?pid=S008...
, corroborating the findings of this research. Still, it may indicate that venous ulcers are wounds that have different sizes.

The ICC and the CCC data demonstrate that the measurements have intraclass correlation and excellent concordance, i.e., the use of both software programs is suitable for the measurement of venous ulcers.

However, both software programs seem to be more accurate when used to measure large wounds (with an area > 10 cm²), as the limits of agreement were clinically acceptable, and only one measurement was out of bounds. Considering that, when analyzing small wounds (area ≤ 10 cm²), the UCL was clinically questionable and there were several measurements outside the limits of concordance.

Another study55. Reis CLD, Cavalcante JM, Rocha Jr EP, Neves RS, Santana LA, Guadagnin RV, et al. Mensuração de área de úlceras por pressão por meio dos softwares Motic e do AutoCAD(r). Rev Bras Enferm. [Internet]. 2012 [Acesso 6 jun 2016];65(2):304-8. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000200016
http://www.scielo.br/scielo.php?script=s...
that compared the AutoCAD(r) software with another program also found that marking offsets, the relative point of view, is larger in wounds with a smaller area, since it requires hand movements of the operator.

One study1212. Bilgin M, Günes UY. A comparison of 3 wound measurement techniques. J Wound Ostomy Continence Nurs. 2014;40(6):590-3. doi: 10.1097/01.WON.0000436668.79024.f9.
https://doi.org/10.1097/01.WON.000043666...
compared three methods of pressure ulcer area measurement - ruler, tracing paper with graduated acetate, and digital planimetry; it found that all three methods were appropriate for measuring the surface area of small circular wounds (area ≤ 10 cm²); however, in irregularly shaped wounds > 10 cm², the ruler overestimated the size.

A systematic review1313. O'meara SM, Bland JM, Dumville JC, Cullum NA. A systematic review of the performance of instruments designed to measure the dimensions of pressure ulcers. Wound Repair Regen. 2012;20:263-76. doi: 10.1111/j.1524-475X.2012.00783.x.
https://doi.org/10.1111/j.1524-475X.2012...
that evaluated the performance of instruments designed to measure the dimensions of pressure ulcers found that digital photography, combined with software for the measurement of wounds, demonstrated satisfactory agreement.

The limitations of this study are: the type of sampling (non-probabilistic), the lack of studies available on the subject, and the lack of training on basic concepts regarding the evaluation of wounds, such as defining the surface and wound edges.

Further studies are suggested with this software, with comparisons between invasive and non-invasive methods, and analyzing different types of chronic wounds. Still, the incorporation of digital photographs and measurement by means of computer programs is supported, as it enables a most accurate record of the aspects and measures of the wound.

Conclusion

The sizes of the ulcers showed great variance, however no statistically significant difference was found between the measurements made with the two software programs. The intraclass correlation coefficient and agreement were excellent, and both software programs were suitable for the measurement of venous ulcers; they may be more accurate when used to measure wound areas > 10 cm².

References

  • 1
    Verdú J, Marinello J, Armans E, Carreño P, March JR, Soldevilla J, coordenadores. Conferencia nacional de consenso sobre úlceras de la extremidad inferior. Documento de consenso C.O.N.U.E.I. Barcelona : Edi Kamed SL; 2009.
  • 2
    Silva DV, Budó MLD, Schimith MD, Ecco L, Costa IKF, Torres GV. Experiences constructed in the process of living with a venous ulcer. Cogitare Enferm. [Internet]. 2015. [Access Jun 6, 2016]5;20(1):13-9. Available from: http://revistas.ufpr.br/cogitare/article/view/37784/24830
    » http://revistas.ufpr.br/cogitare/article/view/37784/24830
  • 3
    Jesus PBR, Brandão ES, Silva CRL. Nursing care to clients with venous ulcers an integrative review of the literature. Rev Pesqui Cuid Fundam. (Online). [Internet]. 2015 [Access Jun 6, 2016]; 7(2):2639-48. Available from: http://www.seer.unirio.br/index.php/cuidadofundamental/article/view/2176/pdf_1560
    » http://www.seer.unirio.br/index.php/cuidadofundamental/article/view/2176/pdf_1560
  • 4
    Eberhardt TD, Kessler M, Soares RSA, Lima SBS, Fonseca GGP, Rodrigues LR. Mensuração de feridas: revisão da literatura. Ciência & Saúde. [Internet]. 2015 [Acesso 6 jun 2016];8(2):79-84. Disponível em: http://revistaseletronicas.pucrs.br/ojs/index.php/faenfi/article/view/19947/13476
    » http://revistaseletronicas.pucrs.br/ojs/index.php/faenfi/article/view/19947/13476
  • 5
    Reis CLD, Cavalcante JM, Rocha Jr EP, Neves RS, Santana LA, Guadagnin RV, et al. Mensuração de área de úlceras por pressão por meio dos softwares Motic e do AutoCAD(r). Rev Bras Enferm. [Internet]. 2012 [Acesso 6 jun 2016];65(2):304-8. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000200016
    » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000200016
  • 6
    Sousa ATO, Vasconcelos JMB, Soares MJGO. Software Image Tool 3.0 as an instrument for measuring wounds. Rev Enferm UFPE On Line. [Internet]. 2012 [Access Jun 6, 2016];6(10):2569-73. Available from: http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/download/3078/4533
    » http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/download/3078/4533
  • 7
    Universidade Federal de Santa Catarina. Centro Tecnológico. Departamento de Engenharia Civil. Programa de Educação Tutorial da Engenharia Civil PET/ECV. Curso Básico de AutoCAD 2015: Desenho em 2D para Engenharia Civil. Florianópolis: UFSC; 2015.
  • 8
    Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45:255-68.
  • 9
    Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135-60.
  • 10
    Sant'ana SMSC, Bachion MM, Santos QR, Nunes CAB, Malaquias SG, Oliveira BGRB. Úlceras venosas: caracterização clínica e tratamento em usuários atendidos em rede ambulatorial. Rev Bras Enferm. [Internet]. 2012 [Acesso 6 jun 2016];65(4):637-44. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000400013
    » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000400013
  • 11
    Malaquias SG, Bachion MM, Sant'Ana SMSC, Dallarmi CCB, Lino Junior RS, Ferreira PS. People with vascular ulcers in outpatient nursing care: a study of sociodemographic and clinical variables. Rev Esc Enferm USP. [Internet]. 2012 [Acesso 6 jun 2016];46(2):302-10. Disponível em: http://www.scielo.br/scielo.php?pid=S0080-62342012000200006&script=sci_arttext&tlng=en
    » http://www.scielo.br/scielo.php?pid=S0080-62342012000200006&script=sci_arttext&tlng=en
  • 12
    Bilgin M, Günes UY. A comparison of 3 wound measurement techniques. J Wound Ostomy Continence Nurs. 2014;40(6):590-3. doi: 10.1097/01.WON.0000436668.79024.f9.
    » https://doi.org/10.1097/01.WON.0000436668.79024.f9
  • 13
    O'meara SM, Bland JM, Dumville JC, Cullum NA. A systematic review of the performance of instruments designed to measure the dimensions of pressure ulcers. Wound Repair Regen. 2012;20:263-76. doi: 10.1111/j.1524-475X.2012.00783.x.
    » https://doi.org/10.1111/j.1524-475X.2012.00783.x
  • 1
    Paper extracted from Master's Thesis "Measurement of venous ulcers area using the softwares Autocad(r) and ImageTool: reproducibility of methods", presented to Universidade Federal de Santa Maria, Santa Maria, RS, Brazil. Supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) and Programa de Iniciação Científica (PROIC) do Hospital Universitário de Santa Maria.

Publication Dates

  • Publication in this collection
    2016

History

  • Received
    09 June 2016
  • Accepted
    22 Sept 2016
Escola de Enfermagem de Ribeirão Preto / Universidade de São Paulo Av. Bandeirantes, 3900, 14040-902 Ribeirão Preto SP Brazil, Tel.: +55 (16) 3315-3451 / 3315-4407 - Ribeirão Preto - SP - Brazil
E-mail: rlae@eerp.usp.br