Mobile app for patient education about breast cancer surgical treatment

Aplicativo móvel para a educação de pacientes sobre o tratamento cirúrgico do câncer de mama

Abstract

Introduction:

Breast cancer is a relevant public health problem with high incidence, morbidity and mortality. Patients experience a feeling of uncertain future, low self-esteem, fear of death and mutilation. In this situation, the rational absorption of information is compromised and technology can help.

Objective:

To develop and validate a mobile application (app) for patient education on the surgical treatment of breast cancer. Patient education in breast cancer helps set patients’ preoperative expec-tations and satisfaction with the surgical experience and outcomes.

Methods:

The app was created in five stages: 1. analysis - situational diagnosis and integrative literature review; 2. design - planning and creation of didactic content; 3. development - definition of the app navigation; 4. implementation - configuration of tools and creation of an environment to download the app from the internet; 5. testing - application of usability, performance, compatibility, and functional tests. The developed app was validated by 13 doctors experienced in the treatment of breast cancer and 19 female breast cancer patients, using the Delphi technique.

Results:

Consensus was achieved, with an excellent overall content validity index of 1.00, both among the doctors and the patients in the first round of consultation.

Conclusion:

The app, named OncoMasto Cirurgia App, was developed after an integrative literature review, tested and validated for content by specialist doctors and by breast cancer patients, showing great agreement among the study participants.

Keywords:
Breast conservation therapy; Breast neoplasms; Mammaplasty; Mastectomy; Mobile applications

Resumo

Introdução:

O câncer de mama é um relevante problema de saúde pública com alta incidência, morbidade e mortalidade. Os pacientes experimentam uma sensação de futuro incerto, baixa autoestima, medo da morte e mutilação. Nessa situação, a absorção racional de informações fica comprometida e a tecnologia pode ajudar.

Objetivo:

Desenvolver e validar um aplicativo móvel (app) para a educação de pacientes sobre o tratamento cirúrgico do câncer de mama. A educação do paciente em câncer de mama ajuda a definir as expectativas pré-operatórias e a satisfação dos pacientes com a experiência e os resultados cirúrgicos.

Métodos:

O app foi elaborado em cinco etapas: 1. análise - diagnóstico situacional e revisão integrativa da literatura; 2. design - planejamento e criação de conteúdo didático; 3. desenvolvimento - definição da navegação do aplicativo; 4. implantação - configuração de ferramentas e criação de ambiente para download do aplicativo pela internet; 5. testes - aplicação de testes de usabilidade, desempenho, compatibilidade e funcionais. O aplicativo desenvolvido foi validado por 13 médicos com experiência no tratamento do câncer de mama e 19 mulheres diagnosticadas com a doença, utilizando a técnica Delphi.

Resultados:

O consenso foi alcançado, com um excelente índice de validade de conteúdo geral de 1,00, tanto entre os médicos quanto entre as pacientes, na primeira rodada de consulta.

Conclusão:

O app, denominado OncoMasto Cirurgia App, foi desenvolvido após revisão integrativa da literatura, testado e validado quanto ao conteúdo por médicos especialistas e por pacientes com câncer de mama, apresentando ótima concordância entre os participantes do estudo.

Palavras-chave:
Terapia de conservação da mama; Neoplasias mamárias; Mamoplastia; Mastectomia; Aplicações móveis

Introduction

Breast cancer is a public health problem throughout the world and a cause of great concern, especially among women, due to the high rates of morbidity and mortality, and associated sequelae.¹ It may result in decreased self-esteem and psychological distress, interfering in social, personal, professional, and affective relationships.^1,2 Breast cancer is the leading cause of cancer death among women and the second most incident malignancy in the world. It was estimated that about 2.1 million cases of breast cancer and 627 thousand deaths from the disease occurred in 2018 worldwide.³ About 66,280 new cases of breast cancer are estimated for 2020 in Brazil, corresponding to an estimated risk of 61.6 cases for every 100 thousand women. Female breast cancer is the second most frequent type of cancer in Brazil, just behind non-melanoma skin tumors.^2,4

At their surgical consultation, breast cancer patients receive information about the treatment that will follow, but may not remember it due to information overload, denial, or learning readiness, possibly related to psychological distress.^5,6 Several techniques are used for surgical treatment of breast cancer, resulting in discrepancies between the preoperative information provided and what patients hear and understand, leading to unrealistic expectations.⁷ Patients' preoperative expectations may not correspond to the postoperative results.

Patient education in breast cancer helps set patients’ preoperative expectations and satisfaction with the surgical experience and results.⁸ Informed breast cancer patients who participate in surgical decisions usually have reduced anxiety and depression, enhanced physical and psychological functioning, and a more optimistic attitude towards the future.⁹

At present, a wide variety of technological products facilitates the patient-doctor relationship.⁸ The Internet provides information that may assist in the planning of therapeutic interventions and surgical treatments, surgical simulations, and patient follow-up.¹⁰ Due to their portability, smartphones and tablets are utilized by health professionals to access and use this information in the medical practice and education.^11-13 The content of a mobile application (app) used as a health education strategy has to be based on evidence and provide information and activities for the recovery, development or strengthening of physical, mental or social abilities, aiming to promote health and social integration.^14,15

The development of a mobile app to deliver information about the most common concerns related to surgical treatments, possible complications, and likely outcomes to patients who are scheduled to undergo or had undergone breast cancer surgery, as well as to their family members, is essential to help them face this new reality of their lives. Thus, the aim of this study was to develop and validate a mobile app for patient education on the surgical treatment of breast cancer.

Methods

This was a methodological study in technology-based learning focused on the development and validation of a mobile app for patient education on the surgical treatment of breast cancer. The study was approved by the Research Ethics Committee of the “Dr. José Antônio Garcia Coutinho” School of Medical Sciences, Universidade do Vale do Sapucaí (UNIVÁS), Brazil (approval no. 3.584.628).

The contextualized instructional design (CID) was used in the development of the mobile app. The CID involves a constructivist proposal, combining intentional planning, development, and use of strategies that favor contextualization.^2,16,17

The mobile app was created in five stages: analysis, design, development, implementation, and testing. The analysis involved a situational diagnosis, with the identification and evaluation of the problem, and an integrative review of the literature. A search was conducted in the Scientific Electronic Library Online (SciELO), Latin American and Caribbean Literature in Health Science (LILACS), and U.S. National Library of Medicine (MEDLINE) databases using the keywords "mastectomy", "conservative breast surgery", "breast reconstruction", as well as their combinations in Portuguese, Spanish, and English, and the boolean operator "AND" (e.g., breast neoplasms AND mastectomy; conservative breast surgery AND breast reconstruction).

Only primary works directly related to the topic, written in Portuguese, English or Spanish, published between 2010 and 2020, and available in full text were included in the study. Theses, dissertations, monographs, technical reports, duplicate publications, articles not related to the topic and those classified as level VI evidence (based on expert opinions) were excluded from the sample. The strength of evidence of the studies were classified into six levels: I - meta-analysis of multiple controlled and randomized clinical trials; II - individual experimental studies; III - quasi-experimental studies, including nonrandomized controlled trials, matched case-control studies or time series; IV - descriptive or qualitative studies; V - case reports or studies based on high-quality data; and VI - expert opinions.¹⁸

The design stage consisted in the planning and creation of didactic content based on the selected articles, definition of topics, writing of subjects, selection of media, and interface design (layout). Text and drawings were structured into topics.

In the development of the mobile app, the open-source frameworks Apache Cordova 7.0.1 and Angular 4.0, and the programming languages Java Script, HyperText Markup Language 5 (HTML5), and Cascading Style Sheets 3 (CSS3) were used for defining its layout, structure, and navigation. The implementation involved the configuration of tools and digital resources, and building an environment to download the mobile app from the internet to the mobile device.

The mobile app testing included usability, performance, compatibility and functional testing. Usability tests assessed whether the user could intuitively navigate through the interface to achieve an outcome. Performance tests were conducted to verify the responsiveness of the software to each command, such as time taken to initialize, change screens, and complete tasks. Compatibility tests were performed to find out how well the application performs in a given environment, including hardware, network, operating system, among others. Functional tests were conducted to verify that all functions were performing correctly according to design specifications. The testing process was conducted by both the researchers and a system analyst.

The developed mobile app was then validated using the Delphi technique.¹⁹ The content of the instrument was evaluated and judged by specialist doctors and by consulting members of the target population through questionnaires in the search for a consensus among evaluators.¹⁹ Two questionnaires were used in the study, one for the doctors and one for the breast cancer patients. The Delphi technique usually involves two to three rounds or cycles of consultation, but more rounds may be necessary to achieve consensus.¹⁹

The study was based on non-probability convenience samples. The developed mobile app was validated following the Brazilian standard ABNT ISO/IEC 25062: 2011, which recommends a minimum sample of ten participants.²⁰

Specialist doctors and female breast cancer patients were invited via email to participate in the study. The electronic message included an invitation letter and an explanation about the study. Those who agreed to participate also received a message containing the written informed consent to be signed, access to the mobile app, the specific questionnaire, detailed instructions for their effective participation in the study, and were asked to return the completed questionnaire within 30 days after receiving it.

The group of doctors was composed of plastic surgeons, oncologists, breast cancer specialists, gyne-cologists, obstetricians, and radiotherapists with five years or more of experience in the treatment of breast cancer. The doctors who agreed to participate in the study, but failed to return the completed questionnaire within 30 days, were excluded from the sample.

The doctors’ questionnaire was used to assess their opinion on the thematic content, design, sequence of topics, content clarity, ease of reading and vocabulary, and theoretical content adequacy of topics, including pre- and postoperative care, and types of breast surgery, axillary surgery, and breast reconstruction. The items were rated on a four-point Likert-type scale ranging from "inadequate" to "very adequate".

The mobile app was also tested with female breast cancer patients recruited from Santa Casa de Misericórdia de Passos and Regional Cancer Hospital (Minas Gerais, Brazil). Patients who were illiterate and those who accepted to participate in the study, but failed to return the completed questionnaire within 30 days, were excluded from sample.

Before responding to the questionnaire, the selected patients were asked to navigate through the app. The patients’ questionnaire was composed of closed questions assessing their opinions about the sequence of topics, content clarity, usefulness of the images and information provided, ease of reading and understanding, and whether the patient felt motivated to navigate through the application and would indicate it to a friend or relative. The items were rated on a four-point Likert-type scale ranging from "disagree" to "strongly agree".

The content validity index (CVI) was used to measure the proportion of doctors and patients who agreed on certain aspects of the mobile app. The CVI for the doctors was calculated by dividing the number of responses ‘adequate’ and ‘very adequate’ for each item by the total number of responses. The CVI for the patients was obtained by dividing the number of responses "agree" and "strongly agree" for each item by the total number of responses. The overall CVI for a questionnaire is calculated by dividing the sum of all CVI values by the total number of items. In this study, an agreement among doctors equal or greater than 80% (CVI ≥ 0.8) for each item was considered necessary to validate the instrument.²¹ The data were entered into an Excel spreadsheet (Microsoft Corporation, Redwood, WA, USA) and descriptive analysis was conducted.

Results

The developed mobile app, named OncoMasto Cirurgia and registered at the National Institute of Industrial Property in Brazil, has 27 screens and 30 images describing pre- and postoperative care and types of breast cancer surgery. Examples of menu screens showing the user interface are seen in Figure 1.

Figure 1
Examples of menu screens showing the graphical user interface of the mobile app (click on the images or zoom in to see it clearly).

Once the mobile app was developed in Brazilian Portuguese, the images below were translated into English for better understanding.

The first step for the development of the app was an integrative review of the literature: 42,211 articles were identified, by means of crossing two descriptors, therefore, only 31 were included in the analysis (Figure 2). The selected articles offer clarification on doubts and provide knowledge for patients and their families about pre-operative care, types of surgery and post-operative care for breast cancer surgical treatment.

Figure 2
Flowchart of identification, selection and inclusion of integrative literature review studies for application development.

A total of 36 specialist doctors were invited to participate in the validation of the mobile app; 13 of them accepted the invitation and returned the completed questionnaire within 30 days after receiving it. The group was composed of four gynecologists, three clinical oncologists, two breast cancer specialists, two plastic surgeons, one surgical oncologist, and one radiotherapist.

A consensus was achieved among doctors in the first round of consultation, with an excellent CVI of 1.0 per item (Table1), corresponding to 100% of agreement, thus validating the mobile app for use in the medical practice and education on the surgical treatment of breast cancer. The doctors made no suggestions regarding the educational content; only minor grammar and spelling mistakes were detected and corrected.

Table 1
Results from the specialist doctors assessment of the mobile app content, and content validity index (CVI) values for the first round of consultation

Assessed topics a	Inadequate		Partially adequate		Adequate		Very adequate		Total		CVI
	n	%	n	%	n	%	n	%	n	%
Ease of understanding	0	0	0	0	2	15.4	11	84.6	13	100	1.00*
Grammar and spelling	0	0	0	0	3	23.1	10	76.9	13	100	1.00*
Design	0	0	0	0	2	15.4	11	84.6	13	100	1.00*
Sequence of topics	0	0	0	0	0	0	13	100	13	100	1.00*
Ease of reading	0	0	0	0	3	23.1	10	76.9	13	100	1.00*
Content clarity	0	0	0	0	2	15.4	11	84.6	13	100	1.00*
Vocabulary	0	0	0	0	3	23.1	10	76.9	13	100	1.00*
Content adequacy
Preoperative care	0	0	0	0	3	23.1	10	76.9	13	100	1.00*
Breast-conserving surgery	0	0	0	0	4	30.8	9	69.2	13	100	1.00*
Mastectomy	0	0	0	0	4	30.8	9	69.2	13	100	1.00*
Types of breast reconstruction	0	0	0	0	6	46.2	7	53.8	13	100	1.00*
Types of axillary surgery	0	0	0	0	5	38.5	8	61.5	13	100	1.00*
Postoperative care	0	0	0	0	5	38.5	8	61.5	13	100	1.00*
Overall CVI	-	-	-	-	-	-	-	-	-	-	1.00*

• Note: ^aQuestionnaire items were abbreviated to simplify table construction; *Excellent CVI value.

The mobile app was also tested with members of the target population. Twenty-one female breast cancer patients were invited to participate in the study; 19 of them accepted the invitation and returned the completed questionnaire within 30 days after receiving it. Among the patients, nine had primary education, seven had high school education, and three had college education. A consensus was also achieved among the selected patients in the first round of consultation, with an excellent CVI of 1.0 for each item (Table 2).

Table 2
Results from the patient assessment of the mobile app content, and content validity index (CVI) values for the first round of consultation

Assessed topicsa	Disagree		Partially agree		Agree		Strongly agree		Total		CVI
	n	%	n	%	n	%	n	%	n	%
Understood the sequence of topics	0	0	0	0	4	21.1	15	78.9	19	100	1.00*
Understood the provided information	0	0	0	0	3	15.8	16	84.2	19	100	1.00*
The images were useful	0	0	0	0	0	0	19	100	19	100	1.00*
Understood the vocabulary	0	0	0	0	6	31.6	13	68.4	19	100	1.00*
Learned something new	0	0	0	0	0	0	19	100	19	100	1.00*
Would indicate the app to a friend or relative	0	0	0	0	0	0	19	100	19	100	1.00*
Other patients will understand the app content	0	0	0	0	0	0	19	100	19	100	1.00*
Felt motivated to navigate through the app	-	-	-	-	1	5.3	18	94.7	19	100	1.00*
Overall CVI	-	-	-	-	-	-	-	-	-	-	1.00*

• Note: ^aQuestionnaire items were abbreviated to simplify table construction; *Excellent CVI value.

Discussion

In daily clinical practice, the authors have observed that breast cancer patients may find it difficult to understand all the information provided by health professionals about their treatment and self-care. Faced with doubts about the procedures, patients search the internet for answers because they consider it an important source of health information. However, the internet provides access to a vast number of contents, which may be inaccurate, such as unreliable websites publishing false information for profit or motivated by different reasons.

Due to patients’ difficulty in understanding the information provided during the medical consultation, scarcity of reliable online information, and the wide-spread use of the internet, a freely available mobile app was developed for the lay public, containing evidence-based guidelines and information on the surgical treatment of breast cancer.

A participatory approach was used in the situational diagnosis of the knowledge of breast cancer patients about surgical treatments for breast cancer. The participation of breast cancer patients was fundamental for identifying the main topics that represented the needs of this population. The development of a mobile app has to take into consideration the characteristics, education level, and needs of the population for which it is intended. All this information should be obtained through a participatory, communicative and collective approach.^22-24

Educational technologies in medical and health sciences help improve patients' knowledge and satisfac-tion. The information content of educational resources has to be simple, clear, and easy to understand.^14,25 Illustrations and images enhance comprehension of texts, add background information, attract the readers’ attention, and maintain their interest in reading. The layout and design also make the instrument more attractive to the user and easier to read.^14,26-28

The mobile app developed in this study was validated by doctors with experience in breast cancer treatment and tested by members of the target population. The participation of experienced professionals from different specialties in the valida-tion process was a positive aspect of the study, as the diversity of viewpoints may contribute to the quality of the work produced and enhances the effectiveness of the evaluation of educational materials.²⁴ Thus, a multidisciplinary group of doctors with experience in assistance, research, and teaching was essential for the validation of the mobile app.^23,24,28

The testing of an application by members of the target population contribute to improve the users’ learning experience and understanding of the content, making the educational tool adequate and representative of the needs of the population for which it is intended.^14,28 A positive evaluation of an educational technology indicates that the instrument can be used as a means of information and education.^14,26,27

The specialist doctors made no suggestion for content improvement, except for minor grammar and spelling mistakes, which were corrected. Evaluators' suggestions help increase the quality of an instrument, contributing to the ease of understanding the content, enhancing the learning experience, and motivating its use and acceptance by the target population.^26,29-31

The perspective of the study will be the development of the application in the English language.

Conclusion

A mobile app, named “Oncomasto Cirurgia App”, was developed after an integrative literature review and tested for usability, performance, compatibility, and functionality. The app was validated by specialist doctors and by breast cancer patients. Consensus was achieved in the first round of consultation, with 100% agreement both among the specialist doctors and the patients, validating it for use in clinical practice and education on the surgical treatment of breast cancer, with an excellent overall CVI value.

Acknowledgments

The first author thanks CNPq for the Productivity Scholarship in Technological Development and Inno-vative Extension - DT-2.

References

• ¹
  Firouzbakht M, Hajian-Tilaki K, Moslemi D. Analysis of quality of life in breast cancer survivors using structural equation modelling: the role of spirituality, social support and psychological well-being. Int Health. 2020;12(4):354-63. DOI
  » DOI
• ²
  Sousa SMMT, Carvalho MGFM, Santos Jr LA, Mariano SBC. Access to treatment of women with breast cancer. Saude Debate. 2019;43(122):727-41. DOI
  » DOI
• ³
  Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394-424. DOI
  » DOI
• ⁴
  Instituto Nacional de Câncer José Alencar Gomes da Silva (INCA). Estimativa 2020: Incidência de câncer no Brasil. Rio de Janeiro: INCA; 2019. Full text link
  » Full text link
• ⁵
  Feijó AM, Linck CL, Viegas AC, Santos BP. Os caminhos de cuidado das mulheres com diagnóstico de câncer de mama. Av Enferm. 2016;34(1):58-68. DOI
  » DOI
• ⁶
  Magalhães PAP, Loyola EAC, Dupas G, Borges ML, Paterra TSV, Panobiaco MS. The meaning of labor activities for young women with breast neoplasms. Texto Contexto Enferm. 2020; 29:e20180422. DOI
  » DO
• ⁷
  Tedesco D, Loerzel V. Breast reconstruction: impact of patient-centered, expectations-based education on women undergoing reconstructive surgery after mastectomy. Clin J Oncol Nurs. 2020;24(2):186-94. DOI
  » DOI
• ⁸
  Charbonneau DH, Hightower S, Katz A, Zhang K, Abrams J, Senft N, et al. Smartphone apps for cancer: a content analysis of the digital health marketplace. Digit Health. 2020;6: 2055207620905413. DOI
  » DOI
• ⁹
  Jesus AS, Ajala SR, Saldanha CA, Spexoto MCB. Factors associated with health-related quality of life of cancer patients undergoing clinical treatment. Rev Bras Cancerol. 2019;65(2):e-15395. DOI
  » DOI
• ¹⁰
  Potdar R, Thomas A, DiMeglio M, Mohiuddin K, Djibo DA, Laudanski K, et al. Access to internet, smartphone usage, and acceptability of mobile health technology among cancer patients. Support Care Cancer. 2020;28(11):5455-61. DOI
  » DOI
• ¹¹
  Milne-Ives M, Lam C, De Cock C, Van Velthoven MH, Meinert E. Mobile apps for health behavior change in physical activity, diet, drug and alcohol use, and mental health: systematic review. JMIR Mhealth Uhealth. 2020;8(3):e17046. DOI
  » DOI
• ¹²
  Galvão ECF, Püschel VAA. Multimedia application in mobile platform for teaching the measurement of central venous pressure. Rev Esc Enferm USP. 2012;46(Spe):107-15. DOI
  » DOI
• ¹³
  Pereira FGF, Silva DV, Sousa LMO, Frota NM. Building a digital application for teaching vital signs. Rev Gaucha Enferm. 2016;37(2):e59015. DOI
  » DOI
• ¹⁴
  Alves NF, Salomé GM. App “SICKSEG” in mobile platforms for the prevention of skin injuries. J Nurs UFPE. 2020;14: e244152. DOI
  » DOI
• ¹⁵
  Salomé GM, Bueno JC, Ferreira LM. Aplicativo multimídia em plataforma móvel para tratamento de feridas utilizando fitoterápicos e plantas medicinais. Rev Enferm UFPE. 2017;11 (Supl. 11):4579-88. Full text link
  » Full text link
• ¹⁶
  Cruz NS, Soares DKS, Bernardes A, Gabriel CS, Pereira MCA, Évora YDM. Nursing undergraduates' technical competence in informatics. Rev Esc Enferm USP. 2011;45(Spe):1595-9. DOI
  » DOI
• ¹⁷
  Salomé GM, Ferreira LM. Developing a mobile app for prevention and treatment of pressure injuries. Adv Skin Wound Care. 2018;31(2):1-6. DOI
  » DOI
• ¹⁸
  Stetler CB, Morsi D, Rucki S, Broughtonb S, Corriganb B, Fitzgeraldb J, et al. Utilization-focused integrative reviews in a nursing service. Appl Nurs Res. 1998;11(4):195-206. DOI
  » DOI
• ¹⁹
  Mancussi e Faro AC. Técnica Delphi na validação das intervenções de enfermagem. Rev Esc Enferm USP. 1997;31(2): 259-73. DOI
  » DOI
• ²⁰
  Associação Brasileira de Normas Técnicas (ABNT). Requisitos e avaliação da qualidade de produto de software (SQuaRE) - Formato comum da indústria (FCI) para relatórios de teste de usabilidade. Rio de Janeiro: ABNT; 2011. Full text link
  » Full text link
• ²¹
  Wynd CA, Schmid, B, Schaefer MA. Two quantitative approaches for estimating content validity. West J Nurs Res. 2003;25(5):508-18. DOI
  » DOI
• ²²
  Albuquerque AFLL, Pinheiro AKB, Linhares FMP, Guedes TG. Technology for self-care for ostomized women’s sexual and reproductive health. Rev Bras Enferm. 2016;69(6):1099-106. DOI
  » DOI
• ²³
  Rodrigues LN, Santos AS, Gomes PPS, Silva WCP, Chaves EMC. Construction and validation of an educational booklet on care for children with gastrostomy. Rev Bras Enferm. 2020; 73(3):e20190108. DOI
  » DOI
• ²⁴
  Lima ACMACC, Bezerra KC, Sousa DMN, Rocha JF, Oriá MOB. Development and validation of a booklet for prevention of vertical HIV transmission. Acta Paul Enferm. 2017;30(2):181-9. DOI
  » DOI
• ²⁵
  Normansell R, Kew KM, Mathioudakis AG. Interventions to improve inhaler technique for people with asthma. Cochrane Database Syst Rev. 2017;3(3):CD012286. DOI
  » DOI
• ²⁶
  Hortense FTP, Bergerot CD, Domenico EBL. Construction and validation of clinical contents for development of learning objects. Rev Bras Enferm. 2018;71(2):306-13. DOI
  » DOI
• ²⁷
  Nascimento MHM, Teixeira E. Educational technology to mediate care of the "kangaroo family" in the neonatal unit. Rev Bras Enferm. 2018;71(Suppl. 3):1290-7. DOI
  » DOI
• ²⁸
  Alexandre DS, Alpes MF, Reis ACMB, Mandrá PP. Validation of a booklet on language developmental milestones in child-hood. Rev CEFAC. 2020;22(2):e16219. DOI
  » DOI
• ²⁹
  McGilton KS. Development and psychometric evaluation of supportive leadership scales. Can J Nurs Res. 2003;35(4):72-86. Full text link
  » Full text link
• ³⁰
  Brandão ES, Santos I, Lanzillotti RS. Validation of an instrument to assess patients with skin conditions. Act Paul Enferm. 2013;26(5):460-6. Full text link
  » Full text link
• ³¹
  Carvalho MRF, Salomé GM, Ferreira LM. Construction and validation of algorithm for treatment of pressure injury. J Nurs UFPE. 2017;11(Suppl. 10):4171-83. Full text link
  » Full text link

Edited by

Associate editor:

Eduardo Mendonça Scheeren

Publication Dates

• Publication in this collection
  15 July 2022
• Date of issue
  2022

History

• Received
  27 July 2021
• Reviewed
  06 May 2022
• Accepted
  27 June 2022