Development and content validation of an instrument for evaluating competencies in Pediatric Surgery Residency

Desenvolvimento e validação de conteúdo de instrumento para avaliação das competências na residência em cirurgia pediátrica

Abstract:

Introduction:

a specific instrument is needed to monitor the development of competencies in pediatric surgery during the residency.

Objective:

to develop an instrument in conformity with the “Milestone Project”, using the competencies determined by the Brazilian Association of Pediatric Surgery, for use in the Pediatric Surgery Residency in Brazil.

Method:

the research was developed in three stages: the development of the initial instrument, qualification with a focal group of experts, and instrument evaluation by Brazilian pediatric surgeons in order to validate and quantify the instrument’s acceptance.

Result:

The initial instrument was created with 4 competencies and 13 sub-competencies, each with 5 levels of assessment. Four experts performed the initial qualification, which resulted in 44 adaptations, and the instrument was finalized with 4 competencies subdivided into 10 sub-competencies, each with 5 levels of assessment. Subsequently, the instrument was evaluated by the Brazilian Pediatric Surgery Group and the Brazilian Pediatric Urology Group. There were 40 expert responses, with a total of 2394 positive responses from the 50 assessment items. The instrument had a general acceptance of 91.2%, being considered applicable (96.7%), reproducible (93.3), relevant to the covered topics (96%), technically (93.6%) and theoretically appropriate (93.3%), reliable (85.5%), and dependable (79.8%).

Conclusion:

an instrument was developed to assist in the assessment of competencies developed during residency in pediatric surgery in conformity with the Milestone Project. This instrument has been validated by experts and considered applicable, reproducible, relevant, technically and theoretically adequate, reliable and dependable.

Keywords:
Medical Competencies; Pediatric Surgery; Assessment Instrument

Resumo:

Introdução:

Uma adequada formação na residência em cirurgia pediátrica deve avaliar e acompanhar constantemente o desenvolvimento de competências e, para isso, necessita de um instrumento específico como ferramenta de avaliação.

Objetivo:

Este estudo tem como objetivo apresentar um instrumento de avaliação nos moldes do “Milestone Project” com base nas competências determinadas pela Associação Brasileira de Cirurgia Pediátrica, para uso nos programas de residência médica em cirurgia pediátrica no Brasil.

Método:

Adotaram-se as seguintes etapas: desenvolvimento do instrumento, qualificação de um grupo de especialistas e avaliação do instrumento por cirurgiões pediatras brasileiros, a fim de validar e quantificar a aceitação do instrumento quanto à fidedignidade, confiabilidade, aplicabilidade, reprodutibilidade, relevância dos temas abordados e adequação dos pontos de vista técnico e teórico.

Resultados:

O instrumento inicial possuía quatro competências gerais e 13 subcompetências específicas, com cinco níveis de avaliação. Quatro experts realizaram a qualificação que gerou 44 adaptações, finalizando o instrumento com quatro competências gerais subdividias em dez subcompetências, com cinco níveis avaliativos. Sequentemente, o instrumento foi avaliado pelo Grupo Brasileiro de Cirurgia Pediátrica e pelo Grupo Brasileiro de Urologia Pediátrica. Houve 40 respostas de especialistas, 2.394 respostas positivas dos 50 itens. O instrumento teve aceitação de 91,2% e foi considerado aplicável (96,7%), reprodutível (93,3), relevante (96%), tecnicamente adequado (93,6%), teoricamente adequado (93,3%), confiável (85,5%) e fidedigno (79,8%).

Conclusão:

Esse instrumento nos moldes do “Milestone Project” foi validado por cirurgiões pediatras e considerado aplicável, reprodutível, relevante, adequado, sob os pontos de vista técnico e teórico, confiável e fidedigno.

Palavras-chave:
Competências Médicas; Cirurgia Pediátrica; Instrumento de Avaliação

INTRODUCTION

The current approach to health education is based on the competencies that will be developed during the physician and the specialist’s training. Competency can be defined as the correlation between the use of knowledge, clinical and technical skills, communication skills and reasoning, reflection and professionalism in daily clinical practice for the benefit of people or the community¹.

In Brazil, medical training follows the guidelines of the National Curriculum Guidelines (DCNs), which incorporate the most modern concepts of adult education, and establish the general and specific competencies to be developed during training, based on the desired profile of the recently graduated students².

Similarly, also in medical residency, competencies continue to be developed and evaluated. According to the National Commission of Medical Residency (CNRM), the Resident Physician will be periodically evaluated in their training through written, oral, practical or performance tests, and also through attitude scales, which include attributes such as: ethical behavior, relationship with the health team and the patient, interest in the activities, and other criteria of the institution’s Medical Residency Commission (COREME)^1),(3.

For the Brazilian Association of Pediatric Surgery (ABCP), the training in pediatric surgery aims to educate the specialist, previously trained as a general surgeon, to work as a pediatric surgeon in the following areas: neonatal, general, and urological pediatric surgery, pediatric oncologic, trauma, and video-assisted surgery. With this training, the pediatric surgeon will adequately advise families and peers in other specialties, define and perform clinical and surgical procedures, use technical resources, and perform appropriate interprofessional integration⁴.

With the institution of expected medical competencies, considering the DCNs, COREME and ABCP, tools and techniques can be developed to carry out the physician’s training, and the assessment of the acquisition of these competencies. The Milestone Projects of the North American Accreditation Council for Graduate Medical Education (ACGME) is a project developed with the American Society of Specialists for semiannual and/or annual monitoring of the developmental milestones of knowledge, skills and attitudes during specialist training^5)-(8.

In this scenario, the objective of this research is to present an assessment instrument that can be used as a tool to evaluate the acquisition of competencies throughout the pediatric surgery residency in Brazil, following CNRM regulations and the Brazilian Association of Pediatric Surgery guidelines^3),(4.

METHOD

This is a descriptive exploratory methodological study that combined the investigation of methods used to obtain, organize and analyze data, and discusses the development and validation of an instrument that is applicable, dependable, reproducible, reliable, relevant and appropriate from the technical and theoretical points of view.

RESULTS

This research was developed in three stages, designed according to AMEE Guide n. 87¹⁰. The AMEE Guides cover education topics related to medical and healthcare professions and provide information, practical advice and support. They are designed to be individually used by teachers to inform about their practice and are widely used in staff development programs.

Stage 1: Identification of the competencies required for the training of pediatric surgeons, according to the regulations of the Ministry of Education and the Brazilian Association of Pediatric Surgery. It is a survey of General Medical Competencies, Medical Surgery Competencies, and Medical Competencies in Pediatric Surgery through the analysis of international documents, such as CanMEDS Framework, Paediatric Surgical Education and Training Regulations, and the Milestone Projects^3)-(5. The development of the assessment instrument is based on the Milestone Projects, with the stratification of competencies being carried out by residency training period, to be applied at the end of the first 6 months, at the end of the first and second years, and at the end of residency⁵.

Stage 2: Qualification of the instrument created using a semi-structured questionnaire, by a group of experts from the research institution, selected based on the following criteria: being a pediatric surgery specialist and a pediatric surgery residency service preceptor.

Stage 3: Content validation and quantification of expert acceptance. The assessment instrument was adapted to the Survey Monkey ^(R) program, as well as the Informed Consent Form, in the form of a semi-structured questionnaire, in which each acceptance item has a binomial answer (YES or NO), and space for considerations by the survey respondent. The invitation to participate in the research was sent by e-mail from the national pediatric surgery discussion groups: the Brazilian Pediatric Surgery Group (GBCIPE) and the Brazilian Pediatric Urology Group (UROPED), to quantify the instrument acceptance and evaluate dependability, reliability, applicability, reproducibility, relevance of the addressed topics, and adequacy from the technical and theoretical points of view.

Subsequent documental analysis of step 1, stage 1, called “Instrument Development”, comprising 13 sub-competencies to be developed during training were established^4)-(6.

Medical Expertise:

1. Bases of pediatric surgical clinic;

2. Diseases of general and neonatal surgical pediatrics;

3. Video surgery in pediatric surgery;

4. Pediatric genitourinary tract surgery;

5. Pediatric surgical diseases of the chest.

Continuing Medical Education:

1. Active search for the updating and self-organization of the study;

2. Critical evaluation of medical literature and evidence-based medicine.

Professionalism:

1. Compassion, integrity and respect for others;

2. Accountability and responsiveness to the needs of patients, society and the profession;

3. Respect for the individuality and autonomy of the patient and family;

4. Legal knowledge.

Medical Career Management:

1. Communication with the patient and family;

2. Communication with physicians and other health professionals;

3. Preparation for teamwork;

4. Informed consent;

5. Practice-based quality and safety improvement processes.

In stage 2, named “Instrument Qualitative Assessment”, of 7 selected experts, 4 effectively sent their considerations on the assessment instrument, totaling 44 considerations and suggestions for changes. At the end, the competencies were as follows:

Medical Expertise:

1. Bases of Pediatric Surgical Clinic;

   1. General principles of pediatric surgery and preoperative preparation;

   2. Hydration, analgesia, soft tissue infection, healing, judicious use of antimicrobials;

   3. Venous access, gastrostomy, colostomy, surgical preparation of the newborn;

   4. Complex drug management, management of postoperative complications, and co-management of critically-ill patients;

   5. Complex case management and application of evidence-based medicine.

2. General and Neonatal Pediatric Surgery:

   1. Pre- and postoperative care of appendicitis, hernias, hemangiomas, lymphangiomas, acute abdomen, and pediatric trauma;

   2. Performance of the surgical procedures in item “a”, as well as pre- and postoperative care of: branchial arches, ovarian diseases, childhood abuse, intestinal invagination, Meckel’s diverticulum, cholelithiasis, and hypertrophic pyloric stenosis;

   3. Performance of surgical procedures in item “b” and application of pre- and postoperative care of: congenital megacolon, gastroesophageal reflux disease, intestinal atresia, necrotizing enterocolitis, intestinal malrotation, neuroblastoma, gonadal tumors and teratoma;

   4. Performance of the surgical procedures in item “c” and application of pre- and postoperative care, as well as surgical management of: esophageal atresia, anorectal anomaly, omphalocele, gastroschisis, biliary atresia, choledochal cyst, and abdominal tumors;

3. Video-assisted surgery in Pediatric Surgery;

   1. Basic knowledge on video surgery equipment, and the clinical implications of video-assisted surgery

   2. Assembly of the laparoscopic equipment, correct portal insertion, identification of clinical implications of laparoscopic surgery in the neonatal patient;

   3. Performance of diagnostic procedures: laparoscopy, thoracoscopy, biopsies, appendectomy, identification of surgical complications;

   4. Performance of complex procedures such as treatment of gastroesophageal reflux, cholecystectomy, correction of intestinal malrotation, treatment of surgical complications;

   5. Innovative approaches in video-assisted surgery.

4. Pediatric Urology:

   1. Diagnosis and surgical management: phimosis, acute scrotum, adhesions of labia minora (synechia vulvae), cystic scrotal lesions;

   2. Pre- and postoperative care of: glans hypospadias, interlabial masses, hematuria, penile malformations, nocturnal urinary incontinence, nephrectomy, cryptorchidism;

   3. Surgical procedures in item “b” and application of pre- and postoperative care of: vesicoureteral reflux, hydronephrosis, renal cystic disease, ureteropelvic junction stenosis, megaureter, renal duplication, posterior urethral valve, and heminephrectomy;

   4. Procedures in item “c” and application of pre- and postoperative care of: penoscrotal hypospadias, epispadias, intersex, nocturnal enuresis, neurogenic bladder, bladder enlargement. Knowledge on the management of bladder and cloacal exstrophy;

   5. Procedures in item “d” and management of kidney transplantation, neovagina, neopenis and exstrophies.

5. Chest surgery in pediatrics:

   1. Diagnosis and management of laryngeal and tracheal obstruction, performance of adequate airway approach;

   2. Pre- and postoperative management of: pneumothorax, pleural effusion, empyema, and tracheostomy.

   3. Procedures in item “b”, and pre- and post-operative care of: pulmonary abscess, pulmonary necrosis, and pulmonary decortication;

   4. Surgical procedure for the treatment of vascular rings and slings, bronchogenic cyst, bronchiectasis, mediastinal masses, congenital lobar emphysema, cystic adenomatoid malformation, pulmonary sequestration, and congenital diaphragmatic hernia;

   5. Complex case management and application of evidence-based medicine.

Continuing Medical Education:

1. Continuing education and critical evaluation of the literature:

   1. Critical evaluation of the literature;

   2. Identification of resources available in the literature, identification of the limits of technical knowledge, use of a study plan;

   3. Application of evidence-based medicine to the treatment plan;

   4. Comparison of their results with medical literature;

   5. Contribution to the literature regarding the management and treatment of complications of complex cases (only).

Professionalism:

1. Responsibility with the patient and Teamwork:

   1. Punctuality, commitment, ethical principles;

   2. Adequate recording in the patient’s file;

   3. Compassion, empathy and respect for patients and families, colleagues, and other team members;

   4. Application of standards in patient care, and reception of feedback on their attitudes;

   5. Participation in institutional committees, and presence of ethical attitude;

   6. Leadership, recognition of ethical implications, and management of complex issues.

Medical Career Management:

1. Communication with patient and family;

   1. Listening ability and good communication in the clinical routine;

   2. Checking Patient Understanding, provision of time for questions, and explanation of care plan;

   3. Coping with stressful situations, bad news communication, and socio-cultural knowledge involved in different daily situations;

   4. Knowledge on protocols for communication of bad news or death, knowledge on how to report medical error and use of a management plan for medical error management; educating patients and guardians regarding patient’s clinical problems;

   5. Participating and assisting in solving challenging situations.

2. Communication with physicians and other healthcare professionals and teamwork:

   1. Understanding the importance of teamwork;

   2. Demonstrating knowledge of their roles in the patient care team;

   3. Behaving effectively in a multiprofessional team;

   4. Working with interdisciplinarity for better patient treatment outcome, responding appropriately to consultation requests;

   5. Effectively guiding other professionals;

3. Informed consent:

   1. Understanding the importance of the informed consent;

   2. Involving patients and decision makers, getting informed consent for simple procedures;

   3. Using appropriate language, and an interpreter if necessary, for shared decision-making, and obtaining consent for complex procedures;

   4. Participating in multidisciplinary and conference teams;

   5. Defining complex decision models.

In stage 3 of the research, called “Content Validation and Quantification of Expert Acceptance”, at the deadline for reception of the answers, the online questionnaire was evaluated by 40 pediatric surgeons, comprising a total of 2394 answers out of 50 competencies.

The total number of affirmative answers, with the respective percentage values in each evaluation item of the evaluation instrument is shown in Table 1.

Table 1
Instrument evaluation, according to investigated criteria.

Assessed Criterion	Overall Instrument assessment
Applicable	379 (96.7%)
Dependable	296 (79.8%)
Reproducible	347 (93.3%)
Reliable	318 (85.5%)
Relevant	356 (96.0%)
Appropriate from a theoretical point of view	349 (93.3%)
Appropriate from a technical point of view	349 (93.6%)
Overall	2394 (91.2%)

• Source: Study Data, 2016.

After an overall analysis of the results, the instrument was considered applicable in 96.7% of the answers, reproducible in 93.3%, relevant in relation to the covered topics in 96%, technically adequate in 93.6%, and appropriate from a theoretical point of view in 93.3%.

DISCUSSION

To conclude stage 1, a survey of the overall and specific objectives of the residency programs in Pediatric Surgery was made, based on the definition of medical competencies, and following the regulations of the Ministry of Education, and according to the Brazilian Association of Pediatric Surgery⁴.

The construction was performed by similarity of meanings of the items, which allowed the synthesis and creation of an initial list with 4 competencies: Medical Expertise, Continuing Medical Education, Professionalism, and Medical Career Management. Subsequently, the minimum requirements for the training of the pediatric surgeon were analyzed, so that no essential items were excluded, and items that were repeated or complementary were not unnecessarily included, which would make the instrument too long, hindering adherence to the research.

The model instrument: “The Pediatric Surgery Milestone Project” provides a framework for competency milestones to be developed by the resident physician, to be evaluated every 6 months. It consists of 22 questions, distributed in progression levels from 1 to 5. For each period, the evaluation and reporting will imply the selection of the milestone that best represents the performance of each resident’s current attributes. Progressing from level 1 to level 5 means moving from beginner to expert. The selection of a level indicates that the resident can substantially perform the principles assignments of that level⁵. The training of the Brazilian pediatric surgeon includes several specific technical competencies, which are not covered by the American instrument; therefore, adaptation, rather than translation of the model instrument, was chosen. To better adapt the instrument to the Brazilian reality, it was suggested that the competency levels should be assessed at the end of the first 6 months of specialization, at the end of the first and second years, and at the end of the third year.

During the first year of training, two evaluations are expected: the first one taking place at the end of the sixth month, with a diagnostic function, because general surgeons from different training services may have developed different competencies during their initial training. The following evaluations take place annually and have a formative nature, because they follow the acquisition of competencies that are becoming increasingly complex, from the theoretical and technical points of view, as the specialization takes place^9),(11. The interval between the evaluations is now annual, because the diseases that are part of the second and third-year resident’s assessment are less prevalent.

As some residencies offer an additional year of specialization, focused on the training of the ‘super specialist’, such as pediatric urology, pediatric liver transplantation, neonatal surgery, an additional column was created in the evaluation timeline, being related to the competencies expected only from fellows.

At the end of the first stage of the research, the instrument had 4 general competencies, subdivided into 13 specific sub-competencies, divided in 5 levels, evaluated at the end of the sixth month, at the end of the first, second, and third years, and an additional progression level, expected only from fellows, but which some third-year residents may reach. Thus, there was a total of 65 levels of competency assessment, and of these, 25 correspond to competencies that are specific of the specialty. Therefore, it is considered that the first 2 steps compared to those suggested by the AMEE Guide n. 87 were met at this stage^5),(10.

In stage 2, an invitation was sent by email to 7 selected experts. Of the 6 who agreed to participate, 4 effectively sent their considerations on the assessment instrument within the recommended time frame for the study, totaling 44 considerations and suggestions for changes. Only 4 evaluators were accepted, because they had more than 80% agreement on the suggested changes.

At the end of the second stage of the research, it can be considered that steps 3, 4, 5 and 6 of AMEE Guide n. 87 for construction were followed, and the competencies were stratified into 4 overall competencies, subdivided into 10 sub-competencies. Each of them has 5 assessment levels, thus comprising 50 assessment items¹⁰.

In stage 3, the instrument was adapted to the Survey Monkey ^(R) program for online application. In this evaluation phase of the pilot project - last step of the AMEE guide n. 87 - the research was sent by email for acceptance of the research participation, along with the Informed Consent Form, by the Brazilian Group of Pediatric Surgeons and the Brazilian Group of Pediatric Urologists; specialists from all over Brazil had the opportunity to judge and evaluate whether the proposed items or sub-competencies were applicable, dependable, reproducible, reliable, relevant, technically and theoretically appropriate.

The results of step 3 suggests that the instrument can be applied in other pediatric surgery training services as a form of continuous and serial evaluation. Regarding reliability, the questionnaire had an 85% acceptance, suggesting that the respondents of the instrument agree that the items evaluated are appropriately approached. Regarding dependability (79.8%), the research instrument had a lower acceptance rate. When an item is considered dependable, it reflects the reality and can be considered as true. This can be translated by the fact that 79.8% of those who agree that the developed instrument represents the reality of the pediatric surgeon’s training, by training period. As the developed instrument was validated in a hospital with a long tradition in training pediatric surgeons, this can lead to the interpretation that the specialist training may not occur uniformly in Brazil.

It is worth mentioning that the present study contributed to the creation of the pediatric surgery competence, published em 2020¹⁶.

CONCLUSIONS

Given the results of this research, an assessment instrument was developed, based on the Milestone Projects, following the objectives for training pediatric surgeons of the National Commission of Medical Residency and the Brazilian Association of Pediatric Surgery, for application in medical residency programs in Pediatric Surgery in Brazil.

The developed instrument was qualitatively validated regarding its content, with experts in the field, and the relevant changes were made, so that it can be continuously and serially applied to pediatric surgery resident physicians.

This instrument has been considered applicable, reproducible, relevant, and adequate from a technical and theoretical point of view. Regarding dependability and reliability, the instrument had a lower rate of positive acceptance in the overall assessment of the 4 major competencies.

Thus, it can be inferred that it is an important tool for formative evaluation and for determining it; it can be widely applied, helping to determine which training milestones have already been reached, guiding both residents and preceptors on the status of competence development during training.

Thus, it is a guiding tool for different pediatric surgery residency programs to promote effective evaluation and, consequently, to provide linear and homogenous competent training during the Pediatric Surgery Medical Residency programs in Brazil, as well as to guide the necessary adjustments to the services that offer residency in their specialty.

REFERENCES

• ¹
  Lemle A. A Questão da Pesquisa nos Hospitais Universitários. Ciência e Cultura 1989; 41(4)329-38.
• ²
  Brasil. Ministério da Educação. Conselho Nacional de Educação. Câmara de Educação Superior. Resolução CNE/CES nº3 de 20 de junho de 2014. Institui diretrizes curriculares nacionais do curso de graduação em Medicina. Diário Oficial da União. Brasília, 23 de jun. 2014; Seção 1, p.8-11.
• ³
  Brasil. Ministério da Educação. Comissão Nacional de Residência Médica. Resolução CNRM 02/2006. [access in 5 jan 2017]. Available from:Available from:http://portal.mec.gov.br/escola-de-gestores-da-educacao-basica/247-programas-e-acoes-1921564125/residencia-medica-2137156164/13079-resolucao-residencia-medica
  » http://portal.mec.gov.br/escola-de-gestores-da-educacao-basica/247-programas-e-acoes-1921564125/residencia-medica-2137156164/13079-resolucao-residencia-medica
• ⁴
  Associação Brasileira de Cirurgia Pediátrica. Requisitos Mínimos dos Programas de Residência Médica. [access in 28 jun 2021]. Available from: Available from: https://cipe.org.br/novo/matriz-de-competencias-dos-programas-de-residencia-medica-em-cirurgia-pediatrica-e-publicada-no-diario-oficial-da-uniao/
  » https://cipe.org.br/novo/matriz-de-competencias-dos-programas-de-residencia-medica-em-cirurgia-pediatrica-e-publicada-no-diario-oficial-da-uniao/
• ⁵
  Accreditation Council of Graduate Medical Education. Milestones. [access in 5 jan 2017]. Available from: Available from: http://www.acgme.org/What-We-Do/Accreditation/Milestones/Overview
  » http://www.acgme.org/What-We-Do/Accreditation/Milestones/Overview
• ⁶
  Accreditation Council of Graduate Medical Education. A Product of Joint Iniciative of the ACGME. [access in 18 feb 2010]. Available from: Available from: http://www.acgme.org/outcome/assess/toolbox.pdf
  » http://www.acgme.org/outcome/assess/toolbox.pdf
• ⁷
  Accreditation Council of Graduate Medical Education. Core Competencies. [access in 5 jan 2017]. Available from: Available from: http://www.ecfmg.org/echo/acgme-core-competencies.html
  » http://www.ecfmg.org/echo/acgme-core-competencies.html
• ⁸
  Accreditation Council of Graduate Medical Education. Toolbox of Assessment Methods. [access in 5 jan 2017]. Available from: Available from: http://www.acgme.org/Outcome/assess/Toolbox.pdf
  » http://www.acgme.org/Outcome/assess/Toolbox.pdf
• ⁹
  Capel LMCM, Aparecida FC. Finalidades e função da avaliação na formação de enfermeiros. Rev. bras. enferm. 2007;60(6)641-5.
• ¹⁰
  Artino AR Jr, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Medical Teacher. 2014;36(6)463-74.
• ¹¹
  Bollela RV, Machado JLM. O currículo por competências e sua relação com as diretrizes curriculares nacionais para graduação em medicina. Science in Health. 2010;1(2)126-42.
• ¹²
  Brandão HP. Mapeamento de Competências: métodos, técnicas e aplicações em gestão de pessoas. São Paulo: Atlas, 2012.
• ¹³
  Carbone PP, Brandão HP, Leite JBD, Vilhena RM. Gestão por competências e gestão do conhecimento. Rio de Janeiro: Fundação Getúlio Vargas, 2005.
• ¹⁴
  Miller GE. The assessment of clinical skills/competence/performance. Academic medicine.1990; 65(9)63-7.
• ¹⁵
  Ruas R, Comini GM. Aprendizagem e Desenvolvimento de Competências: Articulando Teoria e Prática em Programas de Pós-graduação em Formação Gerencial. Cad. EBAPE. 2007;(5)1-14.
• ¹⁶
  Brasil. Ministério da Educação. Comissão Nacional de Residência Médica. Resolução CNRM 07/2020. [access in 5 jan 2017]. Available from:Available from:https://www.in.gov.br/en/web/dou/-/resolucao-cnrm-n-7-de-30-de-dezembro-de-2020-297625315
  » https://www.in.gov.br/en/web/dou/-/resolucao-cnrm-n-7-de-30-de-dezembro-de-2020-297625315

• 2
  Evaluated by double blind review process.

• SOURCES OF FUNDING

  The authors declare no sources of funding.

Edited by

Chief Editor: Rosiane Viana Zuza Diniz. Associate Editor: Pedro Tadao Hamamoto Filho.

Publication Dates

• Publication in this collection
  16 July 2021
• Date of issue
  2021

History

• Received
  26 Jan 2021
• Accepted
  08 June 2021