Health evaluation: health assessment for people with ostomies

Evaluación de salud: evaluación de salud para las personas ostómicas

ABSTRACT

Objectives:

to perform an evaluation study of the Ostomy Care Service in Brazil.

Methods:

a qualitative methodology was used, with a collaborative approach of the participants using the service, based on the seven elements proposed by Thurston and Ramaliu, which include service modeling and evaluation model proposal. Document analysis, direct observation, consensus workshops, and interviews with key informants were used.

Results:

the description and construction of the theoretical-logical model of the ostomy care service, the development of evaluative questions, the construction of the analysis and judgment matrix, and the evaluation proposal were presented.

Final Considerations:

it was found that it is feasible to systematically evaluate the service based on its structural elements, such as its theory, material and human resources. The proposed evaluation aims to provide support for improving the quality of care.

Descriptors:
Health Evaluation; Ostomy; Public Policy; Interdisciplinary Placement; Health Services

RESUMEN

Objetivos:

realizar un estudio de evaluación del Servicio de Atención de Ostomía en Brasil.

Métodos:

se utilizó una metodología cualitativa, con un enfoque colaborativo de los actores involucrados en el servicio, basado en los siete elementos propuestos por Thurston y Ramaliu, que incluyen el modelo de servicio y la propuesta de modelo de evaluación. Se utilizaron análisis de documentos, observación directa, talleres de consenso y entrevistas con informantes clave.

Resultados:

se presentaron la descripción y construcción del modelo teórico-lógico del servicio de atención de ostomía, la elaboración de las preguntas evaluativas, la construcción de la matriz de análisis y juicio y la propuesta de evaluación.

Consideraciones Finales:

se encontró que es factible evaluar sistemáticamente el servicio a partir de elementos estructurales, como su teoría, materiales y recursos humanos. La evaluación propuesta tiene como objetivo proporcionar apoyo para mejorar la calidad de la atención.

Descriptores:
Evaluación en Salud; Estomía; Política Pública; Prácticas Interdisciplinarias; Servicios de Salud

RESUMO

Objetivos:

realizar Estudo de Avaliabilidade do Serviço de Atenção às pessoas com ostomia no Brasil.

Métodos:

empregou-se metodologia qualitativa, com abordagem colaborativa dos atores envolvidos no serviço, baseada nos sete elementos propostos por Thurston e Ramaliu, que incluem a modelização do serviço e proposta de modelo avaliativo. Utilizou-se análise documental, observação direta, oficinas de consenso e entrevistas com informantes-chave.

Resultados:

foi realizada a descrição e a construção do modelo teórico-lógico do serviço de atenção às pessoas com ostomia, a elaboração das perguntas avaliativas, a construção da matriz de análise e julgamento, e apresentada proposta de avaliação.

Consideraç

ões Finais: verificou-se que é viável a avaliação sistemática do serviço a partir de elementos estruturantes, tais como sua teoria, recursos materiais e humanos. A avaliação proposta almeja fornecer subsídios para melhoria da qualidade da atenção.

Descritores:
Avaliação em Saúde; Ostomia; Política Pública; Práticas Interdisciplinares; Serviços de Saúde

INTRODUCTION

People with ostomies undergo a surgical intervention with externalization of the digestive, respiratory, or urinary system, creating an external orifice called "ostoma"⁽¹⁾. Although there are no definitive data on the number of people with ostomies in the country, some estimates can be made from information published by the Ministry of Health, and in newsletters or Brazilian and international journals of the associations of people with ostomies. According to Santos⁽²⁾, the International Ostomy Association estimates the prevalence of one person with an ostomy per 1,000 people in countries with a good level of health care. The author projected an estimate of 170,000 patients with ostomies in Brazil, based on the 2000 population census.

To deal with this situation, the Ministry of Health published Ordinance GM/SAS/MS No. 400/2009, establishing National Guidelines for the Health Care of People with Ostomy within the national health service (SUS)⁽¹⁾. These guidelines aim to provide actions oriented at self-care, prevention of ostomy complications, providing safe collection supplies, and adjuvant equipment in care services classified as Type I and Type II services⁽¹⁾. In addition to the actions common to both types, Type II care services include surgical treatments with a broader health team.

The person with an ostomy is considered a disabled person⁽³⁾, so ostomy care services are part of the Disabled Persons' Care Network⁽⁴⁾. This network considered the need to expand and diversify SUS services for people with physical, hearing, intellectual, visual, ostomies and multiple disabilities, organized in specialized care⁽⁴⁾.

In the state of Rio de Janeiro, the services that make up the Disabled Persons' Care Network were defined⁽⁵⁾. Specialized Rehabilitation Centers (CER) are organized as CER II, III or IV, according to the composition of two, three, or four types of disability, respectively⁽⁴⁾. The municipality of Niterói has type II ostomy care service (SAPE), qualified in only one rehabilitation service, which is not accredited as a CER^(4-5). The assessment of ostomy care is part of the national guidelines, and its premise is to reflect reality, seeking to improve health care actions⁽¹⁾.

Evaluability study

Evaluability, translated as "the extent to which the program can be evaluated," is the systematic and preliminary examination of a program, in theory and practice, to determine whether there is justification for program evaluation, as well as to identify critical areas to be prioritized in the evaluation⁽⁶⁾. Evaluability studies (ES) are a set of procedures that precede the evaluation stage, so that it is developed more easily, maximizing its potentials as well as favoring the rationalization of resources, which are often scarce for the evaluation process. It seeks a coherent description of the intervention, followed by a plan for evaluation, making it more consistent and more credible. The ES provides a complete description of the program, elaboration of the logical and theoretical models, the key issues to be addressed by the evaluation, an evaluation plan, the agreement between participants, and it can serve as a formal program evaluation proposal^(7-8).

Thurston and Ramaliu⁽⁷⁾ propose evaluability as a participatory tool for planning evaluations, including identifying existing program documentation regarding program objectives and activities as a valuable element. An evaluability study can be seen as a formative assessment that uses systematic research methods. In this sense, an ES is particularly valuable for managers who wish to promote organizational development, as it generates learning and improvement in the institutional life⁽⁷⁾.

In order to support managers in the process of evaluating actions developed for the person with an ostomy, and to gather information to make decisions about improvements of these interventions, the ES was used, identifying the theory behind this intervention, and the feasibility of their evaluation and planning.

OBJECTIVES

To perform the evaluability of the Ostomy Care Service (SAPE). And, the specific objectives were: to develop an evaluative model of the program, and to propose recommendations for planning future evaluative research.

METHODS

Ethical aspects

This study was approved by the Research Ethics Committee of the Antônio Pedro University Hospital, with Opinion No. 1,795,572, of October 27, 2016 (CAAE: 59600.516.8.0000.5243). After the presentation of the research objectives, and with the appropriate clarifications, the consent of the participants was obtained by their signing of the Terms of Free and Informed Consent Form.

Type of study

This was a study with a collaborative approach⁽⁹⁾ based on the seven elements proposed by Thurston and Ramaliu⁽⁷⁾: (E1) program development and identification of its goals, objectives, and activities; (E2) identification and analysis of intervention documents to be evaluated; (E3) development of the logical model of the program; (E4) program oversight or preliminary understanding of how the program operates; (E5) development of a theoretical evaluation model; (E6) identification of participants and those involved in the assessment; and (E7) definition of the evaluation procedure.

Methodological procedures

The research was conducted between July of 2017 and January of 2018, in the SAPE of Niterói. The various participants involved in the intervention and interested in the assessment (stakeholders) participated in the seven steps mentioned. The evaluators intentionally formed a partnership environment between evaluators and stakeholders, in a collaborative approach to strengthen the evaluative design, broaden the information gathering and the understanding of the results by all those involved, in order to increase use⁽⁹⁾. The operationalization of the study took place using the following steps: document analysis (E1-E2), direct observation (E4), and interviews with key informants (E4), development of the theoretical-logical model (TLM) (E3), and the matrix of analysis and judgment (MAJ) (E5). The theoretical-logical model also enables documenting the meaning of a program, and conceptualizing the links between structures, processes and outcomes^(10-11). The MAJ is a fundamental tool for evaluation, as it allows judgment of the program based on pre-established criteria or indicators and scores, as well as delimiting the sources of information verification, based on primary and secondary data.

Data Sources

The description of the program, understood in this study as the SAPE, and the development of the logical model, relied on document analysis, direct observation of the actions developed at the Niterói Care Center, and interviews with key informants. For the document analysis, we used the institutional technical documents (specific national norms and ordinances for ostomy care) and the SAPE documents (computerized records; individual user files; available protocols; professional qualification record book; management reports; activity reports; spreadsheets, technical advice). The choice of documents followed Richardson's basic principles: completeness, representativeness, homogeneity, and adequacy⁽¹²⁾. The data collection instrument for the documentary analysis had as its criteria: existence of program objectives, target audience, goals to be achieved, actions/activities that need to be performed to operate the program, resources needed to perform the actions, and results to be achieved⁽¹³⁾. Key informants included a total of eight respondents: a) municipal manager; b) higher level technicians (service coordinator, nurse, and social worker); c) stoma therapist; and, d) health assessment specialists (nurses). A semi-structured guide was used for the interviews, which were recorded and transcribed in their entirety; the information collected was organized into categories and analyzed using the content analysis technique⁽¹⁴⁾.

To validate the TLM and develop the MAJ, two workshops were conducted with experts (E6-E7) and professionals interested in the assessment. At the first workshop, the experts clarified the expected results, and proposed the matrix indicators in order to meet the objective of evaluating the results achieved by SAPE. The TLM was structured based on the technical and scientific literature^(8,13) regarding subject and document analysis^(1,4). The social context into which the intervention was integrated was taken into consideration, and it was extensively discussed with the co-authors of this study and health professionals with experience in the ostomy topic, and in the area of health evaluation. The identification of each of the TLM components took into account the following questions, proposed by Hartz⁽⁸⁾: Is the accomplishment of each activity possible from the listed resources? Do the proposed activities address the program components? Do activities make it possible to achieve results? With the results achieved, is it possible to measure the degree of impact of the program? In the second workshop, TLM and MAJ were validated, with the selection of dimensions, criteria, and indicators. The consensus decision process was obtained through agreement of all participants. Then, the MAJ was developed, consisting of components, pattern, results, and verification source. The MAJ (Chart 2) considered the maximum scores for the "implementation" dimension, resulting from the sum of the indicator group of each component (Σ of the indicator points). For the maximum score (MS), the goals recommended by the national guidelines were considered as standard, in addition to the considerations of those involved in the project implementation. The score verified in the documents was used to define the observed score, when for the same criterion or indicator a score is present for more than one evidence base. In the absence of this, the predominant score was considered. The components were chosen considering the program activities, structure, and expected results, taking as standard what is standardized in the official documents^(1,4). The source of verification was the place and/or individuals from which the data was collected. Triangulation of data obtained through primary sources (interviews and direct observation), and document analysis, was also performed. Triangulation enables verification that different data collection methods obtained similar findings, reinforcing their legitimacy⁽¹⁵⁾.

Chart 2
Analysis and judgment matrix of the Ostomy Care Service implementation context, Niterói, Rio de Janeiro, Brazil, 2018

Component - Organization and planning
Category	Criterion/Indicator	P	MS	VS
Human resources	Coordinator and / or other professionals with specialization in public health, or >2 years of experience in the service	5	15	DA; I.
	Minimum team for care actions (1 nurse, 1 physician, and 1 social worker)	5
	At least one technician for administrative support	5
Reception and register	Reception and registration room, side table, and chair	1	3	DA; OD.
	Reception and registration room, coffee table, chair, and computer	2
	Reception and registration room, coffee table, chair, computer and telephone	3
Clinic	Waterproof gurney	1	10	DA; OD.
	Step ladder	1
	Hand washing sink, liquid soap dispenser, and paper towel	1
	Exclusive bathroom with hygienic shower	1
	Mirror with minimum dimensions of 120 × 50 cm	1
	Sphygmomanometer and stethoscope	1
	Thermometer	1
	Dressing material	1
	Ostomy cleaning and hygiene material	1
	Irrigation material	1
Storage area for collecting supplies	Cabinet with closed doors	3	3	DA; OD.
	Open shelves	2
Technical quality of equipment	Unexpired material	2.5	5	DA; OD.
	Entry and Exit Control material record	2.5
Collecting and adjuvant material	Monthly distribution of sufficient collecting and adjuvant material over the past year	10	10	DA; OD, I.
	Lack of sufficient collection and adjuvant material over the past year, hampering delivery to patients for up to two months	3
	Lack of sufficient collection and adjuvant material over the past year, for more than two months	2
Group meeting room	Meeting room for group service	2	2	DA; OD, I.
Available standards, charts and protocols	Presence of norms, medical records and protocols	1	1
Quantitative and qualitative planning of collection and adjuvant equipment	No annual planning meeting	0	2	DA; I.
	One or more annual planning meetings	2
SER user registration	No user registration in SER	0
	Proof of registration of users in SER	2
Quality of Records	No record of control or patient history in digital media	0	3	DA; OD.
	Presence of control record and patient history in digital media, with at least 75% of fields completed	3
	Presence of control record and patient history in full digital media, below 75% of fields completed	2
Component - Health Care
Category	Criteria or Indicators	P	MS	VS
Dispensation (scheduling and / or spontaneous demand)	Dispensing of bags and adjuvants sufficient for a period of 30 days or more	10	10	DA; OD; I.
	Bags or adjuvants for dispensing not sufficient for 30 days	0
Consultations with social worker	100% of appointments scheduled for the same month	2.5	5	DA; OD; I
	100% of consultations performed by spontaneous demand	2.5
Stomatherapy care	100% of appointments scheduled for the same month	2.5	5
	100% of consultations performed by spontaneous demand	2.5
Medical care	100% of appointments scheduled for the same month	2.5	5
	100% of consultations performed by spontaneous demand	2.5
Nutritional support	100% of appointments scheduled for the same month	2.5	5
	At least 50% of consultations performed by spontaneous demand	2.5
Component - Health Care
Category	Criteria or Indicators	P	MS	VS
Treatment and / or referral of ostomy complications	Presence of physician to treat complications	5	10	DA.
	Presence of nurse to treat complications	5
	Lack of specialized references for ostomy complications	0
Component - Permanent education and formation
Category	Criteria or Indicators	P	MS	VS
Self-care Groups	No self-care group meetings	0	3	DA; I.
	At least one self-care group meeting every two months	2
	A monthly self-care group meeting	3
Knowledge of legal regulations	Lack of knowledge about legal instruments governing care services	0	3	DA; I
	At least the coordinator with knowledge of laws governing care services	2
	All professionals with knowledge of laws governing care services	3
Participation in training courses	Lack of training of technicians, promoted by any management level, regarding the service	0	3	DA; I.
	Training of 10% of the technicians, promoted by any management level, regarding the service, in the last 2 years.	2
	Training of more than 10% of technicians, promoted by any level of management, related to the service, in the last 2 years.	3

Source: Official and technical documents.
• Note: P - Parameter; MS - Maximum Score; DA - Document Analysis; I - Interview; VS - Verification Source.

RESULTS

The results were described based on the seven elements (E1, E2, E3, E4, E5, E6 and E7).

E1 - Program development and identification of goals, objectives and activities

Although not intended to be a program, the SAPE is standardized, with the purpose of conducting orientation actions for self-care, prevention of ostomy complications, and provision of safe collection supplies, and adjuvant equipment.

Although the goals are not clearly defined in the national guidelines, comprehensive and specialized interdisciplinary care for people with ostomy was understood by the diverse participants of this research as the goal of the intervention. The objectives identified were: to orientate patients for self-care; to implement actions to prevent complications in ostomies; to provide safe collection supplies and adjuvant equipment; to have a multidisciplinary team, equipment, and adequate physical facilities.

E2 - Identification and analysis of the SAPE documents

The institution's technical documents, and their main objectives, are presented in the Chart 1. These documents are coherent and they support and articulate the guidelines and concepts of care for people with ostomy.

Chart 1
Ostomy Care Service documents and their main objectives

Document	Year	Main deliberations / objectives
Ordinance GM/MS nº 400	2009	Establishes orientation actions for self-care, prevention of ostomy complications, providing safe collection supplies and adjuvant equipment.
Ordinance GM/MS nº 793	2012	Deliberates the expansion of access and qualification of care for people with disabilities in the SUS; limits the linking of people with disabilities and their families to the points of care.
Ordinance GM/MS nº 835	2012	Contemplates the financial incentive for the construction of CER, orthopedic workshops, renovation or extension for qualification of CER II, CER III and CER IV, acquisition of equipment and other permanent materials.
Ordinance GM/MS nº 2.602	2012	Establishes financial resources to support the activities of the various care centers, incorporating them into the Medium and High Complexity Financial Ceiling (MAC Ceiling) of the States, Federal District and Municipalities.
Deliberation CIB-RJ no 2.790	2014	Upgrades the Care Centers for People with Ostomy of the State of RJ
Book of activity records	2016 to Feb./2018	Records the daily activities of the Care Center, training, service meetings.
Individual sheets of supply dispensing	2016 to Feb./2018	Records the monthly quantity of safe collection supplies, distributed to the patient, and schedule for return.

Source: Official and institutional documents of the research.

E3 - Composition and development of the Theoretical-Logical Model

From the information obtained by means of documentary analysis, direct observation of the actions developed, consensus workshops, and interviews with key informants, the TLM for care for people with ostomy was developed, based on the program theory. The external context into which SAPE is integrated refers to the municipality and its sociodemographic dynamics. The political-organizational context is constituted by the governance and management of care services. The implementation context is related to SAPE itself, where health actions are developed. The actions are represented by the components of TLM: Organization and Planning, Health Care, and Continuing Education and Training.

The organization and planning component demands actions in the health system, state and municipal government, and service management, and is influenced by elements of the external and political-institutional contexts. The components, Health Care and Continuing Education and Training, are also influenced by the political and organizational context, regarding the availability of state and municipal resources. The actions developed in SAPEs are interrelated, and are fundamental to ensure comprehensive care with repercussions on the patients' quality of life.

In the continuing education and training component, the orientation and education activities of the primary care professionals were highlighted, from the SAPEs and the health teams of the hospital units. These actions include education activities regarding preoperative and postoperative care for the patient during the stoma surgeries, including reconstruction of intestinal and urinary transit, as well as the treatment of postoperative complications and specialized techniques. The TLM developed and agreed upon with the stakeholders, included objectives, goals, inputs, activities, target audience, results, as well as the logical relationship between the components.

E4 - Description of the organization and operation of the program

The SAPE-Niterói has existed since 2005, serving individuals of the municipality and metropolitan region II, consisting of the municipalities of São Gonçalo, Maricá, Itaborai, Rio Bonito, Silva Jardim, and Tanguá. It is located in the Orêncio de Freitas Municipal Hospital site, and is linked to the Vice Presidency of Hospital and Emergency Care of the Niterói Municipal Health Foundation. The activities are conducted on weekdays from 8am to 5 pm. The main motivation for the patients coming to the service is to obtain collection supplies. However, this activity is not always performed due to the continuous shortage of these objects.

Figure 1
Theoretical-Logical Model of the Ostomy Care Service, Niterói, Rio de Janeiro, Brazil, 2018

E5 - Composition and development of the analysis and judgment matrix

Indicators related to supplies were applied to human, physical, and material resources. The minimum staff defined by the guidelines is: one physician; one nurse (trained in assisting people with ostomy); one psychologist; one nutritionist; and one social worker. The indicators related to physical and material resources were: clinical office; storage area with adequate furniture; sufficient collection supplies and adjuvants; meeting room; available standards, charts and protocols.

E6 - Identification of patients and those involved in the evaluation.

During the workshops, the co-authors of this study, the coordinator, and other service professionals indicated the municipal manager and the service professionals as the main clients involved in the evaluation of SAPE- Niterói. The participation of the client was verified as a favorable element of the external context, with participation of members of the service management, indicating the priorities for the effective implementation of the actions.

E7 - Definition of evaluation procedure

The study participants found the assessment was necessary, to verify compliance with the guidelines and the achievement of program objectives, as well as to support decision-making based on the follow-up and appropriateness of the actions performed. The evaluative questions were formulated by means of an analysis of questions about program evaluation contained in the interviews, and agreement of the theoretical and logical models. In the end, the following questions were defined: (a)What is the degree of implementation in SAPE-Niterói? (b) Did the political-organizational and external contexts influence the implementation of the program? In what way? (c) Did the degree of implementation of SAPE-Niterói influence the results achieved?

DISCUSSION

The study showed that the SAPE documents were coherent, and that they supported and articulated guidelines and concepts of care.

The financial incentive for acquisition of equipment and other permanent materials was conceded by the National Health Fund^(16-17). Such resources supported the activities of several care centers⁽¹⁸⁾.

Listed in the TLM were the relationships between objectives proposed by the intervention, accredited and qualified as Center I and II in 2012, with their intended activities and results, and the contextual analysis^(19-20). Use of program evaluation results was shared by the main participants interested in the evaluation⁽²¹⁾. In this study, consensus development of the TLM was essential for the knowledge of the intervention, and the future use of the implementation and its results in the evaluation^(22-24). The intervention theory evidenced its basis in the political decision that preceded the operationalization, as demonstrated by Silva et al.⁽²⁵⁾ and SOBEST⁽²⁶⁾, indicating that interventions are planned from evidenced problems, according to the context in which they are developed, and by the individuals involved in its planning and execution.

A review study on logic models from evaluability studies identified three studies in which the program theory was used as a methodological strategy for modeling, becoming a necessary part for future evaluation⁽²⁷⁾. Other ES that used the TLM and MAJ demonstrated the importance of this preliminary step for the correct development of further evaluations^(28-29).

With regards to the organization and operation of the program, orientation activities and stoma therapy consultations were the most present. The most frequent guidelines involved the rights and duties of the person with an ostomy, operation of the service, referral of patients to outpatient consultations with a nutritionist, surgeon and care by the stoma therapist, which includes guidance for self-care. One of the critical aspects of Ordinance 400 is the non-inclusion of the stoma therapist as a minimum staff component. As the stoma therapist has been in the Brazilian Occupation Code since 2013, the ordinance should have been updated. The SAPE activities converge with reports of some authors, such as Moraes⁽¹⁶⁾, for SAPE diagnostic analysis in Minas Gerais and Santos⁽³⁰⁾. These include patient registration, dispensing of collection supplies, and individual appointments.

The discussion considering only dispensing of collection supplies is corroborated by Barros et al.⁽³¹⁾, and assumes that providing materials and equipment by the health units guarantees the patient's autonomy and understands the self-care as a manner of translating the ability to experience new challenges⁽³²⁾.

The public health policies designed for people with ostomy should include access to health needs, by ensuring many levels of care, so as to enable comprehensive care and not just limit care to supplying collection supplies⁽³³⁾. Despite delimiting goals, objectives, and activities, Ordinance 400 establishes care guidelines for people with ostomy, but is not a public health program.

The study by Santos and Sawaia⁽³⁴⁾⁾ analyzed the process of construction and reconstruction of the meanings of the concepts, "ostomy" and the "nurse's role in the management of an ostomy", after the experience of these professionals in the use of the bag. From experience, changes in practice were proposed in order to incorporate the affective, symbolic, and relational dimensions, critical to the management of ostomy patient care. The teaching of self-care assures the person with an ostomy the prevention of stoma complications, enabling independence in their care in relation to their family and health professionals. The help of a professional is relevant in the adaptive process, to clarify questions and reduce concerns regarding the acceptance of a new condition^(32,35-36).

Regarding the identification of users and those involved in the evaluation, several authors^(21,37) highlight the broad involvement of participants, giving priority to obtaining group thoughts and perceptions, and reconciling differences when necessary. Trevisan⁽²¹⁾ proposes working groups in the ES, including the evaluator, program team, and other representatives, to increase the likelihood of using the findings.

Study limitations

The research was restricted to a municipality in southeastern Brazil. Thus, it is suggested that other studies be performed involving other services. In addition, the interviews with few participants, due to the infrequent data collection period and the few professionals from the single municipal center, were also limitations of this study.

Contributions for the public health

Evaluability studies are intended to prepare evaluations of health care services, as they provide support for the planning and development of evaluative studies. Such studies can become the basis for decision making to improve or change the services and programs. The development of an evaluability study in SAPE, where the nurse has an essential role, can contribute as a model for assessment in the services already implemented in the several municipalities of the country.

FINAL CONSIDERATIONS

This ES contributed to the understanding of the intervention, and a better understanding of the conception and logic of its operation. The collaborative approach and methodology used met the objectives of the study and enabled the construction of an intervention (TLM and MAJ), with broad participation, which can increase the knowledge and commitment of those involved in the future use of evaluation results. The constituent elements of TLM show, among themselves, an interdependent relationship that enables the achievement of the expected outcomes.

A formative assessment based on the ES is recommended, including the identification of the aspects to be prioritized in order to achieve the expected outcomes. It is noteworthy that the proposed model could be revised. In addition, it is necessary to update the current ordinance that deals with the care of the person with ostomy, so that a requirement will be included in the legal text for a stoma therapist to be a member of the minimum team.

REFERENCES

• ¹
  Ministério da Saúde (BR). Portaria nº 400, de 16/11/2009. Diretrizes Nacionais para a Atenção à Saúde das Pessoas Estilizadas no âmbito do Sistema Único de Saúde - SUS [Internet]. 2009 [cited 2019 Jun 10]. Available from: http://bvsms.saude.gov.br/bvs/saudelegis/sas/2009/prt0400_16_11_2009.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/sas/2009/prt0400_16_11_2009.html
• ²
  Santos VLCG. [Epidemiology of stomas]. Rev Estima [Internet]. 2007 [cited 2019 Jun 10];5(1):31-8. Available from: https://www.revistaestima.com.br/index.php/estima/article/view/207 Portuguese
  » https://www.revistaestima.com.br/index.php/estima/article/view/207
• ³
  Presidência da República (BR). Lei no 10.690, de 16 de junho de 2003 [Internet]. 2003 [cited 2019 Jun 10]. Available from: http://www.planalto.gov.br/ccivil_03/LEIS/2003/L10.690.htm
  » http://www.planalto.gov.br/ccivil_03/LEIS/2003/L10.690.htm
• ⁴
  Ministério da Saúde (BR). Instrutivo de Ostomia: Portaria GM 793 de 24/04/2012 [Internet]. 2012 [cited 2019 Jun 10]. Available from: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt0793_24_04_2012.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt0793_24_04_2012.html
• ⁵
  Comissão Intergestores Bipartite. Deliberação CIB-RJ, 2.790 de 14/03/2014 [Internet]. 2014 [cited 2019 Jun 10]. Available from: http://www.cib.rj.gov.br/deliberacoes-cib/403-2014/fevereiro/3191-deliberacao-cib-n-2-790-de-14-de-marco-de-2014.html
  » http://www.cib.rj.gov.br/deliberacoes-cib/403-2014/fevereiro/3191-deliberacao-cib-n-2-790-de-14-de-marco-de-2014.html
• ⁶
  Rossi PH, Lipsey MW, Freeman HE. Evaluation: a systematic approach. Thousand Oaks: Sage Publications; 2003.
• ⁷
  Thurston WE, Ramaliu A. Evaluability assessment of a survivors of torture program: lessons learned. Can J Program Eval [Internet]. 2005 [cited 2019 Jun 10];20(2):1-25. Available from: https://evaluationcanada.ca/secure/20-2-001.pdf
  » https://evaluationcanada.ca/secure/20-2-001.pdf
• ⁸
  Hartz ZMA, Vieira-da-Silva LM. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz; 2015.
• ⁹
  Rodríguez-Campos L, Rincones-Gómez R. Collaborative evaluations: step-by-step. Stanford, California: Stanford University Press; 2013.
• ¹⁰
  Carvalhosa SF, Domingos A, Cequeira C. Modelo lógico de um programa de intervenção comunitária - GerAcções. Anal Psicol [Internet]. 2010 [cited 2019 Jun 10];28(3):479-90. Available from: http://www.scielo.mec.pt/pdf/aps/v28n3/v28n3a08.pdf
  » http://www.scielo.mec.pt/pdf/aps/v28n3/v28n3a08.pdf
• ¹¹
  Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Fiocruz; 2011.
• ¹²
  Richardson RJ. Pesquisa social: métodos e técnicas. São Paulo: Editora Atlas S.A; 2015.
• ¹³
  Alves CKA, Natal S, Felisberto E, Samico I. Interpretação e Análise das informações: o uso de matrizes, critérios, indicadores e padrões. In: Samico I, Felisberto E, Figueiró AC, Frias PG, organiz. Avaliação em saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010. p. 89-107.
• ¹⁴
  Bardin L. Análise de conteúdo. Portugal: Edições 70; 2014.
• ¹⁵
  Steckler A, McLeroy KR, Goodman RM, Bird ST, McCormick L. Toward integrating Qualitative and Quantitative Methods: an introduction. Health Educ Q.1992;19(1):1-8. doi: 10.1177/109019819201900101
  » https://doi.org/10.1177/109019819201900101
• ¹⁶
  Moraes JT, Amaral CFS, Borges EL, Ribeiro MS, Guimarães EAA. [Ostomy patients health care services: diagnostic analysis in the State of Minas Gerais, Brazil]. Cad Saúde Coletiva. 2014; 22(1):101-8. doi: 10.1590/1414-462X201400010015 Portuguese.
  » https://doi.org/10.1590/1414-462X201400010015
• ¹⁷
  Ministério da Saúde (BR). Portaria nº 835, de 25 de abril de 2012 [Internet]. 2012 [cited 2019 Jun 10]. Available from: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt0835_25_04_2012
  » http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt0835_25_04_2012
• ¹⁸
  Ministério da Saúde (BR). Portaria GM/MS n. 2.602, de 16/11/2012[Internet]. 2012 [cited 2019 Jun 10]. Available from: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt2602_16_11_2012.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt2602_16_11_2012.html
• ¹⁹
  Medina MG, Silva GAP, Aquino R, Hartz ZMA. Uso de modelos teóricos na avaliação em saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Vieira-da-Silva LM. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz; 2014. p.41-64.
• ²⁰
  Conferências Intergestoras Bipartites. Deliberação CIB-RJ, n. 1649 de 08/03/2012 [Internet]. 2012 [cited 2019 Jun 10]. Available from: http://www.cib.rj.gov.br/deliberacoes-cib/60-2012/marco/1840-deliberacao-cib-n-1649-de-08-de-marco-de-2012.html
  » http://www.cib.rj.gov.br/deliberacoes-cib/60-2012/marco/1840-deliberacao-cib-n-1649-de-08-de-marco-de-2012.html
• ²¹
  Trevisan MS. Evaluability assessment from 1986 to 2006. Am J Eval. 2007;28(3):290-303. doi: https://doi.org/10.1177/1098214007304589
  » https://doi.org/https://doi.org/10.1177/1098214007304589
• ²²
  Thurston WE, Potvin L. Evaluability assessment: a tool for incorporating evaluation in social change programs. Evaluation, 2003;9(4):453-69. doi: dx.doi.org/10.1177/1356389003094006
  » https://doi.org/dx.doi.org/10.1177/1356389003094006
• ²³
  Oliveira LGD, Natal S, Camacho LAB. [Analysis of the implementation of the Tuberculosis Control Program in Brazilian prisons]. Cad Saúde Pública. 2015;31(3):543-54. doi: http://dx.doi.org/10.1590/0102-311x00042914 Portuguese.
  » https://doi.org/http://dx.doi.org/10.1590/0102-311x00042914
• ²⁴
  Oliveira LGD, Natal S. Avaliação da implantação do Programa de Controle da Tuberculose e sua integração com o Programa Médico de Família: estudo de caso em Niterói/RJ. In: Santos EM, Cruz MM (org). Avaliação em saúde: dos modelos teóricos à prática da avaliação de programas de controle de processos endêmicos. Rio de Janeiro: Fiocruz; 2014. p.61-179.
• ²⁵
  Silva RN, Guarda FRB, Hallal PC, Martelli PJL. [Evaluability of the Health Gym Program in Recife, Pernambuco State, Brazil]. Cad. Saúde Pública 2017; 33(4). doi: http://dx.doi.org/10.1590/0102-311x00159415 Portuguese.
  » http://dx.doi.org/10.1590/0102-311x00159415
• ²⁶
  Associação Brasileira de Ostomaterapia. Ostomaterapia-o que é? [Internet]. 2016 [cited 2019 Jun 10]. Available from: http://www.sobest.org.br/texto/5
  » http://www.sobest.org.br/texto/5
• ²⁷
  Esher A, Santos EM, Azeredo TB, Luiza VL, Osorio-de-Castro CGS, Oliveira MA. Logic models from an evaluability assessment of pharmaceutical services for people living with HIV/AIDS. Ciênc Saúde Coletiva. 2011;16(12):4833-44. doi: 10.1590/S1413-81232011001300032
  » https://doi.org/10.1590/S1413-81232011001300032
• ²⁸
  Natal S, Samico I, Oliveira LGD, Assis AMJ. Estudo de avaliabilidade da rede de formação de Recursos Humanos da Secretaria de Vigilância em Saúde do Ministério da Saúde. Cad. Saúde Colet. [Internet]. 2010 [cited 2019 Jun 10];18(4):560-71. Available from: http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2010_4/artigos/CSC_v18n4_560-571.pdf
  » http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2010_4/artigos/CSC_v18n4_560-571.pdf
• ²⁹
  Oliveira LGD, Natal S, Camacho LAB. O programa de controle da tuberculose em unidades prisionais de dois estados brasileiros. Cad Saúde Coletiva [Internet]. 2012 [cited 2019 Jun 10];20(2):250-7. Available from: http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2012_2/artigos/csc_v20n2_250-257.pdf
  » http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2012_2/artigos/csc_v20n2_250-257.pdf
• ³⁰
  Santos VLCG. [Conceptual framework for ostomy care in the institucionalized adult health care área]. Rev Esc Enferm USP. 2000;34(1):59-63. doi: 10.1590/S0080-62342000000100008 Portuguese.
  » https://doi.org/10.1590/S0080-62342000000100008
• ³¹
  Barros EJL, Santos SSC, Gomes GC, Erdmann AL, Pelzer MT, Gautério DP. [Ecosystemic and gerontotechnological actions in complex nursing care to the elderly with ostomy]. Rev. Bras Enferm. 2014;67(1):91-6. doi: 10.5935/0034-7167.20140012 Portuguese.
  » https://doi.org/10.5935/0034-7167.20140012
• ³²
  Silva J, Sonobe HM, Buetto LS, Santos MG, Lima MS, Sasaki VDM. Teaching strategies for self-care of the intestinal stoma patients. Rev Rene. 2014;15(1):166-73. doi: 10.15253/2175-6783.2014000100021
  » https://doi.org/10.15253/2175-6783.2014000100021
• ³³
  33. Almeida EJ, Silva AL. Characterization of the Epidemiological Profile of the Ostomy in Hospitals of the Secretary of State of Health of the Federal District. Rev Estima [Internet]. 2015 [cited 2019 Jun 10];13(1):11-6. Available from: https://www.revistaestima.com.br/index.php/estima/article/view/101/pdf
  » https://www.revistaestima.com.br/index.php/estima/article/view/101/pdf
• ³⁴
  Santos VLCG, Sawaia BB. The pouch acting as a mediator between "being a person with an ostomy" and "being a professional": analysis of a pedagogical strategy. J Wound Ostomy Continence Nurs. 2001;28(4):206-14. doi: https://doi.org/10.1067/mjw.2001.116254
  » https://doi.org/https://doi.org/10.1067/mjw.2001.116254
• ³⁵
  Nascimento CMS, Trindade GLB, Luz MHBA, Santiago RF. [The ostomy patient's living experience: a contribution to nursing care]. Texto Contexto Enferm. 2011;20(3):557-64. doi: 10.1590/S0104-07072011000300018 Portuguese.
  » https://doi.org/10.1590/S0104-07072011000300018
• ³⁶
  Freitas LS, Queiroz CG, Medeiros LP, Melo MDM, Andrade RS, Costa IKFC. Indicators of the nursing outcome ostomy self-care: integrative review. Cogitare Enferm. 2015;20(3):618-25. doi: 10.5380/ce.v20i3.40045
  » https://doi.org/10.5380/ce.v20i3.40045
• ³⁷
  Leviton LC, Khan LK, Rog D, Dawkins N, Cotton D. Evaluability assessment to improve public health policies, programs, and practices. Ann Rev Public Health. 2010;31:213-33. doi: 10.1146/annurev.publhealth.012809.103625
  » https://doi.org/10.1146/annurev.publhealth.012809.103625

Edited by

EDITOR IN CHIEF: Dulce Aparecida Barbosa

ASSOCIATE EDITOR: Alexandre Balsanelli

Publication Dates

• Publication in this collection
  01 June 2020
• Date of issue
  2020

History

• Received
  18 Oct 2018
• Accepted
  04 Sept 2019