Health evaluation: health assessment for people with ostomies

Rev Bras Enferm. 2020;73(4): e20180789 http://dx.doi.org/10.1590/0034-7167-2018-0789 8 of ABSTRACT Objectives: to perform an evaluation study of the Ostomy Care Service in Brazil. Methods: a qualitative methodology was used, with a collaborative approach of the participants using the service, based on the seven elements proposed by Thurston and Ramaliu, which include service modeling and evaluation model proposal. Document analysis, direct observation, consensus workshops, and interviews with key informants were used. Results: the description and construction of the theoretical-logical model of the ostomy care service, the development of evaluative questions, the construction of the analysis and judgment matrix, and the evaluation proposal were presented. Final Considerations: it was found that it is feasible to systematically evaluate the service based on its structural elements, such as its theory, material and human resources. The proposed evaluation aims to provide support for improving the quality of care. Descriptors: Health Evaluation; Ostomy; Public Policy; Interdisciplinary Placement; Health Services.


INTRODUCTION
People with ostomies undergo a surgical intervention with externalization of the digestive, respiratory, or urinary system, creating an external orifice called "ostoma" (1) . Although there are no definitive data on the number of people with ostomies in the country, some estimates can be made from information published by the Ministry of Health, and in newsletters or Brazilian and international journals of the associations of people with ostomies. According to Santos (2) , the International Ostomy Association estimates the prevalence of one person with an ostomy per 1,000 people in countries with a good level of health care. The author projected an estimate of 170,000 patients with ostomies in Brazil, based on the 2000 population census.
To deal with this situation, the Ministry of Health published Ordinance GM/SAS/MS No. 400/2009, establishing National Guidelines for the Health Care of People with Ostomy within the national health service (SUS) (1) . These guidelines aim to provide actions oriented at self-care, prevention of ostomy complications, providing safe collection supplies, and adjuvant equipment in care services classified as Type I and Type II services (1) . In addition to the actions common to both types, Type II care services include surgical treatments with a broader health team.
The person with an ostomy is considered a disabled person (3) , so ostomy care services are part of the Disabled Persons' Care Network (4) . This network considered the need to expand and diversify SUS services for people with physical, hearing, intellectual, visual, ostomies and multiple disabilities, organized in specialized care (4) .
In the state of Rio de Janeiro, the services that make up the Disabled Persons' Care Network were defined (5) . Specialized Rehabilitation Centers (CER) are organized as CER II, III or IV, according to the composition of two, three, or four types of disability, respectively (4) . The municipality of Niterói has type II ostomy care service (SAPE), qualified in only one rehabilitation service, which is not accredited as a CER (4)(5) . The assessment of ostomy care is part of the national guidelines, and its premise is to reflect reality, seeking to improve health care actions (1) .

Evaluability study
Evaluability, translated as "the extent to which the program can be evaluated, " is the systematic and preliminary examination of a program, in theory and practice, to determine whether there is justification for program evaluation, as well as to identify critical areas to be prioritized in the evaluation (6) . Evaluability studies (ES) are a set of procedures that precede the evaluation stage, so that it is developed more easily, maximizing its potentials as well as favoring the rationalization of resources, which are often scarce for the evaluation process. It seeks a coherent description of the intervention, followed by a plan for evaluation, making it more consistent and more credible. The ES provides a complete description of the program, elaboration of the logical and theoretical models, the key issues to be addressed by the evaluation, an evaluation plan, the agreement between participants, and it can serve as a formal program evaluation proposal (7)(8) .
Thurston and Ramaliu (7) propose evaluability as a participatory tool for planning evaluations, including identifying existing program documentation regarding program objectives and activities as a valuable element. An evaluability study can be seen as a formative assessment that uses systematic research methods. In this sense, an ES is particularly valuable for managers who wish to promote organizational development, as it generates learning and improvement in the institutional life (7) .
In order to support managers in the process of evaluating actions developed for the person with an ostomy, and to gather information to make decisions about improvements of these interventions, the ES was used, identifying the theory behind this intervention, and the feasibility of their evaluation and planning.

OBJECTIVES
To perform the evaluability of the Ostomy Care Service (SAPE). And, the specific objectives were: to develop an evaluative model of the program, and to propose recommendations for planning future evaluative research.

Type of study
This was a study with a collaborative approach (9) based on the seven elements proposed by Thurston and Ramaliu (7) : (E1) program development and identification of its goals, objectives, and activities; (E2) identification and analysis of intervention documents to be evaluated; (E3) development of the logical model of the program; (E4) program oversight or preliminary understanding of how the program operates; (E5) development of a theoretical evaluation model; (E6) identification of participants and those involved in the assessment; and (E7) definition of the evaluation procedure.

Methodological procedures
The research was conducted between July of 2017 and January of 2018, in the SAPE of Niterói. The various participants involved in the intervention and interested in the assessment (stakeholders) participated in the seven steps mentioned. The evaluators intentionally formed a partnership environment between evaluators and stakeholders, in a collaborative approach to strengthen the evaluative design, broaden the information gathering and the understanding of the results by all those involved, in order to increase use (9) . The operationalization of the study took place using the following steps: document analysis (E1-E2), direct observation (E4), and interviews with key informants (E4), development of the theoretical-logical model (TLM) (E3), and the matrix of analysis and judgment (MAJ) (E5). The theoretical-logical model also enables documenting the meaning of a program, and conceptualizing the links between structures, processes and outcomes (10)(11) . The MAJ is a fundamental tool for evaluation, as it allows judgment of the program based on pre-established criteria or indicators and scores, as well as delimiting the sources of information verification, based on primary and secondary data.

Data Sources
The description of the program, understood in this study as the SAPE, and the development of the logical model, relied on document analysis, direct observation of the actions developed at the Niterói Care Center, and interviews with key informants. For the document analysis, we used the institutional technical documents (specific national norms and ordinances for ostomy care) and the SAPE documents (computerized records; individual user files; available protocols; professional qualification record book; management reports; activity reports; spreadsheets, technical advice). The choice of documents followed Richardson's basic principles: completeness, representativeness, homogeneity, and adequacy (12) . The data collection instrument for the documentary analysis had as its criteria: existence of program objectives, target audience, goals to be achieved, actions/activities that need to be performed to operate the program, resources needed to perform the actions, and results to be achieved (13) . Key informants included a total of eight respondents: a) municipal manager; b) higher level technicians (service coordinator, nurse, and social worker); c) stoma therapist; and, d) health assessment specialists (nurses). A semi-structured guide was used for the interviews, which were recorded and transcribed in their entirety; the information collected was organized into categories and analyzed using the content analysis technique (14) .
To validate the TLM and develop the MAJ, two workshops were conducted with experts (E6-E7) and professionals interested in the assessment. At the first workshop, the experts clarified the expected results, and proposed the matrix indicators in order to meet the objective of evaluating the results achieved by SAPE. The TLM was structured based on the technical and scientific literature (8,13) regarding subject and document analysis (1,4) . The social context into which the intervention was integrated was taken into consideration, and it was extensively discussed with the co-authors of this study and health professionals with experience in the ostomy topic, and in the area of health evaluation. The identification of each of the TLM components took into account the following questions, proposed by Hartz (8) : Is the accomplishment of each activity possible from the listed resources? Do the proposed activities address the program components? Do activities make it possible to achieve results? With the results achieved, is it possible to measure the degree of impact of the program? In the second workshop, TLM and MAJ were validated, with the selection of dimensions, criteria, and indicators. The consensus decision process was obtained through agreement of all participants. Then, the MAJ was developed, consisting of components, pattern, results, and verification source. The MAJ (Chart 2) considered the maximum scores for the "implementation" dimension, resulting from the sum of the indicator group of each component (Σ of the indicator points). For the maximum score (MS), the goals recommended by the national guidelines were considered as standard, in addition to the considerations of those involved in the project implementation. The score verified in the documents was used to define the observed score, when for the same criterion or indicator a score is present for more than one evidence base. In the absence of this, the predominant score was considered. The components were chosen considering the program activities, structure, and expected results, taking as standard what is standardized in the official documents (1,4) . The source of verification was the place and/or individuals from which the data was collected. Triangulation of data obtained through primary sources (interviews and direct observation), and document analysis, was also performed. Triangulation enables verification that different data collection methods obtained similar findings, reinforcing their legitimacy (15) .

E1 -Program development and identification of goals, objectives and activities
Although not intended to be a program, the SAPE is standardized, with the purpose of conducting orientation actions for self-care, prevention of ostomy complications, and provision of safe collection supplies, and adjuvant equipment.
Although the goals are not clearly defined in the national guidelines, comprehensive and specialized interdisciplinary care for people with ostomy was understood by the diverse participants of this research as the goal of the intervention. The objectives identified were: to orientate patients for self-care; to implement actions to prevent complications in ostomies; to provide safe collection supplies and adjuvant equipment; to have a multidisciplinary team, equipment, and adequate physical facilities.

E2 -Identification and analysis of the SAPE documents
The institution's technical documents, and their main objectives, are presented in the Chart 1. These documents are coherent and they support and articulate the guidelines and concepts of care for people with ostomy.

E3 -Composition and development of the Theoretical-Logical Model
From the information obtained by means of documentary analysis, direct observation of the actions developed, consensus workshops, and interviews with key informants, the TLM for care for people with ostomy was developed, based on the program theory. The external context into which SAPE is integrated refers to the municipality and its sociodemographic dynamics. The political-organizational context is constituted by the governance and management of care services. The implementation context is related to SAPE itself, where health actions are developed. The actions are represented by the components of TLM: Organization and Planning, Health Care, and Continuing Education and Training.
The organization and planning component demands actions in the health system, state and municipal government, and service management, and is influenced by elements of the external and political-institutional contexts. The components, Health Care and Continuing Education and Training, are also influenced by the political and organizational context, regarding the availability of state and municipal resources. The actions developed in SAPEs are interrelated, and are fundamental to ensure comprehensive care with repercussions on the patients' quality of life.
In the continuing education and training component, the orientation and education activities of the primary care professionals were highlighted, from the SAPEs and the health teams of the hospital units. These actions include education activities regarding preoperative and postoperative care for the patient during the stoma surgeries, including reconstruction of intestinal and urinary transit, as well as the treatment of postoperative complications and specialized techniques. The TLM developed and agreed upon with the stakeholders, included objectives, goals, inputs, activities, target audience, results, as well as the logical relationship between the components.

E4 -Description of the organization and operation of the program
The SAPE-Niterói has existed since 2005, serving individuals of the municipality and metropolitan region II, consisting of the municipalities of São Gonçalo, Maricá, Itaborai, Rio Bonito, Silva Jardim, and Tanguá. It is located in the Orêncio de Freitas Municipal Hospital site, and is linked to the Vice Presidency of Hospital and Emergency Care of the Niterói Municipal Health Foundation. The activities are conducted on weekdays from 8am to 5 pm. The main motivation for the patients coming to the service is to obtain collection supplies. However, this activity is not always performed due to the continuous shortage of these objects.

E5 -Composition and development of the analysis and judgment matrix
Indicators related to supplies were applied to human, physical, and material resources. The minimum staff defined by the guidelines is: one physician; one nurse (trained in assisting people with ostomy); one psychologist; one nutritionist; and one social worker. The indicators related to physical and material resources were: clinical office; storage area with adequate furniture; sufficient collection supplies and adjuvants; meeting room; available standards, charts and protocols.     Step ladder 1 Hand washing sink, liquid soap dispenser, and paper towel 1 Exclusive bathroom with hygienic shower 1 Mirror with minimum dimensions of 120 × 50 cm 1

Treatment and / or referral of ostomy complications
Presence of physician to treat complications 5 10 DA. Presence of nurse to treat complications 5 Lack of specialized references for ostomy complications 0

Component -Permanent education and formation
Category Criteria or Indicators P MS VS

Self-care Groups
No self-care group meetings 0 3 DA; I. At least one self-care group meeting every two months 2 A monthly self-care group meeting 3

Knowledge of legal regulations
Lack of knowledge about legal instruments governing care services 0 3 DA; I At least the coordinator with knowledge of laws governing care services 2 All professionals with knowledge of laws governing care services 3

Participation in training courses
Lack of training of technicians, promoted by any management level, regarding the service 0 3 DA; I.
Training of 10% of the technicians, promoted by any management level, regarding the service, in the last 2 years. 2 Training of more than 10% of technicians, promoted by any level of management, related to the service, in the last 2 years.

E6 -Identification of patients and those involved in the evaluation.
During the workshops, the co-authors of this study, the coordinator, and other service professionals indicated the municipal manager and the service professionals as the main clients involved in the evaluation of SAPE-Niterói. The participation of the client was verified as a favorable element of the external context, with participation of members of the service management, indicating the priorities for the effective implementation of the actions.

E7 -Definition of evaluation procedure
The study participants found the assessment was necessary, to verify compliance with the guidelines and the achievement of program objectives, as well as to support decision-making based on the follow-up and appropriateness of the actions performed. The evaluative questions were formulated by means of an analysis of questions about program evaluation contained in the interviews, and agreement of the theoretical and logical models. In the end, the following questions were defined: (a) What is the degree of implementation in SAPE-Niterói? (b) Did the political-organizational and external contexts influence the implementation of the program? In what way? (c) Did the degree of implementation of SAPE-Niterói influence the results achieved?

DISCUSSION
The study showed that the SAPE documents were coherent, and that they supported and articulated guidelines and concepts of care.
The financial incentive for acquisition of equipment and other permanent materials was conceded by the National Health Fund (16)(17) . Such resources supported the activities of several care centers (18) .
Listed in the TLM were the relationships between objectives proposed by the intervention, accredited and qualified as Center I and II in 2012, with their intended activities and results, and the contextual analysis (19)(20) . Use of program evaluation results was shared by the main participants interested in the evaluation (21) . In this study, consensus development of the TLM was essential for the knowledge of the intervention, and the future use of the implementation and its results in the evaluation (22)(23)(24) . The intervention theory evidenced its basis in the political decision that preceded the operationalization, as demonstrated by Silva et al. (25) and SOBEST (26) , indicating that interventions are planned from evidenced problems, according to the context in which they are developed, and by the individuals involved in its planning and execution.
A review study on logic models from evaluability studies identified three studies in which the program theory was used as a methodological strategy for modeling, becoming a necessary part for future evaluation (27) . Other ES that used the TLM and MAJ demonstrated the importance of this preliminary step for the correct development of further evaluations (28)(29) .
With regards to the organization and operation of the program, orientation activities and stoma therapy consultations were the most present. The most frequent guidelines involved the rights and duties of the person with an ostomy, operation of the service, referral of patients to outpatient consultations with a nutritionist, surgeon and care by the stoma therapist, which includes guidance for self-care. One of the critical aspects of Ordinance 400 is the non-inclusion of the stoma therapist as a minimum staff component. As the stoma therapist has been in the Brazilian Occupation Code since 2013, the ordinance should have been updated. The SAPE activities converge with reports of some authors, such as Moraes (16) , for SAPE diagnostic analysis in Minas Gerais and Santos (30) . These include patient registration, dispensing of collection supplies, and individual appointments.
The discussion considering only dispensing of collection supplies is corroborated by Barros et al. (31) , and assumes that providing materials and equipment by the health units guarantees the patient's autonomy and understands the self-care as a manner of translating the ability to experience new challenges (32) . The public health policies designed for people with ostomy should include access to health needs, by ensuring many levels of care, so as to enable comprehensive care and not just limit care to supplying collection supplies (33) . Despite delimiting goals, objectives, and activities, Ordinance 400 establishes care guidelines for people with ostomy, but is not a public health program.
The study by Santos and Sawaia (34) analyzed the process of construction and reconstruction of the meanings of the concepts, "ostomy" and the "nurse's role in the management of an ostomy", after the experience of these professionals in the use of the bag. From experience, changes in practice were proposed in order to incorporate the affective, symbolic, and relational dimensions, critical to the management of ostomy patient care. The teaching of self-care assures the person with an ostomy the prevention of stoma complications, enabling independence in their care in relation to their family and health professionals. The help of a professional is relevant in the adaptive process, to clarify questions and reduce concerns regarding the acceptance of a new condition (32,(35)(36) .
Regarding the identification of users and those involved in the evaluation, several authors (21,37) highlight the broad involvement of participants, giving priority to obtaining group thoughts and perceptions, and reconciling differences when necessary. Trevisan (21) proposes working groups in the ES, including the evaluator, program team, and other representatives, to increase the likelihood of using the findings.

Study limitations
The research was restricted to a municipality in southeastern Brazil. Thus, it is suggested that other studies be performed involving other services. In addition, the interviews with few participants, due to the infrequent data collection period and the few professionals from the single municipal center, were also limitations of this study.

Contributions for the public health
Evaluability studies are intended to prepare evaluations of health care services, as they provide support for the planning and development of evaluative studies. Such studies can become the basis for decision making to improve or change the services and programs. The development of an evaluability study in SAPE, where the nurse has an essential role, can contribute as a model for assessment in the services already implemented in the several municipalities of the country.

FINAL CONSIDERATIONS
This ES contributed to the understanding of the intervention, and a better understanding of the conception and logic of its operation. The collaborative approach and methodology used met the objectives of the study and enabled the construction of an intervention (TLM and MAJ), with broad participation, which can increase the knowledge and commitment of those involved in the future use of evaluation results. The constituent elements of TLM show, among themselves, an interdependent relationship that enables the achievement of the expected outcomes.
A formative assessment based on the ES is recommended, including the identification of the aspects to be prioritized in order to achieve the expected outcomes. It is noteworthy that the proposed model could be revised. In addition, it is necessary to update the current ordinance that deals with the care of the person with ostomy, so that a requirement will be included in the legal text for a stoma therapist to be a member of the minimum team.