Construction and validation of a matrix for normative evaluation of the integrated health system of the borders*

Objective: to build and validate a matrix for normative evaluation of the Integrated Health System of Borders. Method: a methodological study, composed by the construction of an evaluation matrix elaborated in three stages: elaboration of the logical model, containing the triad of structure, process and result; definition of evaluative questions and appearance and content validation of the matrix. Appearance and content validation were performed simultaneously by seven judges. For data collection, an online questionnaire and the Delphi technique were used and, for analysis, the Content Validity Index and Content Validity Ratio. Results: the evaluation matrix containing 24 questions was submitted to two evaluations for its appearance and content validation. In the first, the overall mean Content Validity Index was 99.40% and the Content Validity Ratio was 0.90. In the second, the Content Validity Index was 100% and the Content Validity Ratio, 1.0; there were no new proposals and the matrix was made up of 24 questions. The matrix was considered intelligible in terms of appearance validation. Conclusion: the evaluation matrix of the Integrated Health System of the Borders is validated in terms of appearance and content for analyzing the performance of public actions and policies in border regions.


Introduction
The concept of borders has been improved over the years, ceasing to be seen only as barriers, and becoming contact zones and places for regional integration (1) .
In the international borders, between the local and the international space, it is where the border peoples are located, which are the people who live, articulate and establish bonds. In this context, pendular migration takes place, which consists of people moving from their country of origin to the neighboring country, for work, study, or other reasons (2) .
The mobility of people across international borders is a worldwide phenomenon and can cause divergences when the countries present economic discrepancies, as is the case of the United States of America (USA), which adopts rigid standards on the border with Mexico, due to the Mexicans searching for better living conditions in the USA, in addition to the flow of illegal drugs and due to trade; in contrast, this problem does not occur on the border between the USA and Canada, as the countries have economic equivalence (3) .
In the case of Brazil, the fact that it has a free national health system has encouraged the migration of people from other Latin American countries to seek medical care; this phenomenon has caused difficulties in controlling and eliminating diseases such as leprosy (4) .
The surplus foreign population is not counted for financial transfer in border municipalities (5) .
Public policies for borders and transnational integration are recent; previously, they were restricted to security and occupation of the territory. Only in 2005 did the Ministry of Health (MoH) launch the Integrated Border System (SIS Fronteiras) in order to carry out a local health diagnosis, promote the qualification of the professionals, and improve the health service network in border municipalities, effective from 2005 to 2014 (6) .
However, the execution of SIS Fronteiras did not end in 2014; the activities extended over the following years, as planned by each municipality to meet the established goals.
It is considered that the evaluation of public actions and policies in border regions is important for facing the local problems, as well as for regional development (6) . This study aimed to build and validate a matrix for normative evaluation of SIS Fronteiras.  to identify elements such as human resources, financing, cooperation terms, and local health diagnoses, among others. The data were organized in structure, process and results and supported the development of the logical model, which guided the construction of the evaluation matrix for SIS Fronteiras (8) .

Method
Based on the logical model, evaluation questions were identified, which were later validated by judges, according to the criteria of clarity, objectivity and relevance (9) . The validation of the content and appearance of the questions took place from November 2017 to February 2018, using the Delphi Technique, described below (10)(11) .
The study population was composed of seven judges  The questionnaire was designed with 24 evaluative questions, eight of which were related to structure, eight to process and eight to result, which were subsequently subjected to content and appearance validation by judges.
For the evaluation of the questions, a five-point Likert scale was used, being 1 (totally disagree), 2 (partially disagree), 3 (neither disagree nor agree), 4 (partially agree) and 5 (totally agree). The judges were given the option to suggest alternative texts on each question. The deadline for returning the evaluation matrix was 15 days from receipt (10) .
A 75% agreement percentage was considered for each evaluated item; in the case of failure to obtain the degree of agreement between the judges, new cycles should be carried out (11) . At the end, a thank you e-mail was sent to the participants for their collaboration in the research.
Content validity was carried out to analyze the completeness of the proposed items and whether they reflected the theoretical framework and the SIS Fronteiras guidelines, in addition to appearance validation because, despite not being a sophisticated technique, it was intended to observe how content was presented, as well as clarity and ease of reading and the adequacy of the item to each dimension (9,12) .
For data analysis, the Content Validity Index (CVI) was used to measure the degree of agreement between the judges based on their answers. The expected consensus level for this study was 75% (11) .
The CVI was calculated from the sum of answers 4 and 5 (partially agree and totally agree) of each judge in each question of the questionnaire, dividing this sum by the total number of answers (CVI=Number of answers 4 and 5/Number total of responses X 100). The mean CVI of the dimensions was also calculated, adding each percentage obtained in each dimension and dividing by three, that is, by the three dimensions used.
Content Validity Ratio (CVR) (13) was also used. The The CVI varies between 0 and 1; the closer to 1, the better the item will perform according to the judges.
The CVR varies between -1 and 1, the minimum value depending on the number of judges. In this study it was considered as 0.99 (13) .
Appearance validation was performed considering the requirement for clarity of content and the semantics of the questions; thus, it was verified that all items were comprehensible to the respondent population, with opportunities for descriptive manifestations for each question (14) .

Results
In the logical model, the necessary elements for the development of SIS Fronteiras were considered, such as: With regard to appearance validation, the judges presented two proposals for the structure dimension; two suggestions for process; and a recommendation for the result dimension. The suggestions were analyzed and accepted with the necessary adjustments, as shown in Figure 2.

st
Here, in question 5, which refers to "room, equipment and materials for the first assistance of urgencies for 100% of the described population, contemplating", it was necessary to say where it is described and what should be contemplated.

nd
The question was revised and the information referring to the documents that should include the description of the population was included and the word "contemplating" was excluded, since it is already stated that it was equipment and materials.  In question 5, there is the word urgencies, I think it is not necessary, as it says that it is for first care.

nd
In this case, the word urgencies was maintained, for clarification to the reader, as the agreed service was for the first assistance of urgency cases.

st
In question 10, regarding the municipality to elaborate the client description, within 30 days from the Term of Adherence, I suggest extending the term to 30 to 40 days.

nd
The item was not changed, considering that the term of up to 30 days was established under Ordinance 1,189 of June 5th, 2006.

st
In question 15, which deals with the insertion of SIS Fronteiras in the Municipal Health Plan, it is necessary to clarify which period it refers to as, since the implementation of SIS Fronteiras in 2006, there are already at least four plans prepared in that period, with this, I suggest that they are in the Municipal Health Plans.
2 nd Suggestion accepted, changing the text to: "The Municipal Health Plans, elaborated during the term and execution of SIS Fronteiras, contain the Local Diagnosis and the Operational Plan".

st
In question 23, there is a contingency plan for unusual events, but they were not exclusive to SIS Fronteiras, I think they could be removed.

nd
The contingency plan for unusual events is a requirement for the release of financial resources from SIS Fronteiras; therefore, it was maintained.

The Ministry of Health allocated financial resources for the development of SIS Fronteiras.
2. The municipality provided a room for the coordination of the project, containing 1 computer; 1 printer; 1 nobreak; 1 table; 1 typing chair and 1 cabinet.
3. The municipality provided infrastructure for the implementation of health networks, including: computers; information systems; primary and tertiary care points, to support primary health care, with specialized actions at outpatient and hospital levels, diagnostic and therapeutic support, meeting room and car for commuting.
4. The municipality provided offices, with equipment, furniture and materials for basic outpatient care.
5. The municipality provided a room, equipment and materials for first emergency care to 100% of the population described in the SIS Fronteiras Term of Adherence and Operational Plan.
6. Infrastructure, made available by the municipality, for new services proposed in the Local Health Diagnosis, which may include constructions; reforms; acquisition of equipment; vehicle purchase; consumables, etc.
Rev. Latino-Am. Enfermagem 2021;29:e3433. The matrix constructed and validated in this study obtained a positive evaluation, with an overall mean CVI of 100% for the three dimensions, namely: structure, process and results; and CVR of 1.0, therefore being considered adequate and superior to the established 75% cutoff. In the literature, the cutoff point for reaching consensus among judges ranges from 50% to 80%, for Nursing research, a percentile greater than 75% (11) has been considered.
In the appearance validation, the matrix was considered intelligible, after the adjustments suggested validity (14,17) .
In validated matrix aspects of structure, process and result were addressed and it will allow managers, This is due to the asymmetry of the health systems of the countries bordering Brazil and to the demand of foreigners for health care; and SIS Fronteiras, through financial transfer, was a crucial strategy to improve the situation in these locations (18) . 18. Qualified professionals in terms of management (economics, planning, organization of the health systems and supplementary health management), health care (protocols and specific border procedures), health surveillance, indigenous health, information systems and social control.   (19) .
In the process dimension, in the first round a CVI The mobility of patients to other countries for the purpose of health treatment, although it has always existed, has considerably increased in the last decade due to the greater commercial integration between different countries of the world. However, the main causes for patients crossing borders seeking health care are the difficulty of access or the lack of necessary services in their countries of origin (21) .  (22) .
A study carried out in Costa Rica identified that the country also experiences a high demand from foreigners and people in cross-border transit and, in view of this situation, adopted the strategy to establish a shelter home for the purpose of humanitarian aid and the identification of this immigrant population (23) .
In this light, in Brazil the local health diagnosis of SIS Fronteiras was pointed out by the judges, in this study, as an important evaluative requirement of this public policy and represented a relevant action for the identification of the foreign population.
These aspects corroborate with a national study that considers that, in Brazil, there is no availability of accurate data regarding care in border regions, and that this information is often camouflaged and/or omitted by the patients to gain access to the services of the Unified Health System (24) .
The Delphi technique was presented as an important strategy for obtaining consensus among the judges and to conduct the necessary adjustments regarding the content and appearance of some questions in the evaluation matrix. The satisfaction and recognition of the relevance of the Delphi Technique for this study corroborates with other research studies developed with the use of this method, carried out through consensus of experts (25) .
Another potentiality of the Delphi technique in this study was the use of Google Forms, currently an online technology that is gaining prominence in scientific research, due to the ease of contacting people without having to travel and at lower costs. In this sense, research studies developed on the Delphi technique show that one of its advantages is the possibility of not being face-to-face, as it adds wealth to the study with the participation of specialists in specific themes from different geographical locations (26)(27) .
Despite the numerous advantages perceived in the use of the method, we identified some limitations, such as the delay in acceptance and the difficulty in explaining the relevance of the study, without physical contact with the participant. Another limitation observed in the present study refers to the fact that Brazil has a vast territorial extension, with specificities in different regions; therefore, there may be certain difficulty in the applicability of the matrix throughout the national territory.
Methodological studies are viewed with great relevance by the scientific community. Thus, this study, through the logical model and the evaluation matrix, has as strength the availability of a tool that will allow for the evaluation of SIS Fronteiras and, in this way, it can contribute to the formulation of new public health policies aimed at international standards of border regions.

Conclusion
The normative evaluation matrix of SIS Fronteiras, elaborated and validated in this study, proved to be an Rev. Latino-Am. Enfermagem 2021;29:e3433.
innovative tool in the evaluation of public health policies in international border regions.
The content and appearance validity of the matrix was considered adequate, in view of the careful process of analyzing its items and the suggestions to improve it, thus ensuring that the evaluation questions and the structure meet the SIS Fronteiras guidelines and are in accordance with the theoretical assessment approach.
The need to expand research studies related to health issues in border regions is highlighted, as well as the development of new tools to assess these very specific realities.