CONSTRUCTION AND VALIDATION OF AN INSTRUMENT TO EVALUATE ASSISTANCE TO THE PERSON WITH HYPERTENSION IN PRIMARY CARE¹

ABSTRACT

Objective:

describing the process of construction and validation of an instrument to evaluate the implantation of the Program of Assistance to Hypertensive Individuals in Basic Health Care Units.

Method:

a methodological study with a quantitative approach. The instrument was developed from October 2012 to March 2013, and in its construction was adopted the theoretical reference of Donabedian.

Results:

in the development of the instrument was used the reference proposed by Hartz e Silva, consisting of seven stages that include the definition and creation of evaluation criteria from the consultation of secondary sources and classification of the criteria according to the dimensions, besides the production of a Logical Model and an Analysis Matrix validated by 12 judges. The instrument addresses three dimensions: structural, care practices and organization of care.

Conclusion:

the instrument showed to be an adequate tool to evaluate health care programs for people with hypertension, as the matrix that subsidized its formulation was validated and its use allows identifying the strengths and weaknesses related to the implementation of the Program.

DESCRIPTORS:
Health evaluation; Hypertension; Primary Health Care

RESUMO

Objetivo:

descrever o processo de construção e validação de um instrumento para Avaliação da Implantação da Assistência à Pessoa com Hipertensão Arterial na Atenção Básica

Método:

estudo metodológico de abordagem quantitativa. O instrumento foi desenvolvido no período de outubro de 2012 a março de 2013, e na sua construção adotou-se o referencial teórico de Donabedian.

Resultados:

no desenvolvimento do instrumento foi utilizado o referencial proposto por Hartz e Silva, constituído de sete etapas que incluem a definição e criação de critérios de avaliação a partir da consulta a fontes secundárias e classificação dos critérios de acordo com as dimensões, além da produção de um Modelo Lógico e de uma Matriz de Análise validada por 12 juízes. O instrumento aborda três dimensões: estrutural, práticas assistenciais e organização da atenção.

Conclusão:

o instrumento mostrou ser uma ferramenta adequada para avaliar programas de atenção à saúde da pessoa com hipertensão arterial, na medida que a matriz que subsidiou sua formulação foi validada e sua utilização permite identificar as fortalezas e fragilidades relacionadas à implantação do Programa.

DESCRITORES:
Avaliação em saúde; Hipertensão; Atenção primária em saúde

RESUMEN

Objetivo:

describir el proceso de construcción y validación de un instrumento para la evaluación de la implantación de la asistencia a la persona con hipertensión arterial en la atención primaria

Método:

estudio metodológico de abordaje cuantitativo. El instrumento fue desarrollado en el período de octubre de 2012 a marzo de 2013, y en su construcción se adoptó el referencial teórico de Donabedian.

Resultados:

en el desarrollo del instrumento se utilizó el referencial propuesto por Hartz e Silva, constituido de siete etapas que incluyen la definición y creación de criterios de evaluación a partir de la consulta a fuentes secundarias y clasificación de los criterios de acuerdo con las dimensiones, además de la producción de un modelo lógico y de una matriz de análisis validada por 12 jueces. El instrumento aborda tres dimensiones: estructural, prácticas asistenciales y organización de la atención,

Conclusión:

el instrumento mostró ser una herramienta adecuada para evaluar programas de atención a la salud de la persona con hipertensión arterial, en la medida que la matriz que subsidió su formulación fue validada y su utilización permite identificar las fortalezas y fragilidades relacionadas a la implantación del Programa.

DESCRIPTORES:
Evaluación en salud; Hipertensión; Atención primaria en salud

INTRODUCTION

Changes in the health profile of the population cause great impact and create important challenges for health services, chronic diseases are part of this challenge and require multiple actions for their control and treatment. The work process needs to be reviewed, discussed and frequently adapted, as it is supported by verticalized programs and protocols which should be frequently reviewed so that they can incorporate the concept of assistance and meet the real needs of the people seeking these services. This requires willingness, interest and commitment on the part of managers and health professionals.

In 2001 the Ministry of Health initiated a number of actions, including the Brazillian national strategy for the Reorganization of Care for Arterial Hipertension and Diabetes Mellitus, in order to meet these demands and implement organized, resolutive and quality assistance in the public health network services. The initial purpose was to minimize the occurrence of injuries through actions such as updating and reeducation of the professionals regarding the basic network, ensuring diagnosis and linking the patient to health units for treatment and follow-up care.¹

However, in spite of these preventive and health promotion actions,²and the advances resulting from this new policy - such as increased access to consultations and medicines- the services, for the most part, continue to serve patients under a logic which is solely centered on disease and medicalization. In addition, existing information systems are only used for bureaucratic purposes and not for monitoring and evaluating programs, which could improve health actions among the users.³

The incorporation of evaluation in the health services can favor the effectiveness of the assistance practices, as well as monitoring the entire planning and management processes of policies or programs, making decision making and prioritization of assistance possible.³ Thus, the transformation of care provided to people with chronic non-communicable diseases is fundamental and should be considered as a daily activity for managers and health professionals. However, despite the advantages and possibilities of evaluating services, there are still few instruments available for evaluating health programs and services. The use and development of specific methods of evaluation are based on the need to provide a guide to users and health teams that favors and facilitates this action. In general, it is practically impossible for evaluation to occur in the daily fragmentation of the work process in health services. In attempting to implement the evaluation, many services conduct a superficial review of their activities and goals to solve the list of problems, but this activity is often limited to linear social relationships of cause and effect, which do not always stand up to a judicious analysis.

In this respect, the evaluation process does not have a definite outline, because in order to obtain an understanding of the status of a service, it is necessary to use appropriate material that is often not available. Thus, constructing instruments which make evaluation of the services possible is fundamental for advancements in the quality of care offered to users of the Brazilian Unified Health System(SUS).⁴ Motivated by this problem, the purpose of this study is to describe the construction and validation process and content of an instrument developed with the purpose to evaluate the level of implementation Brazillian national strategy for the Reorganization of Care for Arterial Hipertension in Basic Health Units (UBS).

METHOD

A methodological study with a quantitative approach. Methodological research investigates, organizes and analyzes data in order to construct, validate and evaluate research instruments and techniques.⁵

The instrument was developed between the period of October 2012 and March 2013, and its construction was based on the Donabedian theoretical framework, which consists of three categories: structure, process and result.⁶

The structure refers to the relatively stable characteristics of its providers, the instruments and resources, and the physical and operational conditions of the services. The process, in turn, refers to the activities, goods and services that are provided and how they are; and the results refer to the effects observed from the objectives proposed by the intervention, ⁶ that is to say, the verified changes in the health condition of patients related to knowledge and behaviors, as well as the satisfaction of those involved in assistance .⁷

As a methodological reference, the model proposed by Hartz and Silva was adopted, which consists of seven stages, each of which gives a foundation for the next one. They include: 1) Selection of secondary sources; 2) Creation and classification of the evaluation criteria according to the dimensions; 3) Production of the Logic Model; 4) Construction of the analysis matrix; 5) Validation by judges and statistical treatment; 6) Elaboration of the Implantation Analysis Tool / Instrument and Standardization of scores; 7) Implementation of the pilot study.⁸

The instrument was developed for initial application in a medium-sized municipality, with 357,077 thousand inhabitants, ⁹ located in the Northwest Region of the State of Paraná, which is home to the 15th Health Regional of the State of Paraná.

The implementation of the aforementioned steps, resulted in an instrument consisting of 121 questions which is used for evaluation the implementation of attention to the person with arterial hypertension in primary health care.

The development of the study met the national and international norms of ethics in research involving human beings and its project was approved by the Permanent Committee of Ethics in Research Involving Human Beings of the state university of Maringá (Opinion n. 170.666 / 2012).

Step 1 - Secondary source selection

The national secondary sources selected included: V Brazilian Guidelines on Arterial Hypertension from the Brazilian Society of Cardiology,¹⁰ National strategy for the Reorganization of Care for Arterial Hipertension and Diabetes Mellitus/¹¹ Primary Health Guidelines for HAS Secretariat for Health Care from the Ministry of Health,¹² Guidelines nº 2 and 3 for the Improvement in Quality for the Family Health Strategy Evaluation process,^13-14 Summary document for the evaluation of the National Program for improving the access quality of primary health care (PMAQ) 2012,¹⁵ Protocol for hypertension care in the municipality and Municipal Health Plan - 2010-2011.^16-17

The selected documents serve as practical guidelines for health professionals, which support professional training and the decision-making of managers. From the thorough reading of these documents, all the actions, activities and recommendations related to the infrastructure and human resources contained in these protocols that relate to the operationalization of a hypertensive care program were listed.

Step 2 - Creation and classification of evaluation criteria according to the dimensions

The actions and activities of the program, as well as the recommendations related to physical structure, supplies, and human resources, were classified in the structure and process dimensions. From this classification, criteria were developed which composed of a preliminary matrix that served as a guide for the construction of the logic model.

Step 3 - Construction of Logic Model

The logic model is the first step in evaluation planning and the assumptions that guide its formulation should be clearly explained, which corresponds to the objective image of the program.¹⁸

The model must present three key components: the organizational plan (the essential components), the plan of use (the services and related practices), and the impact of the program (results).¹⁹

In this respect, the aspects related to its organization, the work process and to the articulation with its elements are presented in a visual scheme and functions as a reference for the appreciation of the level of implementation of the program in Primary Health Care. At this stage the central concern was to ensure that all the elements described in the official documents were included in the developed logic model. Thus, the actions and activities listed in the matrix were classified as prevention, care and management actions. For each of the components, the integrated actions were identified, and the structure required to execute them was described. Finally, the expected results were identified.

Step 4 - Construction of the analysis matrix

The establishment of criteria, indicators and parameters or standards is a necessary condition for evaluation and may be arranged in a descriptive matrix of the program, which must be constructed from the logic model. These criteria will allow not only to describe the Program, but also to make a judgment and contribute to the production of results, with numerical and qualitative information.

The description of all the activities that are punctuated in the logical model were attempted during the construction of the evaluation matrix. The dimensions were based on the Donabedian theoretical model of evaluation⁶, which consists of three categories of analysis - structure, process and results, although it was identified in the construction of the logic model (step 3) the latter was not considered. The aspects covered in the structure category are allocated in the structural dimension and dimension of care practices, and those related to the process category in the health care organization dimension.

The structural dimension encompassed five sub-dimensions: physical area, materials and equipment, emergency equipment, materials for health education, supplies, complementary exams, medicines and human resources. The assistance practices dimension was divided into two sub-dimensions: health promotion and individual care. Finally, the organization of attention dimension was subdivided into managerial sub-dimension and aspects of care organization, according to the author’s perception.

Step 5 - Validation of the matrix by judges and statistical treatment

The validation of the matrix by specialist evaluation occurred using the foundations of the Delphi Technique, which require judges with extensive experience in the subject in question or who have worked for some time in the area to evaluate and judge the matrix. It is defined as a group process technique and is intended to obtain, compare, and direct the judgment of the experts to a consensus on a particular topic.²⁰

Considering that the validity of an instrument is related to its accuracy in measuring what it is intended to measure,²¹ it will only be valid when its construction and applicability allow the true measurement of what is intended to be measured. The best known techniques for instrument validation are: content, appearance, criterion, and construct validity.²² In this study content validity was used, which determines if the content and measure of an instrument effectively explores the requirements for measuring a certain phenomenon. Content validity consists of a subjective evaluation based on the judgment of content experts in order to determine if the instrument explores all dimensions and domains relevant to the concept or construct under study.

Thus, the matrix was submitted to a group of 12 judges formed by: two physicians and two nurses who were members of the Family Health Strategy of the city under study for more than two years; Three managers, two cardiology specialists; Two doctoral professors and researchers from the evaluation area; and one nurse with a doctorate in the area of health evaluation.

Each participant received the matrix of dimensions and the guidelines by e-mail and were asked to fill them in. All judges were oriented to analyze the matrix and assign scores of zero (little or no importance) to ten points (very important criterion) according to the individual conception regarding to the relative importance of each of the criteria listed in the matrix for the evaluation of the program. The judges could also suggest the exclusion or inclusion of other aspects or even modify the classification of criteria, dimensions and sub dimensions.

The scores attributed by the respondents were consolidated and recorded on an Excel® worksheet. In succession, the arithmetic mean of each criterion was calculated from the sum of the values assigned by the judges. The standard deviation was also calculated, which served to estimate the degree or lack of consensus among the judges regarding each of the criteria.

Subsequently, in order to define the criteria to be included in the instrument under construction, the same cut-off points proposed by the idealizers of this evaluation model²³ were used and adopted in a study evaluating the implementation of a program related to reproductive health,²⁴ which included: an average of more than seven was considered important; below seven was classified as not important and, therefore, excluded from the objective image. In addition, all criteria with a standard deviation lower than three were considered consensual and those with a standard deviation higher than three were considered non-consensual and for this reason were also excluded. Therefore, criteria with an average of seven or more and a standard deviation equal to or greater than three, although important, were not considered for the composition of the objective image because they were not consensual.

It should be noted that the adoption of a consensus not only increases the possibility of criteria validity, but also confers greater legitimacy to the evaluation process.

The image- the objective of the assessment of the hypertension care program, resulting from the documentary analysis and judges’ judgment, included 121 items distributed in the three dimensions. The structural dimension consists of six sub-dimensions: physical area with 15 items, materials and equipment (19 items), emergency equipment (seven items), materials for health education (five items), supplies, and human resources (14 items). The practical assistance dimension consists of the sub dimensions health promotion (14 items) and individual assistance (11 items). Finally, the third dimension - organization of attention - has no sub-dimension and consists of 17 items.

Figure 1
Logical Model of the Program

It should be highlighted that the calculation of the standard deviation was the indicator for instrument reliability, and the lower the standard deviation, the greater the degree of item reliability. In addition, all criteria were extracted from official documents from the Ministry of Health and the protocol regarding hypertension from the studied municipality.

In relation to the items initially proposed from the revision of the official documents, only five were excluded, and this was due to a standard deviation greater than three. In addition, two more were eliminated because, in addition to the high standard deviation (greater than three), they obtained a score lower than seven, which demonstrates the consistency of the matrix.

Step 6 - Elaboration of the Instrument for Program Implementation Analysis and Standardization of Scores

Based on the relative importance of each dimension and the criteria defined from the evaluation and consensus of the specialists, the instrument was developed to evaluate the level of implementation of the Programa de Atenção ao Hipertenso no município de Maringá-PR.

All criteria validated in the matrix were used in the elaboration of questions in the instrument, in such a way that each question evaluated, individually or jointly, indicated the level of implementation of the Program in: elementary, intermediate and advanced. For this, a complex process of standardization of the scores was used. Initially, in the attributed score and in the definition of the scores of each question, the average scores assigned by the evaluators in the matrix were considered.

Thus, questions whose criteria averaged between 7.0 and 8.0 were given a point; Those who obtained an average of between 8.1 and 9.0 received two points, and those that obtained an average of between 9.1 and 10.0 received three points. The questions whose answers were not dichotomous (yes/no) were graded according to the frequency of occurrence, so that their score also had to be distributed. For example, questions with a value of three three had their answers punctuated as follows: always (3 points), almost always (2 points), sometimes (1 point), almost never (0.5 points) and never (0.0).

Subsequently, for the classification of the level of implantation, the maximum total of points that could be obtained in the three dimensions (328 points) and added and divided by three to equalize the total points to be reached in each of the three dimensions (109 , 33 points). Then, the value which was to be assigned to the questions was defined, which was different in each dimension, since the number of questions was not equal. For example, the structural dimension consists of 79 items, of which seven were worth one point and became worth 0.52; 16 items that were worth two points and became worth 1.05; And 56 items that were worth three points and became worth 1.58, totaling 109.33 points. In the practical assistance dimension which consisted of 27 items, those worth three points became worth 4.55 points, and in the assistance organization dimension, 17 items that were worth one point became worth 6.43 points.

As defined in another study, ²⁵ the sum of the points of all questions determined the level of implementation of the assessed item classified as elementary, intermediate and advanced. The first tertile (0 to 33.33%) of the question value represented the limit between the elementary and intermediate level; the second tertile (33.34 to 66.66%) represented the limit between the intermediate and the advanced; and the third (66.67-100%) represented the most advanced level.

Step 7 - Application of the pilot test

The pilot test was performed with three Family Health Strategy ESF nurses working in three Basic Health Units. As a result, it was identified that some items should be complemented or subdivided without, however, changing the essence of the evaluated aspect. For example, in the emergency equipment item, a question related to the existence of training to attend to hypertensive emergencies was added. One more item was added to the question which investigated the performance of home visits: visitation to individuals with motor disabilities, in addition it was subdivided in order to highlight three conditions: non adherent patients, bedridden patients or patients with motor incapacity. The item reference system and counter-referral for cardiologists and specialized examinations were divided into reference system and counter-referral system.

REFLECTIONS ON THE MATRIX CONSTRUCTION PROCESS

A general analysis indicates that most of the criteria listed in the matrix obtained a standard deviation of less than three, which shows that there is a great consensus in most of the selected items, reinforcing the importance of having them defined through consultation with official documents.

In addition, during the pilot study with three nurses in the Family Health Strategy, the nurses considered the issues of the instrument adequate in terms of clarity and objectivity.

In the structural dimension, referring to the physical area, the items that obtained the highest averages and, therefore, were valued most by the judges were: nursing consultation rooms which guarantee patient privacy, valorization of the nursing consultation in the Program, a specific room for the collection of laboratory tests and a room for the community health agents.

These aspects demonstrated the concern which the judges had in relation to the new care model, which incorporates the ACS and the nurse as fundamental actors in health promotion. Regarding materials and equipment items, sufficient sphygmomanometers and stethoscopes for obese adults were those that reached the highest averages, demonstrating the valorization for quality in relation to blood pressure measurement of these individuals.

Attention was drawn to the importance attributed to equipment and training of professionals to attend the emergency room, which obtained averages of nine and a standard deviation of less than one. Although primary care is not the gateway to emergencies, the judges’ notes demonstrate the importance they attach to the fact that the Health Units should be properly equipped and the professionals adequately trained in order to treat hypertensive emergencies. This may draw patients closer to primary care, since adequate care, closer to home with faster service, decreases the possibilities of greater sequelae related to the complications of this condition.

Another item highly valued by the judges was the availability of material for health education with a mean of 9.25, indicating the valorization of health promotion activities. Regarding the sub-dimension inputs, all items received an average above nine, confirming the importance of pharmaceutical care and the access to medicines and other supplies in the care of hypertensive patients. However, at the same time it demonstrates a centrality and overvaluation of drug treatment and access to specialized exams.

The sub-dimension for human resources obtained a positive evaluation by the judges, including suggestions to include the oral health team and a physiotherapist, therefore indicating a perspective of expansion to the list of professionals listed in the analysis matrix, but this was not consensual and therefore the professionals were not included in the instrument, which could occur in the future.

Regarding the assistance practices dimension, there was a great consensus in the health promotion sub-division, especially in relation to the strategies which track new hypertensive patients, which consist of strategies that favor treatment adherence and stimulate healthy lifestyles, which presented scores higher than those obtained while undergoing drug treatment.

Regarding the sub-dimension of individual care, the items: physical examination of the patient, according to the protocol of the Ministry of Health and Brazilian Society of Cardiology, guidelines regarding risk factors for hypertension, with aspects of non-pharmaceutical treatment, elaboration of the care plan for hypertensive individuals at the time of the home visit, and the team seizes the opportunity of the visit to trace new cases , were consensus among the judges receiving a mean of 10 and a standard deviation of 0.0.

In the dimension of care organization, attention was drawn to the fact that reception, as an institutional practice was excluded since although it obtained an average of 8.67, the standard deviation was 3.085. Therefore, user embracement was considered important by the judges, however it was not consensual. It is believed that one of the reasons for the lack of consensus was the difference in the conception of this idea. However, user embracement is considered essential for the humanization and organization of health actions and for the classification of risk and, therefore, must be at the forefront of all interventions, whether in a promotional, preventive, curative or caring nature. Thus, it is noteworthy that, although it is absent from the instrument elaborated, the authors consider that user embracement, with risk classification based on the humanistic approach, allows for the creation of positive relationships between the user and the health team.²⁶

It should be noted that the judges’ validation of the matrix demonstrated that the improvement of care for hypertensive patients is not only a result of a criterion, such as access to medicines, but depends on a well-articulated set of actions that triggers a new assistance dynamic aimed at integrality and health promotion.

Finally, it is emphasized that none of the judges proposed any change in the classification of the criteria between different dimensions and sub dimensions, demonstrating consistency in the initial classification and consequently in the final proposal.

CONCLUSION

Institutionalization of evaluation of the everyday work in health services is still considered a challenge, and the improvement of monitoring and evaluation practices remain necessary. In this respect, the instrument proved to be an adequate tool to evaluate hypertensive health care programs because the matrix that assisted its formulation was validated by judges and also because it allows to identify strengths and weaknesses related to the implantation of this program, revealed in the dimensions and sub-dimensions.

The expansion of Primary Health Care services must determine an evaluative culture and, despite its incipient implementation, it is necessary to consolidate it not only in order to obtain better results, but to contribute to the consensus regarding the principles of Primary Health Care APS which include valorization of initiatives and multi-strategic actions and the many diverse dimensions of the services.

The challenge of developing an instrument to evaluate the implementation of the Hypertension Care Program in the municipality of Maringá included the requirement of an evaluation concept, the construction of a logic model and the definition of an evaluation matrix, obtaining its validation by specialists in order to apply and use it in the UBSs.

The importance of this instrument was evident during the construction process considering that its elaboration occurred from the Brazilian guidelines for coping with chronic non-communicable diseases. This importance was revealed through the consensus of the judges which pointed to the agreement and reliability of the proposed indicators, resulting in it being an appropriate instrument for the evaluation of health programs of assistance to the hypertensive patient and applicable to the planning of health and management services.

Thus, it is considered that following the seven steps proposed in the methodological framework adopted in the construction of the instrument was fundamental. In addition, the importance of building instruments based on legal aspects is reinforced, but, on the other hand, that they meet the reality of each municipality. Although the developed instrument can be used in municipalities with different population sizes, it is essential that each municipality or service incorporates the evaluation of everyday work in the health services, using instruments purposefully constructed to evaluate other scenarios, even if it may be necessary to make some adjustments and adaptations to these cases.

Thus, the present study opens the way to new proposals, such as the practical application of the instrument, which will enable the identification of the degree of implementation of the Hypertension Assistance Program in different scenarios and different contexts.

However, despite the fact that some content / aspects approaches in the instrument was guaranteed by the construction process itself, even with consensus among the judges, to consolidate its use in different scenarios and contexts, it would be advisable to test its internal agreement and reliability and also its external validity, in order to verify the consistency for different contexts and the effectiveness in explaining the observed results.

Finally, the non-inclusion of the political assistance dimension in the list of aspects addressed by the instrument may be a limitation of the study, since it would allow a more global understanding of how the Hypertension Assistance Program is being implemented, since the political scenario Institutional structure explains, at least in part, the success or failure of this deployment. Similarly, the incorporation of workers and users in the construction process of the instrument would allow closer approximation to the reality of the assistance.

• 1
  Extracted from thesis - Evaluation of the implantation of the program of assistance to people with arterial hypertension in a municipality of the State of Paraná, presented to the Postgraduate Program in Nursing, Universidade Estadual de Maringá (UEM), in 2013.

REFERÊNCIAS

• ¹
  Roese A, Pinto JM, Gerhart TE, Silveir DT. Perfil de hipertensão arterial sistêmica e de Diabetes Mellitus a partir de dados nacionais em municípios de pequeno porte no Rio Grande do Sul. Rev. APS. 2011; 14(1):75-84.
• ²
  Barreto MS, MSS. Participação familiar no tratamento da hipertensão arterial na perspectiva do doente. Texto Contexto Enferm. 2014, 23(1):38-46.
• ³
  Felisberto E, Freese E, Alves CK de A, Bezerra LCA, Samico I. Política de monitoramento e avaliação da atenção básica no Brasil de 2003 a 2006: contextualizando sua implantação e efeitos. Rev Bras Saúde Mater Infant. 2009; 9(3):339-57.
• ⁴
  Felisberto E. Monitoramento e avaliação na atenção básica. Cienc. Saude Colet. 2011; 16(2):523-33.
• ⁵
  Polit DF, Beck CT. Fundamentos de pesquisa em enfermagem: avaliação de evidências para a prática de enfermagem. 7ª ed. Porto Alegre: Artmed, 2011.
• ⁶
  Donabedian A. Explorations in quality assessment and monitoring: the definitions of quality and approaches to its assessment. Ann Harbor: Health Administration Press I, 1994.
• ⁷
  Kantorski LP, Machado RA, Lemões MAM, Quadros LCM de, Coimbra VCC, Jardim VMR. Avaliação da estrutura e processo na visão de familiares e usuários de saúde mental. Cienc. Cuid Saúde, 2012; 11(1):173-80.
• ⁸
  Hartz ZMA, Silva LM, organizadores. Avaliação em saúde: dos modelos teórico à prática na avaliação de programas e sistemas de saúde. Salvador: EDUFBA; Rio de Janeiro (RJ): Fiocruz, 2005.
• ⁹
  Instituto Brasileiro de Geografia e Estatística. Cidades. Rio de Janeiro (RJ):IBGE; 2013 [cited 2013 Abr 12]. Available from: http://cidades.ibge.gov.br/xtras/perfil.php?lang=&codmun= 411520&search=parana|maringa
  » http://cidades.ibge.gov.br/xtras/perfil.php?lang=&codmun= 411520&search=parana|maringa
• ¹⁰
  Sociedade Brasileira de Cardiologia, Sociedade Brasileira de Hipertensão, Sociedade Brasileira de Nefrologia. Diretrizes Brasileiras de Hipertensão, Arq Bras Cardiologia. 2010; 95(supl 1):1-51.
• ¹¹
  Ministério da Saúde (BR). Secretaria de Atenção a Saúde. Departamento de Atenção Básica. Hipertensão arterial sistêmica para o Sistema Único de Saúde. Brasília (DF):MS; 2006.
• ¹²
  Ministério da Saúde (BR). Secretária de Políticas de Saúde. Plano de reorganização da atenção à hipertensão arterial e ao diabetes mellitus: manual de hipertensão arterial e diabetes mellitus. Brasília (DF): MS; 2001.
• ¹³
  Ministério da Saúde (BR). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Manual do instrumento de avaliação da atenção primária à saúde. Brasília (DF): MS; 2010.
• ¹⁴
  Ministério da Saúde (BR). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Manual do instrumento de avaliação da atenção primária à saúde. Brasília (DF): MS; 2010.
• ¹⁵
  Ministério da Saúde (BR). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): manual instrutivo. Brasília (DF): MS; 2012.
• ¹⁶
  Prefeitura Municipal de Maringá (PR). Protocolo de atenção ao hipertenso. Maringá (PR): Prefeitura Municipal de Maringá; 2004.
• ¹⁷
  Prefeitura Municipal de Maringá (PR). Plano municipal de saúde 2010-2013. Maringá (PR): Prefeitura Municipal de Maringá; 2010.
• ¹⁸
  Medina MG, Silva GAP, Rosana A, Hartz ZMA. Uso de modelos teóricos na avaliação em saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Silva LMV, organizadores. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador (BA): Ed UFBA; 2005. p. 41-63.
• ¹⁹
  Hartz ZMA, Felisberto E, Silva LMV. Meta avaliação da atenção básica à saúde: teoria e prática. Rio de Janeiro (RJ): Fiocruz; 2008.
• ²⁰
  Spínola AWP. Delfos: proposta tecnólogica alternativa. São Paulo (SP): Faculdade de Saúde Pública da USP; 1984.
• ²¹
  Urbina S. Essentials of psychological testing. New Jersey (US): John Wiley & Sons, Inc., 2004.
• ²²
  Belluci Júnior JA, Matsuda LM. Construção e validação de instrumento para avaliação do acolhimento com classificação de risco. Rev Bras Enferm, 2012: 65(5):751-7.
• ²³
  Souza LEPF, Silva LMV, Hartz ZMA. Conferência de consenso sobre a imagem-objetivo da descentralização da atenção à saúde no Brasil. In: Hartz ZMA, Silva LMV, organizadores. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador (BA): Ed UFBA; 2005. p. 65-102.
• ²⁴
  Nagahama EEI. Avaliação da implantação de serviços de saúde reprodutiva no município de Maringá, Paraná, Brasil. Cad. Saúde Pública, 2009: 25(2):279-90.
• ²⁵
  Silva LMV, Hartz ZMA, Chaves SCM, Silva GAP. Metodologia para análise da implantação de processos relacionados à descentralização da atenção à saúde no Brasil. In: Hartz ZMA, Silva LMV, organizadores. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador (BA): Ed UFBA; 2005. p. 201-53.
• ²⁶
  Ministério da Saúde (BR). Secretaria de Atenção à Saúde. Núcleo Técnico da Política Nacional de Humanização. Acolhimento nas práticas de produção de saúde: humanização. 2a ed. Brasília (DF): MS; 2006.

Publication Dates

• Publication in this collection
  2017

History

• Received
  02 June 2014
• Accepted
  11 Sept 2015