Perceptions and evaluations of front-line health workers regarding the Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ): a mixed-method approach

As percepções e avaliações dos profissionais de saúde que trabalham na ponta em relação ao Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): uma abordagem com métodos mistos

Percepciones y evaluaciones de los trabajadores de salud de primera línea, en relación con el Programa Nacional de Mejora del Acceso y Calidad de la Atención Básica (PMAQ): un enfoque metodológico mixto

Abstract:

Although it is well known that a successful implementation depends on the front-liners’ knowledge and participation, as well as on the organizational capacity of the institutions involved, we still know little about how front-line health workers have been involved in the implementation of the Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ). This paper develops a contingent mixed-method approach to explore the perceptions of front-line health workers - managers, nurses, community health workers, and doctors - regarding the PMAQ (2^nd round), and their evaluations concerning health unit organizational capacity. The research is guided by three relevant inter-related concepts from implementation theory: policy knowledge, participation, and organizational capacity. One hundred and twenty-seven health workers from 12 primary health care units in Goiânia, Goiás State, Brazil, answered semi-structured questionnaires, seeking to collect data on reasons for adherence, forms of participation, perceived impact (open-ended questions), and evaluation of organizational capacity (score between 0-10). Content analyses of qualitative data enabled us to categorize the variables “level of perceived impact of PMAQ” and “reasons for adhering to PMAQ”. The calculation and aggregation of the means for the scores given for organizational capacity enabled us to classify distinct levels of organizational capacity. We finally integrated both variables (Perceived-Impact and Organizational-Capacity) through cross-tabulation and the narrative. Results show that nurses are the main type of professional participating. The low organizational capacity and little policy knowledge affected workers participation in and their perceptions of the PMAQ.

Keywords:
Public Policy; Health Policy; Incentive Reimbursement; Primary Health Care

Resumo:

Sabe-se que a implementação bem-sucedida de um programa depende do conhecimento e da participação dos profissionais que trabalham na ponta, além da capacidade organizacional das instituições envolvidas. No entanto, ainda sabemos pouco sobre o envolvimento desses profissionais na implementação do Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ). O artigo desenvolve uma abordagem de métodos mistos de tipo contingente para explorar as percepções dos profissionais de saúde na ponta - gestores locais, enfermeiros, agentes comunitários de saúde e médicos - sobre o segundo ciclo do PMAQ, além de relacionar essas percepções às avaliações a respeito da capacidade organizacional da unidade de saúde. O estudo é orientado por três conceitos inter-relacionados e relevantes da teoria da implementação: conhecimento das políticas, participação e capacidade organizacional. Cento e vinte e sete profissionais de saúde de 12 unidades de atenção primária em Goiânia, Goiás, Brasil, responderam questionários semiestruturados, buscando coletar dados sobre os motivos pela adesão, formas de participação, impacto percebido (perguntas abertas) e avaliação da capacidade organizacional (pontuações de 0 a 10). As análises de conteúdo dos dados qualitativos permitiram categorizar as variáveis “nível percebido de impacto do PMAQ” e “motivos pela adesão ao PMAQ”. Os cálculos e agregação da média das pontuações para capacidade organizacional permitiram classificar os diferentes níveis dessa capacidade. Finalmente, foram integradas as duas variáveis (Impacto Percebido e Capacidade Organizacional) através da tabulação cruzada e da narrativa. Os resultados mostram que os enfermeiros são o principal tipo de profissional que participa no programa. A baixa capacidade organizacional e o conhecimento limitado da política afetaram a participação dos professionais de saúde e suas percepções em relação ao PMAQ.

Palavras-chave:
Política Pública; Política de Saúde; Reembolso de Incentivo; Atenção Primária à Saúde

Resumen:

Pese a que es bien sabido que una implementación exitosa depende del conocimiento y participación de los trabajadores de salud de primera línea, así como la capacidad organizativa de las instituciones involucradas, todavía sabemos poco sobre cómo los trabajadores de salud en primera línea de atención han sido involucrados en la implementación del Programa Nacional de Mejora del Acceso y Calidad de la Atención Básica (PMAQ) brasileño. Este trabajo desarrolla un enfoque metodológico mixto aleatorio para investigar las percepciones de los trabajadores de primera línea de la salud -gestores, enfermeras, trabajadores comunitarios de cuidados de salud, y doctores- en relación con el PMAQ (segunda fase), y asociarlos con sus evaluaciones respecto a la capacidad organizativa de las unidades de salud. La investigación está guiada por tres conceptos relevantes interrelacionados, provenientes de la teoría de la implementación: política de conocimiento, participación, y capacidad organizativa. Ciento veintisiete trabajadores de salud, procedentes de 12 unidades de atención primaria de salud en Goiânia, respondieron a cuestionarios semiestructurados, buscando recoger datos sobre: adherencia, formas de participación, impacto percibido (preguntas de final abierto), y evaluación de la capacidad organizativa (notas entre 0-10). El contenido de los análisis cualitativos de datos nos permitió categorizar las variables “nivel del impacto percibido de PMAQ” y “razones de adherencia al PMAQ”. El cálculo y agregación de medios para los marcadores proporcionados, respecto a la capacidad organizativa, nos permitieron clasificar los distintos niveles de la misma. Integramos, finalmente, ambas variables (Impacto Percibido y Capacidad Organizativa) mediante tabulación cruzada y narrativa. Los resultados muestran que las enfermeras son la mayor parte de las profesionales que participan. La baja capacidad organizativa y el escaso conocimiento sobre las políticas que deben ser implementadas afectó a la participación y sus percepciones sobre el PMAQ.

Palabras-clave:
Política Pública; Política de Salud; Reembolso de Incentivo; Atención Primaria de Salud

Introduction

This paper aims to start filling a relevant gap ¹ in our knowledge concerning the implementation of the Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ). This program was adopted in 2011 to improve the quality and performance of the primary health care in Brazil, which is broadly known through its main policy: the Family Health Strategy (FHS). Both the program itself, as the analyses conducted about it until now, either overemphasize the results of performance indicators, predominantly result measures, or use those results to develop further studies ^2,3. The PMAQ is currently in its third round and its implementation process is still under-investigated. So far, little research has drawn attention to implementation issues, developing qualitative studies or policy process analysis focusing on health team actions and perspectives. Amongst these, studies by Cunha et al. ⁴ and Telles et al. ⁵ are good examples. The study by Cunha et al. ⁴ aimed at observing the perceptions of health professionals and managers of the PMAQ, revealing that the implementation of the PMAQ depends on the involvement of local managers and did not bring about changes in the performance of clinical practices. Telles et al. ⁵ verified how health professionals evaluate the results and the impact of PMAQ on health care organization, highlighting the lack of knowledge of professionals regarding the program and their discontent due to an absence of feedback regarding the results of it. In other low- and middle-income countries, these aspects are widely explored ^6,7,8 by diverse methods and types of data collection, considering contextual issues and contradictory organizational problems.

The PMAQ aims to increase access to and improve primary health care, guaranteeing comparable national, regional, and local standards of quality. Among the specific PMAQ objectives are: (i) to promote quality and innovation in primary care management, strengthening self-assessment, monitoring and assessment, institutional support, and permanent education processes in the three spheres of government; (ii) to improve the use of information systems as a primary care management tool; (iii) to institutionalize a primary care assessment and management culture in the Brazilian Unified National Health System (SUS), based on the follow-up of processes and results; (iv) to stimulate the focus of primary care on the service user, promoting management processes and transparency ⁹.

PMAQ is as a program designed to impact the quality of primary care by strengthening its evaluation and institutionalizing an assessment culture, involving and mobilizing primary health care professionals.

Theoretical studies on implementation have considered the perceptions and actions of street-level bureaucrats (or front-liners), as well as the institutional and organizational contexts or constraints of street-level bureaucracies ^10,11. This literature offers three main themes and concepts that influence the actions of front-line actors during implementation. They can be considered drivers for successful implementation and change, which has to do with the knowledge and attitudes of front-liners regarding the program/policy. Those factors are considered essential to better understand the front-liners’ adhesion. Moreover, reasons for adhering, associated with knowledge, are employed to explain the way they engage or participate (or not) in implementation amidst contextual factors such as workload and other types of pressures, and with organizational capacity of the institution where such implementation takes place. This means that front-liners’ knowledge and perceptions about programs directly affect the way they act and participate in the implementation process. Their actions and perceptions are also influenced by contextual issues/pressures related to the organizational capacity of the institution for service delivery.

This literature has been showing that greater levels of knowledge and involvement/participation of front-line workers tend to lead to an implementation with greater potential for change and improvement. Low levels of organizational capacity can negatively influence the way they participate in programs. Regarding the implementation of pay for performance programs, specifically, an absence or low levels of those policy drivers can contribute to the emergence of alternative logics, such as gaming and cheating during the implementation process ¹². From a policy perspective, we can argue that contexts with low organizational capacity and under-funded health systems, where there is a considerable amount of pressure to obtain performance targets/goals, may be more prone to generating unexpected results or alternative logics in the implementation of rational programs. In those contexts, the low performance of implementation drivers may contribute to the emergence of unexpected outcomes or alternative logics.

This article employs those policy or implementation drivers to explore the perceptions of front-line health care workers - managers, nurses, community health workers, and doctors - about the second round of PMAQ and associate those perceptions to their evaluations concerning organizational barriers to the implementation of primary health care services. The research was carried out in selected health care units in Goiânia to interview local managers and health teams, and at the Municipal Health Secretary (SMS) to interview district managers. An active search was also made to locate and interview doctors who were no longer working in the FHS during the period of the research. Our main questions were: How have front-line workers adhered to PMAQ? What are their perceived impacts of the program? What is the relation between the front-liners’ perceived impact of the PMAQ and the way they evaluate the organizational capacity of the FHS at the frontline?

The city of Goiânia, capital of Goiás State, located in the central region of the country, can be regarded as a relevant example for structural and informational/management problems that may influence the implementation of a new assessment and monitoring culture in primary health care units. According to PMAQ data, Goiânia has inadequate infrastructure for the accomplishment of FHS, where 51.2% of the health units did not have a procedure room, 34.9% did not have a pharmacy, and 25% in which community health workers do not amount to the minimum team ¹³. Nationally, those percentages correspond to 53.2%, 34.9%, and 15.3% of health care units. Also, in Goiânia, 44.1% of the team declared that the management made information available to assist with analysis and 44.8% of the team declared that they received monthly consolidated reports from the information system. Those percentages are 75% and 82.5% in Fortaleza (Ceará State), 96.9% and 85.4% in Recife (Pernambuco State), and 86.8% and 85.9% throughout Brazil ¹⁴.

Current state of knowledge about PMAQ

The literature about the PMAQ results almost entirely from the Brazilian Ministry of Health database ^2,3, originating from the assessment research undertaken by the Brazilian Ministry of Health itself, in which representatives of the public health community in Brazil were and are involved. Therefore, a wealth of research is available in Brazil, generating new knowledge about the diverse configurations and formats of Brazilian primary health care. However, a large part of this research remains unpublished. There are also thousands of micro experiences reported in the “practices community” that are not registered in a systematized or analytical way. This is a virtual community involving primary care practitioners from all over the country.

Therefore, a relevant gap in the Brazilian literature about PMAQ can be observed due to the prevalence of quantitative analyses in comparison to qualitative or process analyses of PMAQ. Saddi & Peckham ¹ have already analyzed the relevant gap on PMAQ from the perspective of global health and public policy and highlighted some implications for research and policy.

Method

We adopted a contingent mixed-method approach ¹⁵, in which qualitative data are summarized, counted and sorted, and integrated with quantitative data. We finally use a qualitative analysis to further understand those integrated results.

Semi-structured questionnaires were used to identify front-line actors’ main perceptions about the PMAQ (qualitative) and to associate these perceptions with the evaluations of the actors, concerning organizational barriers to the implementation of FHS (quantitative). Most of the semi-structured interviews occurred between March and April 2015 and during September 2015, and additional ones occurred in January and February 2016. Questionnaires were applied to 124 healthcare professionals, comprising 10 local and 12 district managers and 102 workers from health teams: 55 community health workers, 27 nurses, and 20 doctors. We employed a mixed method sampling strategy, in which purposive sampling is more dominant. The sampling process involved consecutive phases where we selected health districts (HD), health units, health teams/team members. Based on the SMS list of health districts and basic health units, and with the collaboration of SMS managers at this first step, we selected three HD, representing those where FHS is more representative in Goiânia. We selected an old district with a tradition of conducting research - due to its geographical proximity to the University Campus I - and two younger districts where research was needed, consisting of neighborhoods where the lowest-income population resides in the city. The research team proceeded with the selection process, considering the time and restriction of resources, randomly selecting 4 units in each HD, and afterward making sure PMAQ had been employed in the unity and local managers were willing to collaborate with the research. Telephone calls were made to local managers, and all of them certified their interests in collaborating with the research. For each unit, we decided to research two health teams (total number of health teams in the sample = 24) on the condition they had a doctor during the time of PMAQ. Health teams included comprised about 33% of the health teams in each of the HD selected, and about 19% of the total health teams implemented via the FHS in the city (total number of health teams in the city = 130) ¹⁶. Members of the health teams were chosen conveniently, on arrival at the health unit. Whenever the doctor who participated in the PMAQ was no longer working at the unity, we made an active search to locate him/her. Their contact was given by members of the team.

The application of questionnaires lasted between 30 and 45 min and was undertaken by the main investigators (F.C.S.) and three graduate students (A.K.C.S., H.R.A. and D.S.C.). To develop the research, the main investigator contacted the manager of the health units and scheduled a fieldwork date. Health district managers were approached after a meeting at the SMS. Participants were presented with a research information sheet and briefed on the confidentiality and anonymity of any information provided. A signed consent was obtained from each participant.

Most units involved in the study (83% of them) are small or medium, with at most two teams of health professionals per unit. Teams had on average between 2 and 8 community health workers. We interviewed between 2 and 6 community health workers/team. All the nurses of the 12 teams and some nursing technicians were interviewed. There was one doctor per team, and 15 of them were not working in the unit during the period of the PMAQ. Given this turnover of doctors in primary health care, we developed a research strategy to look for doctors who had worked in the unit during the 2^nd round of the PMAQ, to locate them in other jobs and/or cities. Doctors’ contact details were provided by members of the health team, and we have also performed internet searches. We managed to apply questionnaires to a further 15 doctors in January and February 2016, four of them living in other cities.

Semi-structured questionnaire

The questionnaire contains open questions about the reasons for adhering to the PMAQ; open-ended questions asking whether changes occurred in the working process during the PMAQ, which changes were these, and if the PMAQ improved the quality of care; as well as open questions about innovations and difficulties. They also contained closed quantitative questions where respondents evaluated (from 0 to 10) the organizational capacity of the health unit. In the analysis, as we will see, we firstly adopted separate but concurrent qualitative and quantitative analyses, followed by a descriptive statistical analysis during the integration phase.

Concurrent qualitative and quantitative analyses: constructing the variables

Categorical data coming from questionnaires was used to build the indicators level of perceived impact of the PMAQ (Perceived-Impact) and reasons for adhering to PMAQ (Reasons-Adhering), whereas numerical data was employed to determine the variable levels of organizational capacity (Organizational-Capacity).

Perceived-Impacts were classified into three levels (high, medium and low), combining two answers from the questionnaire (Table 1): (1) how the PMAQ altered work routines (open-ended), and (2) whether the PMAQ contributed to the improvement of care quality (binary open-ended). Perceived-Impact was considered high level if they answered yes for “improvement of quality” and answered with one of the following options regarding changes in the working process: (1) produced innovative changes, (2) work guided toward PMAQ goals, (3) created work overload, (4) did not alter my work routine, (5) other. Perceived-Impact was classified as medium level if they answered yes for “improvement of quality” and for one of the following questions regarding changes in the working process: (3) created work overload or (4) did not alter my work routine. Finally, Perceived-Impact was classified as low if they answered ‘no’ for improvement of quality and indicated one of the following questions, regarding changes taking place in the working process: (3) created work overload, (4) did not alter my work routine, or (5) other. The Reasons-Adhering and Perceived-Impact indicators were constructed according to local managers and to each type of health work professional.

Table 1
Levels of the Perceived-Impact of Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ) by all front-liners.

Improvement of quality?/How has the PMAQ altered the work routine?	Health workers levels of Perceived-Impact *								Total		Ranking
	High		Medium		Low		No response
	n	%	n	%	n	%	n	%	n	%
Yes
Produced innovative changes in the work	17	50							17	14	2nd
Guided work toward PMAQ goals	13	38							13	10	4th
Did not alter my work routine			9	43					9	7	7th
Other: added extra work + work done with resistance, without support/guidance, without structure			10	48					10	8	6th
No response	4	12							4	3	8th
Created an excessive workload			2	10					2	2	9th
Total/Health worker	34	27	21	17					55	44
No
Did not alter my work routine					29	53			29	23	1st
Created an excessive work load					13	24			13	10	4th
Guided work toward PMAQ goals					12	22			12	10	5th
Other: added extra work + work done with resistance, without support/guidance, without structure					1	2			1	1	10th
Total/Health worker					55	44			55	44
No response
No response					10	100	4		14	11	3rd
Total/Health worker					10	8	4		14	11
Total/Health workers	34	27	21	17	65	52	4	3	124	100

• * Results are presented for all front-liners together: manager, nurses, doctors, and community health workers. We performed the Person chi-square test and the results were p < 0.05, showing that there is a significant relationship between the variables.

Reasons-Adhering reveals the perception of the actors during the PMAQ implementation (Table 2). Reasons-Adhering was classified based on the reasons given by workers, in open questions. We conducted a content analysis on the main answers, summarizing them by type. We counted and attributed percentages to those types, which were ranked to reveal the main reasons for adhering to the PMAQ.

Table 2
Relationships between levels of Perceived-Impact and reasons for Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ) adherence by the front-liners.

Levels/Reasons for adhering PMAQ	Manager		Community health workers		Nurse		Doctor		Total		p-value *
	n	%	n	%	n	%	n	%	n	%
High
1. Order/Imposition			2	11	4	27			6	18
2. No response			6	32	0	0			6	18
3. Improved work quality/Attendance			8	42	6	40			14	41
4. Public funding			3	16	0	0			3	9
5. No response			0	0	5	33			5	15
Total high level			19	35	15	56			34	27	0.005
Low
1. Order/Imposition	6	75	18	58	8	80	16	100	48	74
2. No response	0	0	9	29	0	0	0	0	9	14
3. Improved work quality/Attendance	2	25	1	3	2	20	0	0	5	8
4. Public Funding	0	0	3	10	0	0	0	0	3	5
Total low level	8	36	31	56	10	37	16	80	65	52	0.007
Medium
1. Order/Imposition	2	20	1	20	0	0	0	0	3	14
2. No response	0	0	2	40	0	0	0	0	2	10
3. Improved work quality/Attendance	8	80	1	20	0	0	4	100	13	62
4.Work valued	0	0	1	20	0	0	0	0	1	5
5. No response	0	0	0	0	2	100	0	0	2	10
Total medium level	10	45	5	9	2	7	4	20	21	17	0.001
No response
No response	4	100							4	100
Total no response	4	18							4	3
Total
1. Order/Imposition	8	36	21	38	12	44	16	80	57	46
2. Improved work quality/Attendance	10	45	10	18	8	30	4	20	32	26
3. Public funding	0	0	6	11	0	0	0	0	6	5
4. Work valued	0	0	1	2	0	0	0	0	1	1
5. No response	4	18	17	31	7	26	0	0	11	9
Total	22	100	55	100	27	100	20	100	124	100	0.000

• * Pearson chi-square test.

In the organizational capacity section of the questionnaire, health workers evaluated the organizational capacity of their unit, giving a score from 0 to 10 for items such as availability of professionals, medicines, materials, vaccinations, infrastructure, and safety. Based on this data, we calculated the general mean attributed to the FHS organizational capacity, for these units and actors (Table 3). We also calculated the confidence interval (CI) for those results. Based on the calculation of means, we transformed or classified the general mean of Organizational-Capacity into three distinct levels: high (> 7), medium (> 5 < 7), and low (< 5), and performed a bootstrap for percent analysis in the IBM SPSS (https://www.ibm.com), that generated the CI (%) for each level and type of health worker. Descriptive statistics were performed through the SPSS.

Table 3
Evaluation of Organizational-Capacity levels of f Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ) by front-liners.

Front-liners	General aggregated mean of the notes (0-10) attributed to Organizational-Capacity			Means aggregated by levels of Organizational-Capacity
				High (mean > 7)			Medium (> 5 < 7)			Low (< 5)
	Statistics		95%CI *	n	%	95%CI **	n	%	95%CI **	n	%	95%CI **
Nurses	n	270
	Mean	5.02	4.33-5.68	3	11.1	0.0-22.2	11	40.7	22.2-59.3	13	48.1	29.6-66.7
	SD	1.75	1.26-2.10
	Variation	3.06	1.64-4.39
Community health workers	n	55
	Mean	4.63	4.23-4.98	3	5.5	0.0-10.9	19	34.5	21.8-49.1	33	60.0	47.3-72.7
	SD	1.43	1.17-1.66
	Variation	2.05	1.36-2.77
Managers	n	10
	Mean	6.79	6.12-7.48	12	54.5	31.8-72.7	10	45.5	27.3-68.2
	SD	1.16	0.69-1.48
	Variation	1.35	0.47-2.19
Doctors	n	5
	Mean	5.09	4.51-5.67				8	38.1	19.0-61.9	12	57.1	33.3-80.8
	SD	0.76	0.12-0.83
	Variation	0.57	0.01-0.68
Total		124		18			48			58

• 95%CI: 95% confidence interval; SD: standard deviation.
• * Bootstrap. Unless otherwise noted, bootstrap results are based on 1,000 bootstrap samples;
• ** Bootstrap for %. Unless otherwise noted, bootstrap results are based on 1,000 bootstrap samples.

We also developed a content analysis of the two open questions regarding information about innovations and difficulties related to the PMAQ. The answers were summarized or synthesized by themes underlined by respondents and associated with implementation drivers.

Final data integration: hypothesis and analysis

We finally integrated the variables Perceived-Impact and Reasons-Adhering (Table 2), and the variables Perceived-Impact and the Organizational-Capacity (Table 4) through two cross-tabulations/descriptive statistics and narrative, to explore the relations between them by type of professional and through all front-liners. In the tables, we compared front-liners with different types of Reasons-Adhering (Table 2) and distinct Organizational-Capacity (Table 4), both associated with the Perceived-Impact variable. Firstly, we tested the hypothesis (Table 1) that associations between Reasons-Adhering are negatively associated to Perceived-Impact, due to a top-down type of adhesion, characterized by lack of knowledge and low participation, which negatively influenced the front-liners’ perceived impact of the program. We also test the hypothesis (Table 4) that high levels of organizational capacity positively influence the front-liners’ perceived impact on the program, while low organizational capacity negatively influences their perceived impact. Therefore, in agreement with the literature on implementation, our general hypothesis is that both the Reasons-Adhering and Organizational-Capacity variables influenced the Perceived-Impact on the frontline. In each one of the two cross-tabulations, we calculated the Pearson chi-square test to obtain the p-value, which would be the probability of encountering a significant relationship between the variables.

Table 4
Relationships between levels of Organizational-Capacity and levels of Perceived-Impact of Brazilian National Program for Improving Access and Quality to Primary Care (PMAQ) by front-liners.

Levels of Organizational-Capacity/Front-liners	Levels of Perceived-Impact										p-value *
	High		Low		Medium		No response		Total
	n	%	n	%	n	%	n	%	n	%
Low (< 5)
Community health workers	13	65	16	55	4	44			33	57
Nurses	7	35	5	17	1	11			13	22
Doctors	0	0	8	28	4	44			12	21
Total low level	20	59	29	45	9	43		0	58	47	0.040
High (> 7)
Community health workers	2	50	1	100	0	0	0	0	3	17
Nurses	2	50	0	0	1	9	0	0	3	17
Doctors	0	0	0	0	10	91	2	100	12	67
Total high level	4	12	1	2	11	52	2	50	18	15	0.009
Medium (> 5 < 7)
Community health workers	4	40	14	40	1	100	0	0	19	40
Nurses	6	60	5	14	0	0	0	0	11	23
Managers	0	0	8	23	0	0	2	100	10	21
Doctors	0	0	8	23	0	0	0	0	8	17
Total medium level	10	29	35	54	1	5	2	50	48	39	0.013
Total
Community health workers	19	56	31	48	5	24	0	0	55	44
Nurses	15	44	10	15	2	10	0	0	27	22
Managers	0	0	8	12	10	48	4	100	22	18
Doctors	0	0	16	25	4	19	0	0	20	16
Total	34	100	65	100	21	100	4	100	124	100	0.000

• * Pearson chi-square test.

The existing connections between the results of these two cross-tabulations are further explored and interrelated via narrative, considering the field-work diaries and perceptions regarding difficulties and innovations related to the PMAQ implementation.

This research was approved by the Federal University of Goiás Research Ethics Committee/National Research Ethics Committee (CONEP), registration n. CAAE 26584514.3.0000.5083 of May 2014.

Results and discussion

Characteristics of respondents

Among the 22 managers who participated in the research, 12 of them were health district managers and 10 were local managers at the units. 50% of local managers (n = 5) worked in the units from 8 months to one year, amongst whom a considerable part (60%, n = 3) had a high degree of understanding of the PMAQ since they occupied previous positions in HD or the SMS. As for the nurses, 51.9% had between 11 and 20 years of experience with the FHS. All of them were in FHS at the time of the PMAQ and acted directly on its processes of management discussion, data collection, and registration, playing mainly a leadership role in this process in the health unit. In terms of community health workers, the mean working experience in FHS was 11 years, and 25% consulted the nurses to answer the question about the improvement of quality due to the PMAQ. Nurses had more knowledge and control over the process, while community health workers generally had to perform other tasks concurrently, such as visits, work in triage and in the reception of the units, and with other/new demands.

Regarding the doctors, the FHS is characterized by a significant turnover of doctors. Firstly, we managed to apply questionnaires to 5 doctors in 2015, and with the adoption of an active search strategy, we managed to interview doctors who were no longer working at the FHS during the research period but were there during the time of PMAQ.

Levels of the Perceived-Impact of the PMAQ

Regarding the actors’ Perceived-Impact of the PMAQ, a cross-tabulation (Table 1) shows that almost half (52%) (n = 65) of the front-line health workers from the sample hold a low Perceived-Impact, while 27% hold a high Perceived-Impact (Table 1).

If we analyze the results by types of front-liners, they show that 56% of the nurses (n = 15) have a high Perceived-Impact. What also draws our attention is that 56% of community health workers (n = 31) and 80% (n = 16) of the doctors hold a low Perceived-Impact. A more balanced perception can be observed amongst managers, with 36% (n = 8) with low Perceived-Impact and 45% (n = 10) with medium Perceived-Impact.

It is interesting to observe that nurses’ and community health workers’ Perceived-Impact are distributed between the high and low levels, while those of local managers’ sit between medium and no response. The most significant level for nurses was high, and the most significant level for community health workers was low. This explains the community health workers’ more critical perception and/or greater resistance to the PMAQ when compared to nurses. Nurses assume a managerial or coordinating role in the team, and the same occurs with the PMAQ. Therefore, being more involved with and knowledgeable about the PMAQ explains the nurses’ higher Perceived-Impact in comparison with community health workers, and the proximity of a health district and some local managers regarding the SMS management explains the medium Perceived-Impact among managers, as they are in an intermediary position between the SMS and the frontline.

Perceived-Impacts and reasons for adhering to PMAQ

We have also made a cross-tabulation between Perceived-Impact-PMAQ and Reasons-Adhering-PMAQ (Table 2). When observing reasons for adhering to PMAQ, for 46% (n = 57) of health workers this was due to the PMAQ being imposed by the SMS. Only 26% (n = 32) said that they had adhered to try to improve service quality. This result suggests that a top-down, non-participative culture of adhesion may have been prevalent in the initial PMAQ implementation process. According to the implementation literature, lack of adherence and knowledge does not favor engagement or participation of front-line workers in the implementation process.

The Table 2 also reveals that there is a significant (p = 0.007) relation between low Perceived-Impact and reasons for adhering to the PMAQ (total n = 65, 52%). This relation is significant for all types of front-liners, and more expressive in terms of the reason “imposition from policy makers/managers” (total n = 74, 48%). Additionally, in Table 1, we observe that low Perceived-Impact is also associated with work overload or does not lead to changes in the working process or work focus on the program goals.

Table 2 also shows that there is a significant relation (p = 0.005) between high Perceived-Impact and reasons for adhering to the PMAQ (total n = 34, 27%). This was the most expressive relation for nurses and community health workers and regarding the reason “improvement of quality” (n = 14, 41%). Additionally, in Table 1 we observe that for all categories of front-liners (n = 17, 14%) the fact that high Perceived-Impact is associated with “produced innovative changes in the working process” was significant (p < 0.05).

This means that, at the front line, high Perceived-Impact is strongly associated with what could be considered as a high level of policy knowledge and understanding regarding the program, as well as a more effective quality improvement. Low Perceived-Impact, on the other hand, was highly associated with a top-down type of adherence, low level of knowledge, and prevalence of challenges regarding quality improvement during the work process.

The relation between medium Perceived-Impact and adherence was also significant, and more expressive for managers (45%, n = 10) and regarding the reason “improvement of quality” (Table 2). We also observed that medium Perceived-Impact (Table 1) is associated with “did not alter my work routine” or “work done with resistance and without support.” Though medium Perceived-Impact is associated with the Reasons-Adhering, they are also interconnected with barriers to implementing quality improvement.

Adherence to the PMAQ, except for managers and nurses, was deprived of knowledge about the PMAQ, and of its valorization at the frontline. In fact, the PMAQ was perceived as a top-down policy, with a partial participation of front-line professionals, and was more focused on the participation of nurses. The field diary showed that in only two units was there a group participation by all the professionals of the team in the discussions and planning of the PMAQ, with the participation of doctors being less evident.

Associations between Perceived-Impact and levels of Organizational-Capacity

Cross-tabulation made between front liners Perceived-Impact-PMAQ and their evaluation regarding the organizational capacity of the unit presents some additional information regarding challenges involved in the implementation process of PMAQ. Managers’ evaluations regarding Organizational-Capacity were distributed between high and medium levels (Table 3). Community health workers, doctors, and nurses concentrate their evaluation at the low Organizational-Capacity.

There is a significant relationship between low Organizational-Capacity and Perceived-Impact-PMAQ 47% (n = 58) (p = 0.040) (Table 4). This relationship is more expressive for low and medium levels (45%, n = 29; 43%, n = 9) of Perceived-Impact, for community health workers, nurses, and doctors, respectively. Additionally, we observed that there is a significant association between medium Organizational-Capacity and low Perceived-Impact (p = 0.013) (Table 4), with 54% (n = 35) of the front-liners with low Perceived-Impact attributing scores lower than five for organizational capacity, for all the front-liners involved. The relationship between high Organizational-Capacity and medium Perceived-Impact is relevant in the case of managers (n = 12, 67%).

Therefore, low Organizational-Capacity influenced the Perceived-Impact of the doctors, nurses, and community health workers. Additionally, the Organizational-Capacity, together with the type of adhesion on the frontline, influenced how the front-liners effectively participated in the implementation of the PMAQ and perceived its effects on primary health care.

Such interconnections are confirmed by the responses given to the two open questions regarding PMAQ-related innovations and difficulties/challenges. Regarding innovation, the actors underlined the mapping of the area and consequent organization of the work; however, this statement was accompanied by observations such as “we did not follow through with the evaluation”. It was also reported that the discussion was good on the PMAQ, but there was no continuity. Managers and some nurses already highlighted adequacy in the evaluation system of the charts. Additionally, it is worth pointing out that nearly half the interviewees did not respond to the question regarding innovation generated by the PMAQ, and that, when reported, this was done principally by managers and nurses, and this as a general report is related to the question of a lack of continuity.

In terms of difficulties for the implementation, the themes about lack of support or guidance for the work of the PMAQ, a lack of dialogue/feedback and structure to realize the work, and the resistance of professionals were recurrent. Some of the respondents further underlined that they had to do some cleaning, arrangements, or make-up in the unity for the realization of the PMAQ collection. Even the actors who reported innovation themes regarding the PMAQ, also made these observations concerning difficulties or alternative logics prevalent during the implementation. Therefore, themes regarding the lack of knowledge and low levels of participation were also recurrent and associated with the lack of organizational capacity. In a contradictory but meaningful way, they were associated with some innovations by half of the front-liners.

Front-line actors said they did not directly receive the PMAQ resources. It was also noted that half of them did not expect that financial incentives would be passed on to the team. The international literature shows that, in a context of low organizational capacity - where front-liners must answer to competing demands, and hold various motives to participate in the work process -, the influence of financial incentives is partial or requires adaptations/changes to take place. Such knowledge allows us to argue that the transference of the financial award would not necessarily have highly affected the actors' perceptions about the PMAQ, giving the overload of work and other challenges they face in the work process in the FHS. Further research is needed on the subject.

Discussion with the PMAQ literature

Few studies about the PMAQ present relevant policy analyses and results about the PMAQ’s implementation at the frontline. They show that the adhesion to PMAQ at the frontline indeed follows a top-down pattern. Managers and front-liners do not effectively know the program ¹⁷, as they cannot talk clearly about its phases of development ⁵. Front-liners’ knowledge about the program is dependent on how they relate to managers or on the strategies adopted by managers in the delivery of the program ⁴.

Qualitative studies developed in health units in the states of São Paulo and Rio Grande do Sul reveal that the PMAQ did not bring about improvements in clinical practices or in the work process. Front-liners’ perceptions about the PMAQ were also associated with lack of feedback and difficulties in accessing PMAQ’s data ^4,5. Commonly, those studies highlight that implementation strongly depends, among other factors, on the attitudes of managers during the process ^4,5.

However, two studies conducted in small municipalities in the Rio Grande do Norte State develop a comparatively more positive analysis about the program ^18,19. Authors assert that the adoption of the program revealed a momentary motivation among front-liners, being the program afterwards characterized by low levels of participation. Even though, the PMAQ did start to bring some changes in the work process. In those two small municipalities, the authors understand that front-liners held a good knowledge about the PMAQ, due to attitudes of the managers during the process ^18,19.

A systematic review ²⁰ shows that some difficulty was present during the implementation of the PMAQ, such as: lack of physical structure, the necessity to better qualify the work process of health teams, and low levels of management. They state that PMAQ did not manage to re-orient the work process due to difficulties found during the implementation. Those difficulties enable us to understand why/how the front-liners’ perception about the PMAQ tend to vary but remain in a middle or low level by those who effectively worked in the implementation of the program.

All those studies, developed in different units in diverse parts of the country, together with the results presented for Goiânia in this paper, bring some important and broader policy challenges/questions about the PMAQ - related to how a top-down type of public policy can become a practice. The integration of front-line workers in the work process, as well as their relationship with managers, seems to be a relevant aspect of the process. Increases in feedback and use of data regarding the PMAQ could also contribute to increase knowledge and participation on it. The strengthening of PMAQ’s implementation is, therefore, strongly associated with how the program is developed in complex scenarios, in which health workers must cope with several demands. Thus, investments in the organizational capacity of the unit would foster and facilitate workers’ participation in the PMAQ.

Those pathways, which will contribute to enhance front-liners’ knowledge, participation, and perceived impact about the PMAQ, need to be further explored/discovered by other policy/system analyses, as well as followed by both managers and front-liners, together. It would be important to consider the differences in those pathways/routes between municipalities with different levels of income/population and patterns of management to discover the relations/associations between relevant policy variables and types of actors.

Limitations

The small sample size, significantly comprised of more nurses and community health workers than managers and doctors, is acknowledged as a study limitation, specifically in terms of descriptive statistics. However, the sample represented all members of the teams that were involved in the implementation of the PMAQ and reveals the reality of the FHS. Further studies, based on qualitative and quantitative data, need to be developed, seeking other factors that could be further conceptually inter-related.

Conclusion

Local managers and nurses (chiefly) are the main professionals participating in PMAQ, followed by community health workers. Most doctors are either not involved or primarily tend to take part only in initial discussions. The fact that front-liners do not hold a high level of knowledge regarding the PMAQ might also contribute to generating ambiguities/alternative logics regarding the PMAQ during its implementation ¹⁰. Moreover, the challenging organizational context of the primary health care (with a lacking/fragile culture of evaluation and excessive workloads) affects the health worker involvement and their perceptions of PMAQ.

A program aimed at promoting continuity and change in the evaluation culture would have a greater probability of achieving results (or improving its results) if the participation of its actors is stimulated during the policy process: generating better knowledge, understanding, and valorization of the program at the frontline.

In the short term, interactive or dialogic strategies could be adopted to better engage and include more actors in the process. Therefore, the improvement of the PMAQ implementation would entail the qualitative improvement of PMAQ results. This would enhance the levels of Perceived-Impacts, as well as the credibility/legitimacy of PMAQ at the frontline, facilitating the work related to the PMAQ implementation and external evaluation.

Moreover, this study reveals the need to deepen the comparative knowledge between municipalities or regions and types of units regarding who actually participates in the PMAQ and how this participation really takes place. This would mean verifying: (1) if and to what extent the nurse is indeed the main actor involved with PMAQ, (2) how the other members of the health team (and doctors especially) participate, (3) what is truly involved in the PMAQ-related working process at the unit, (4) what are the alternative logics that might affect this process and (5) what is the general open view and evaluation of front-liners regarding the program.

Therefore, more contextualized public policy or health policy research, focusing on front-line workers, could be implemented and used to better explore other variables and strengthen the adoption of innovative strategies that could potentially improve the PMAQ and FHS at the same time.

Acknowledgments

The authors acknowledge the contributions from the Municipal Health Secretary (SMS) of Goiânia for discussions and access given to health units; the Department of Basic Health Care at the Brazilian Ministry of Health (DAB/MS) and Dr. Marta R. Souza (Federal University of Goiás - UFG), Dr. Silvis Barros (São Paulo Health Institute) and Marcelo Pedra (DAB/MS) for observations made during seminars and/or on the first draft of this work. We thank the two blind reviewers for their comments. This article presents the results of Fabiana C. Saddi’s post-doctoral PNPD-Capes Research Fellowship. Dr. Saddi’s fellowship and part of research costs were funded by Graduate Studies Coordinating Board (Capes), Brazilian Ministry of Education. Project approved by UFG’s Research Ethics Committee (Registration number: 26584514.3.0000.5083). One of the graduate students (Wellida Pereira) involved in the research received scholarships from UFG’s Office of the Dean of Extension and Research, two others were attached to UFG’s Scientific Initiation Program as volunteers: Douglas S. da Costa and Ana Karoline C. dos Santos. English correction costs were covered by the Graduation Program in Political Science/UFG and Capes/Brazilian Ministry of Education.

References

• ¹
  Saddi FC, Peckham S. Brazilian payment for performance (PMAQ) seen from a global health and public policy perspective - what does it mean for research and policy? J Ambul Care Manage 2018; 41:25-33.
• ²
  Macinko J, Harris MJ, Rocha MG. Introduction to the Special Edition on the Brazilian National Program to Improve Primary Care Access and Quality (PMAQ). J Ambul Care Manage 2017; 40(2 Suppl):S1-3.
• ³
  Avaliação da atenção básica à saúde no Brasil. Saúde Debate 2014; 38(n. Especial):5-372.
• ⁴
  Cunha GT, Castro CP, Oliveira MM, Santos CSR, Rosa IF, Rodrigues MC. Programa Nacional de Melhoria ao Acesso e a Qualidade da Atenção Básica: um estudo qualitativo. In: 2º Congresso Brasileiro de Política, Planejamento e Gestão em Saúde. http://www.politicaemsaude.com.br/anais/trabalhos/publicacoes/235.pdf (accessed on Oct/2017).
  » http://www.politicaemsaude.com.br/anais/trabalhos/publicacoes/235.pdf
• ⁵
  Telles R, Weigelt LD, Rezende MS, Krug SBF, Ferreira R, Kessler AL, et al. Implementação do Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ-AB). Rev Epidemiol Controle Infecç 2016; 6:1-10.
• ⁶
  Ssengooba F, McPake B, Palmer N. Why performance-based contracting failed in Uganda: an "open-box" evaluation of a complex health system intervention. Soc Sci Med 2012; 75:377-83.
• ⁷
  Olafsdottir AE, Mayumana I, Mashasi I, Njau I, Mamdani M, Patouillard E, et al. Pay for performance: an analysis of the context of implementation in a pilot project in Tanzania. BMC Health Services Research 2014; 14:392.
• ⁸
  Lee JY, Lee S-I, Jo M-W. Lessons from healthcare providers' attitudes toward pay-for-performance: what should purchasers consider in designing and implementing a successful program? J Prev Med Public Health 2012; 45:137-47.
• ⁹
  Departamento de Atenção Básica, Secretaria de Atenção à Saúde, Ministério da Saúde. Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): manual instrutivo. Brasília: Ministério da Saúde; 2012.
• ¹⁰
  Meyers MK, Vorsanger S. Street-level bureaucrats and the implementation of public policy. In: Peters BG, Pierre J, editors. The Sage Handbook of public administration. London: Sage Publication; 2003. p. 245-56.
• ¹¹
  May P, Winter SC. Politicians, managers, and street-level bureaucrats: influences on policy implementation. J Public Adm Res Theory 2009; 19:453-76.
• ¹²
  Pollitt C. The logics of performance management. Evaluation 2013; 19:346-63.
• ¹³
  Departamento de Atenção Básica, Secretaria de Atenção à Saúde, Ministério da Saúde. Retratos da atenção básica: características das unidades básicas de Saúde. v. 1. Brasília: Ministério da Saúde; 2015.
• ¹⁴
  Departamento de Atenção Básica, Secretaria de Atenção à Saúde, Ministério da Saúde. Retratos da atenção básica: gestão do processo do trabalho. v. 1. Brasília: Ministério da Saude; 2015.
• ¹⁵
  Fetters MD, Curry LA, Creswell JW. Achieving Integration in mixed methods designs: principles and practices. Health Serv Res 2013; 48(6 Pt 2):2134-56.
• ¹⁶
  Departamento de Atenção Básica, Secretaria de Atenção à Saúde, Ministério da Saúde. Informações sobre as ações e programas do Departamento de Atenção Básica do Município de Goiânia (GO). Brasília: Ministério da Saúde; 2017.
• ¹⁷
  Sampaio J, Moraes MN, Marcolino EC, Castro ID, Gomes LB, Clementino FS. PMAQ-AB: a experiência local para a qualificação do programa nacional. Rev Enferm UFPE on line 2016; 10:4318-28.
• ¹⁸
  Feitosa RMM, Paulino AA, Lima Júnior JOS, Oliveira KKD, Freitas RJM, Silva WF. Mudanças ofertadas pelo Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica. Saúde Soc 2016; 25:821-9.
• ¹⁹
  Queiroz IPD, Valença CN, Oliveira RFS, Marinho CRS, Azevedo ACB, Azevedo BSBM, et al. National Program for Improving Access and Quality of Primary Care: implications of its implementation in the Brazilian Northeast. Int Arch Med 2017; 10:10.3823/2461.
• ²⁰
  Sossai TN, Valença CN, Oliveira RFS, Marinho CSR, Azevedo BSBM, Silva JA, et al. Evidências sobre o Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica. Rev Bras Pesqui Saúde 2016; 18:111-9.

Publication Dates

• Publication in this collection
  22 Oct 2018
• Date of issue
  2018

History

• Received
  20 Nov 2017
• Reviewed
  19 Mar 2018
• Accepted
  25 May 2018