Evaluation of primary child health care in Brazil: a systematic review of methods and results

Abstract

This systematic review analyzes the methods and instruments employed to evaluate primary child health care in Brazil and their main findings. The review was conducted in accordance with the recommendations of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyzes) statement. Searches of articles focusing on children aged between zero and five years published between 1994-2006 were conducted of the following databases: MEDLINE, LILACS, IBECS, BDENF, PubMed, PsycNET, Cochrane, and CINAHL. The searches yielded 3,004 articles. After initial screening and the application of the STROBE and SRQR criteria, 21 articles were included in the review. About 52% of the articles were conducted in the Southeast region and 95.2% were published as of 2010. The most commonly used evaluation tool was the Primary Care Assessment Tool Child Edition, adapted and validated for use in Brazil (52.4%). The quality of primary child care was inadequate. The main limitations included poor access to services, inadequate facilities, and underqualified health staff. There has been a significant increase in the number of evaluation studies conducted in Brazil in recent years. Despite advances in health care across the country, the findings point to the need for a more effective response to the challenges in ensuring comprehensive primary child care in Brazil.

Key words
Primary Health Care; Child Health; Evaluation; Systematic Review

Resumo

Essa revisão sistemática objetivou analisar os métodos e instrumentos, bem como os principais resultados, das avaliações de qualidade da assistência à saúde da criança na APS no Brasil. Estudo realizado de acordo com as recomendações do Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), em consulta às bases de dados MEDLINE, LILACS, IBECS e BDENF, PUBMED, PsycNET, Cochrane e CINAHL (1994-2016), com foco em crianças de 0 a 5 anos. Foram identificados 3.004 artigos. Após a triagem inicial e a aplicação dos critérios STROBE e SRQR, 21 artigos foram incluídos na revisão. Cerca de 52% dos artigos foram realizados na região sudeste e 95,2% publicados a partir de 2010. O principal instrumento de avaliação utilizado foi o Primary Care Assessment Tool (52,4%). A qualidade da assistência a criança mostrou-se deficitária, com limitações no acesso aos serviços, carência de infraestrutura e baixa qualificação de profissionais. Houve aumento significativo dos estudos avaliativos nos últimos anos no Brasil. Apesar dos avanços na assistência à saúde no país, os limitados índices de qualidade apontam a necessidade de superação de desafios para garantia da atenção integral à saúde da criança.

Palavras-chave
Atenção Primária à Saúde; Saúde da criança; Avaliação; Revisão Sistemática

Introduction

Brazil has made significant strides in improving child health in recent decades as a result of improved living conditions, the advancement of children’s rights, and advances in the country’s health policy^1,2. This is illustrated by a sharp fall in infant mortality rates^1,3, from 85.6 per 1,000 live births in 1980 to 13.8 in 2015⁴, when Brazil achieved target 4 of the Millennium Development Goals (Reduce by two-thirds, between 1990 and 2015, the under-five mortality)³.

Brazil began to implement specific interventions targeting child health in the 1960s with the Maternal and Child Health Program. In the 1990s, initiatives began to incorporate policies designed to expand the coverage of public health services, as in the case of the family health and community health agent programs. With the restructuring of primary health care services, it was possible to improve the distribution of health care professionals across underserved areas and widen access to health services, positively impacting child health indicators². The creation of the National Policy for Comprehensive Child Health Care (PNAISC, acronym in Portuguese), in 2015, was the culmination of interinstitutional efforts to enhance actions directed at young children and vulnerable groups, based on the principles of the right to life, equality, comprehensive humanized care, and participatory management².

Brazil’s Family Health Strategy (FHS) has consolidated primary health care, expanding service coverage across the country. However, numerous factors continue to limit coverage, including the coexistence of traditional and family care models in the same service, structural and organizational factors, and staff shortages, driving the institutionalization of evaluation processes to inform strategies to strengthen primary health care^5-7.

Evaluation is a process in which, based on valid and legitimate information, value judgments are made about interventions, processes, and outcomes with a view to enhancing service management performance, facilitating change, decision-making, and the pursuit of quality⁷. The growing recognition of the importance of evaluation has led to the development of a suite of instruments and methods for evaluating the adequacy of public services in Brazil over the last three decades⁸. This process has been led by the Ministry of Health, with initiatives like the “Sala de Situação” or “Health Situation Room” (2002), the Assessment for Improving the Quality of the Family Health Strategy - AIQ (Avaliação para a Melhoria da Qualidade - AMQ)⁹, the Primary Care Assessment Tool¹⁰ (2010), and the National Program for Improving Access and Quality in Primary Care (2011)⁹. In addition, research groups at universities across the country have also developed a number of initiatives⁸.

In view of the need to synthesize the knowledge accumulated in recent years, this review analyzes the methods and instruments employed to evaluate primary child health care in Brazil and their main findings.

Methods

We conducted a systematic review based on the recommendations set out in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement¹¹. Searches were conducted of the databases Virtual Health Library, Brazil (MEDLINE, LILACS, IBECS, and BDENF), PubMed, PsycNET, Cochrane, and CINAHL, using the following search terms: “primary health care OR family health strategy AND evaluation AND child health OR child”, based on the DeCS (Health Sciences Descriptors) and MeSH (Medical Subject Headings).

We included studies evaluating primary health care services for children aged 0 to 19 years (focusing on the zero to five-years age group) published in English, Spanish and Portuguese between January 1994 (the year in which the FHS was created) and September 2016. Literature reviews, theses, dissertations, editorials, case study reports, studies with data on non-Brazilian children, and studies that did not meet 80% of the required items of the methodological quality scales used by this study were excluded.

The methodological quality of observational studies was assessed in accordance with the STROBE¹² (Strengthening the reporting of observational studies in epidemiology) statement, consisting of a checklist of 22 items that should be addressed in articles. The version translated and validated for use in Brazil in 2008¹³ establishes article quality categories. Articles that meet at least 80% of the items are classified as category “A”^12,13. Qualitative studies were assessed using the Standards for Reporting Qualitative Research (SRQR)¹⁴. Studies that used both quantitative and qualitative methodologies were assessed using both instruments and included if they met at least 80% or higher of the items of one of the instruments. One independent reviewer assessed methodological quality and two reviewers evaluated thematic eligibility.

Results

The searches yielded 3,004 articles, 538 of which were excluded because they were duplicates. A total of 2,333 articles were excluded after the analysis of the titles and abstracts because they failed to meet the study eligibility criteria. The full texts of the remaining citations were then examined, resulting in the exclusion of 99 articles because they failed to meet the study inclusion criteria. The remaining articles were then assessed against the STROBE and SRQR checklists. A total of 13 publications were excluded because they failed to meet at least 80% of the items, resulting in a final sample of 21 articles. The article selection process is shown in Figure 1.

Figure 1
Flow diagram of the article selections process, Belo Horizonte, Minas Gerais, Brazil, 2017.

*Virtual Health Library, Brazil (MEDLINE, LILACS, IBECS, and BDENF), PubMed, PsycNET, Cochrane, and CINAHL.

Table 1 presents the general characteristics of the selected studies. Around 95% of the articles where published after 2009. All the studies used quantitative methodologies and two used a combination of quantitative and qualitative methodologies. We did not find any qualitative studies that met the inclusion criteria. The most commonly used evaluation instrument was the Primary Care Assessment Tool Child Edition (PCAT-CE) adapted and validated for use in Brazil (52.4%). The other studies used instruments developed by the researchers, except for a study conducted in Maceió in 2003¹⁵, which used the AIQ.

Chart 1
Characteristics of the studies included in the systematic review. Belo Horizonte, Minas Gerais, Brazil, 2017.

Author (year)	Journal	Study location	Methodological approach	Evaluation instrument	Sample/Age group	Objective
Silva et al. (2016)18	Rev Bras Enferm	Santa Maria (RGS)	Cross-sectional study	PCAT-CE	71 children and adolescents aged 0 to 19 years living with HIV	To compare the quality health care provided to children and adolescents living with HIV across types of services based on the experiences of family members/guardians
Harzheim et al. (2016)42	Ciên Saúde Colet	Rio de Janeiro (RJ)	Cross-sectional study	PCAT-CE	3,145 children aged 0 to 12 years	To evaluate the limits and possibilities of the advances made in primary health care in Rio de Janeiro, based on the experiences of both adult and child service users
Silva e Fracolli (2016)37	Rev Bras Enferm	Alfenas micro health region (MG)	Quantitative, cross-sectional evaluation study	PCAT-CE	330 parents/guardians of children aged 0 to 2 years.	To evaluate care services for children aged under two years provided under the Family Health Strategy
Oliveira e Verissimo (2015)21	Rev Esc Enferm USP	Colombo (PR)	Cross-sectional study with quantitative design	PCAT-CE	482 parents/guardians of children aged up to one year	To compare the extent to which primary child care attributes are operationalized across local FHS care centers and traditional health centers
Araújo et al. (2014)38	Acta Paul Enferm	A municipality in the State of Paraná (PR)	Descriptive, cross-sectional evaluation study	PCAT-CE	548 children aged under12 years	To identify the extent to which the family centeredness and community orientation attributes are operationalized in primary child care services
Ferrer et al. (2014)20	Health Policy Plan	São Paulo (SP)	Descriptive, cross-sectional analytical study	PCAT-CE	501 children aged 0 to 14 years hospitalized for ambulatory care sensitive conditions	To compare the longitudinality of care in two care models from the perspective of service users, correlating the findings with the utilization of FHS services
Mesquita Filho et al. (2014)39	Ciên Saúde Colet	Pouso Alegre (MG)	Observational, analytical cross-sectional study	PCAT-CE	419 children aged 0 to 24 months registered in the local primary health care service	To evaluate attributes of primary care for children aged 0 to 2 years from the perspective of parents and guardians and identify possible influencing factors
Marques et al. (2014)19	Ciên Saúde Colet	São Francisco (MG)	Descriptive, cross-sectional analytical study	PCAT-CE	76 parents/guardians of children aged 0 to 5 years registered in the FHS	To evaluate the attributes of primary child care from the perspective of a quilombola community in the north of the State of Minas Gerais
Perez et al. (2014)16	Rev Saúde Pública	Vespasiano (MG)	Cross-sectional study	Questionnaire elaborated by the researchers	77 primary care professionals (doctors, nurses, nursing technicians, and community health agents) and 293 children aged under 5 years	To analyze the strengths and weaknesses of the FHS from the perspective of health care professionals and service users
Rocha e Pedroza (2013)26	Texto Contexto Enferm	Queimadas (PB)	Cross-sectional study	Questionnaire elaborated by the researchers	204 children born in 2009 living in Queimadas	To evaluate developmental follow-up in the context of primary health care in Queimadas
Modes e Gaíva (2013)43	Online Braz J Nurs	Cuiabá (MT)	Descriptive evaluation study	Questionnaire elaborated by the researchers	12 primary health care doctors and 14 nurses	To evaluate the structure of primary care centers providing services for children under one year of age from the perspective of doctors and nurses
Modes e Gaíva (2013)40	Esc Anna Nery	Cuiabá (MT)	Descriptive evaluation study	Questionnaire elaborated by the researchers	127 mothers/guardians of children aged under one year	To evaluate the satisfaction of mothers/guardians of children aged under one year regarding care delivery at primary care centers in Cuiabá
Sales et al. (2013)15	J Hum Growth Dev	Maceió (AL)	Descriptive cross-sectional study	Evaluation for Quality Improvement - AIQ (AIQ)	All doctors (66) and nurses (66) working in the FHS	To investigate the extent to which child health care actions are developed in the FHS
Machado et al. (2012)24	Ciên Saúde Colet	Fortaleza (CE)	Cross-sectional population-based study	Questionnaire elaborated by the researchers	350 families of children aged under 5 years using public health services	To describe the dimensions of access to care and factors associated with the satisfaction of mothers of children aged under five years
Leão et al. (2011)22	Rev Bras Saúde Mater Infant	Montes Claros (MG)	Cross-sectional study	PCAT-CE	350 parents/guardians of children aged 0 to 2 years.	To compare primary child care attributes between the FHS and other child care services in the municipality based on the evaluation of parents/guardians
Luhm et al. (2011)25	Rev Saúde Publica	Curitiba (PR)	Descriptive study	Information from the municipal immunization information system	2,637 children aged 12 to 24 months born in 2002 and living in Curitiba	To evaluate an immunization program for children aged between 12 and 24 months in Curitiba, based on information from the municipal immunization information system
Leão e Caldeira (2011)44	Ciên Saúde Colet	Montes Claros (MG)	Cross-sectional study	PCAT-CE	350 parents/guardians of children aged under 2 years	To determine the association between primary care attributes and the qualification of FHS doctors and nurses
Costa et al. (2011)17	Ciên Saúde Colet	Teixeiras (MG)	Cross-sectional study with quantitative and qualitative design	Questionnaire elaborated by the researchers	161 mothers of children aged under 2 years and 35 health workers from 4 primary care centers	To evaluate child health care under the Family Health Program
Ribeiro et al. (2010)41	Cad Saúde Pública	Diamantina (MG)	Descriptive cross-sectional study	PCAT-CE	384 parents/guardians of children aged 0 to 6 years.	To evaluate receptiveness in the FHS as an approach to receiving, and listening to patients and bringing together those who care and those who are cared for, from the perspective of parents and guardians.
Caldeira et al. (2010)23	Ciên Saúde Colet	Montes Claros (MG)	Household survey	Questionnaire elaborated by the researchers	595 mothers of children aged under 2 years	To evaluate the quality of maternal and child care under the FHS and conduct a comparative analysis of traditional and FHS care centers.
Figueiras et al. (2003)27	Cad Saúde Pública	Belém (PA)	Descriptive cross-sectional study	Questionnaire elaborated by the researchers	80 FHS doctors and 80 FHS nurses	To evaluate the knowledge and practice of child developmental surveillance by primary care professionals in Belém

• PCAT-CE: Primary Care Assessment Tool Child Edition, adapted and validated for use in Brazil; FHS: Family Health Strategy.

Only two studies^16,17 included both health care professionals and service users. In 85.7% of the studies, only parents and/or guardians evaluated services, in the majority of cases the children’s mothers. In the other studies, doctors and nurses evaluated services. The studies evaluated children’s and young people’s services, with 61.9% of studies focusing specifically on the zero to five-year age group. Some studies focused on specific groups, such as children living with HIV¹⁸, quilombolas¹⁹, and children hospitalized for ambulatory care sensitive conditions²⁰.

The studies were conducted in areas with varying population characteristics, ranging from state capitals to rural communities. Eleven of the 21 studies were conducted in the Southeast region, nine of which in the State of Minas Gerais (Figure 2).

Figure 2
Distribution of studies by region, Belo Horizonte, Minas Gerais, Brazil, 2017.

Table 2 shows a synthesis of the methods and main conclusions of the selected articles. All the studies assessed the performance of family health teams, 52.4% investigated traditional health centers or a mixture of traditional and FHS centers, and 14.3% addressed other types of models. Almost all of the studies investigated service structure and work processes. The main aspects of care investigated were: primary care attributes (47.7%); specific actions developed by primary health care services, such as immunization programs, acolhimento or “receptiveness”, and child health surveillance (33.3%); care structure (9.5%); and health care professional training and development (9.5%).

Chart 2
Results of the analysis of the articles included in the review, Belo Horizonte, Minas Gerais, Brazil, 2017.

Author (year)	Type of service evaluated	Item evaluated	Service quality indicators/Conclusions
Silva et al. (2016)18	Primary health care services (FHS and traditional care centers) and specialist outpatient services for children and adolescents living with HIV	Primary care attributes*	The attributes scored higher in the specialist service (6.4)** than in the primary health care services (6.1)**. Neither of the services achieved the minimum quality cut-off score (6.6)*** for "general items". The lowest-scoring attributes were "family centeredness" and "community orientation". All other attributes, except "services provided" (6.4)**, achieved the minimum quality cut-off score, indicating strong operationalization of the attributes related to access, longitudinality, comprehensiveness and coordination. Primary care services were not the principal source of care for 56% of the sample.
Harzheim et al. (2016)42	FHS and mixed health centers	Primary care attributes*	There was no significant difference in scores between FHS and mixed model health centers for the "general" and "essential" items. Neither the FHS (6.1)** nor the mixed health centers (6.0)** achieved the minimum quality cut-off score***. The FHS centers scored higher than the mixed health centers. In both types of services, the highest-scoring attributes were "strength of affiliation" (7.6 and 7.4)** and "utilization" (7.9 and 7.9)**. The lowest-scoring attributes were "accessibility" (4.8 and 4.5)**, "family centeredness" (5.4 and 5.5)**, and "community orientation" (5.4 and 3.9)**.
Silva e Fracolli (2016)37	FHS in rural areas	Primary care attributes*	The "general score" (6.2)** and "essential score" (6.4)** failed to achieve the quality cut-off score***. The attributes "utilization" (8.0)**, "longitudinality" (6.7)**, "care integration" (6.9)**, and "information systems" (7.0)** achieved the quality cut-off score. The attributes "accessibility" (4.9), "available services" (5.2)**, "services provided" (6.5)**, "family centeredness" (5.1)**, and "community orientation" (5.7)** failed to achieve the minimum score. The study highlighted that limitations in access may be linked to same-day consultation scheduling problems and difficulties in obtaining guidance by telephone. The poor evaluation of "comprehensiveness" was related to the underprovision of primary care. Low scores for "family centeredness" and "community orientation" indicate that little regard is paid to the opinion of parents/guardians and characteristics of family and community groups. FHS was the main care source in 56.3% of the sample.
Oliveira e Veríssimo (2015)21	FHS and traditional care centers in urban areas	Primary care attributes*	All attributes scored higher in the FHS centers than in the traditional health centers. FHS centers showed stronger operationalization (6.6)*** than traditional health centers (3.9)**. In the FHS centers, the attributes that failed to achieve the quality cut-off score were "accessibility" (5.1)**, "longitudinality" (5.4)**, "information systems" (6.3)**, "available services" (6.2)**, and "community orientation" (5.6)**. In the traditional health centers, none of the attributes achieved the minimum quality cut-off score. The factors that explained low scores in both types of services were: poor access and wait times, lack of availability of information registers, lack of access to child health care services, and insufficient knowledge about the family and the community. Despite satisfactory results for the FHS, the study highlighted the need to review child health care actions.
Araújo et al. (2014)38	Primary health care services	Community orientation and family centeredness*	The attributes "family centeredness" (4.4)** and "community orientation" (5.1)** failed to achieve the quality cut-off score**. The study highlighted that services adopt a predominantly traditional care approach centered on the individual rather than the family/community.
Ferrer et al. (2014)20	FHS and traditional care centers	Longitudinality of care*	The attribute "longitudinality" failed to achieve the quality cut-off score (4.9)**. FHS services scored higher than traditional care centers for this attribute. Around 66% of FHS patients considered the attribute to be adequate.
Mesquita Filho et al. (2014)39	Primary health care services	Primary care attributes*	The study reported weak operationalization of primary care attributes. The "general score" was 5.2** and 85.0% of the evaluations were negative. The only attribute to achieve the quality cut-off score** was "longitudinality" (7.8)**. The lowest-scoring attributes were "accessibility" (4.7)** and "available services" (2.9)**, receiving low scores from 91.5% and 94.6% of service users, respectively. Coordination of primary health care scored higher among child FHS service users than in those registered in traditional primary care centers (PR = 0.34). Around 77.1% of service users reported that the FHS was their preferred service for child care.
Marques et al. (2014)19	FHS in a rural quilombola community	Primary care attributes*	The "general items" failed to achieve the quality cut-off score (6.4)**. The attributes "utilization" (9.1)** and "information systems" (6.9)** achieved the quality cut-off score. The lowest-scoring attributes were "accessibility" (4.8)** and "family centeredness" (3.8)**. The low primary care attribute scores suggest there is a gap between actual and desired actions. The FHS was the preferred service for child care for more than 80% of the service users.
Perez et al. (2014)16	FHS	Service quality, health professional communication, and care problems	Professionals gave higher scores for accessibility and care quality than service users. Parents/guardians and professionals identified similar weaknesses (lack of access, staff shortages, and underqualified professionals) and strengths (communication between community health agents, provision of educational material, and pediatric care). The most commonly reported problem was staff shortages. The majority of professionals highlighted insufficient training in terms of number, content, and methods. In general, both professionals and parents/guardians reported being satisfied with the services, despite the failings. Parent/guardian satisfaction/dissatisfaction tended to be linked to the health professionals and access to health promotion and disease prevention services, respectively.
Rocha e Pedroza (2013)26	FHS in urban and rural areas	Developmental follow-up structure and process	The evaluation of care structure highlighted that the main limitation was staff shortages. With regard to process, 60% of the nurses had not received training for care actions and over 60% reported that consultations over the last three months were not recorded on the children's health cards. 75% of care delivery was based on clinical protocols. The findings show a lack of consolidation of developmental follow-up in primary health care services.
Modes e Gaíva (2013)43	Primary health care services in urban and rural areas	Structure of child care centers	The results show that 61.5% of the care centers did not have reception or waiting rooms. However, all centers had nurse and doctor consulting rooms, vaccination rooms, and dispensing areas. Shortages of materials were reported in all centers. Availability of standard medications and SINAN and SISVAN forms was reported to be good. The findings showed a lack of SIAB forms and vaccination cards. Only 53.8% of the health professionals had computers without access to the internet. The care structure was not in line with the minimum standards set by the Ministry of Health.
Modes e Gaíva (2013)40	Primary health care services	Child health care processes and outcomes	With regard to process, 60.6% of children received developmental follow-up from doctors and nurses. The least tested parameter was blood pressure. The professionals that most used the child health card are nurses; however, the cards were not filled out properly. Around 75.6% of mothers understood the topics discussed with the professionals and 62.2% reported that they had the opportunity to raise their doubts and concerns. The evaluation of care quality and health professionals was fair and good, respectively. Level of satisfaction was associated with guidance on prevention of accidents, violence, and respiratory problem, the opportunity to discuss concerns during the consultation, and free-of-charge medication.
Sales et al. (2013)15	FHS	Child care quality standards recommended by the AIQ	The quality standards were: elementary (84.1%), in development (69.7%), consolidated (58.6%), good (82.1%), and advanced (47.7%). The findings demonstrated that the quality of child health care showed basic development patterns. The results also show that the health teams that had been working for less time showed the best results.
Machado et al. (2012)24	FHS	Access, care and satisfaction	Access to health services: 91.1% of children had had consultations in the last 6 months; 77.4% used the health center closest to their home; 72.6% reported an average waiting time of at least two hours; and 51.7% received visits from a community health agent. Treatment: 74.0% reported that their last consultation was with a doctor; 57.7% knew the name of the health professional that saw them; and 63.7% of professionals called the mother/child by their name. Around two-thirds of the mothers reported being satisfied with the last consultation. The main associated factors were: the fact that the professional calls the mother/child by their name; same-day consultation; and provision of entertainment for children in the waiting room. One of the strengths of the services highlighted by the study was ease of access to services. Weaknesses included the adoption of a curative care model, with little emphasis on health promotion and disease prevention. Service user satisfaction was associated with the patient- health professional relationship.
Leão et al. (2011)22	FHS and other child care services	Primary care attributes*	The FHS scored higher than other care services. The differences between the two types of service were significant. Neither the FHS (6.4)** nor the other services (5.7)** achieved the quality cut-off score for "general items"**. Both services obtained low scores for "accessibility", "available services", "family centeredness", and "community orientation". The FHS was the preferred service for child care for 77.7% of service users.
Luhm et al. (2011)25	Primary health care services	Vaccination coverage, profile of service users, and utilization of primary care services	Child vaccination coverage ranged from 95.3% at age 12 months to 90.3% at age 24 months. Around 98% of the immunization records were computerized. The dose underreporting and duplication rate was 11% and 20.6%, respectively. Coverage was greater among children permanently registered in the services and those who had had 3 or more consultations, and in centers with complete health teams. Vaccination coverage was high and even. Affiliation with services was an important factor in achieving these results.
Leão e Caldeira (2011)44	FHS	Association between primary care attributes* and the qualification of FHS doctors and nurses	The findings show that health teams with professionals who had done family health or family and community medicine residencies obtained higher primary care attribute scores.
Costa et al. (2011)17	FHS	Structure, process, and outcome domains	Care quality was fair for all domains. With regard to structure, the study highlighted limitations in physical facilities, material shortages, lack of protocols, among others. With regard to process, the findings highlight that care was limited and fragmented, a focus on curative care, and inadequate training for health promotion and disease prevention. The findings also highlighted advances in receptiveness, limitations with regard to the development of health promotion and disease prevention, and advances in the reorganization of child care.
Ribeiro et al. (2010)41	FHS	Receptiveness of child primary health care	The findings show that the FHS was the preferred service for child care for 77.6% of respondents, principally due to the patient-health care professional relationship. With respect to receptiveness, 74.2% of service users reported that they believed that the professionals understood what they said and asked; 79.2% responded questions according to their understanding; 77.2% were able to talk to the professionals whenever they needed to; and 73,2% felt comfortable talking to the professionals. Receptiveness was considered satisfactory.
Caldeira et al. (2010)23	FHS and traditional care centers	Child health care process	FHS centers scored higher for all variables. Despite performing better than other services, the care FHS services were inadequate in terms of provision, quality, and new services.
Figueiras et al. (2003)27	FHS and traditional care centers	Knowledge and practice of child developmental surveillance among professionals	Doctors and nurses working in traditional health centers had better knowledge of child developmental surveillance than those in FHS centers. With respect to practices, only 21.8% of mothers reported having been asked about the development of their children, 27.6% stated that the professional had asked about or observed their child's development, and 14.4% had received guidance on how to stimulate their child. The study concludes that primary care professionals are not adequately trained for child developmental surveillance, meaning that surveillance is not carried out satisfactorily.

• ^*
  Primary care attributes proposed by Starfield45;
• ^**
  Primary health care attribute score based on the PCAT-CE adapted and validated for use in Brazil;
• ^***
  Quality cut-off score proposed by the PCAT-CE is 6.6 - scores equal to or greater than the cut-off indicate strong operationalization of the attribute, while scores lower than the cut-off indicate weak operationalization; FHS (Family Health Strategy); SINAN (Brazil's notifiable diseases information system); SISVAN (Brazil's nutrition surveillance system); SIAB (Brazil's primary care information system); AIQ = Assessment for Improving the Quality of the Family Health Strategy; PCAT-CE = Primary Care Assessment Tool Child Edition.

Only one study, conducted in the State of Paraná in 2015²¹ using the PCAT-CE, showed that children’s health care services provided under the FHS achieved the minimum cut-off score for service quality. In the studies that used this evaluation tool, the lowest-scoring primary care attributes were “first contact accessibility”, “family centeredness”, and “community orientation”, while the highest-scoring attributes were “first contact utilization”, “longitudinality”, and “coordination-information system”. The FHS obtained higher scores in four of the six studies that compared the performance of FHS centers and traditional health centers^20-23. Furthermore, the findings of these studies also showed that the FHS was the preferred service among service users.

The studies that employed other instruments used different aspects of primary care to evaluate care quality. One of the highest-scoring aspects was the patient-health care professional relationship^16,24,25. However, these studies also highlighted a number of problems, including: difficulties in accessing services, underqualified professional staff^16,17,26,27, staff shortages^16,26, poor structure and shortage of materials²⁶, and an emphasis on disease-based curative care over health promotion and disease prevention^16,17,24,25.

Discussion

The findings show that the most commonly used tool for evaluating primary child care in Brazil was the PCAT-CE. Furthermore, the studies mainly used quantitative methodologies, involved parents/guardians and health care professionals, covered all regions of Brazil, and concluded that care quality was inadequate.

The fact that the large majority of articles where published after 2009 reflects the trend in national scientific output in recent years^28-30. In this respect, the quality of national publications has been the target for criticism, particularly due to the lack visibility of Brazilian research in international journals^28,30. The large number of articles excluded from this review in the screening for methodological quality stage is a reflection of this situation. The rise in the number of publications in the field of health care evaluation is linked to the expansion of postgraduation programs and an increase in research funding in the country^1,8. Over the last two decades, studies conducted by research groups in partnership with the Ministry of Health have driven the development of conceptual and methodological frameworks and the application of evaluation as management tool⁸.

The predominance of quantitative methods was also found by a review of evaluation studies conducted between 2000 and 2006³¹. In this respect, until recently, qualitative approaches were seen to have less methodological rigor, thus hindering their publication in high-impact journals³². The combination of different methodological approaches provides a broader and deeper understanding of the phenomena and processes of concern³³. The fact that none of the articles used only qualitative methods suggests that the understanding of the underlying factors influencing the evaluations provided by service users and professionals may be limited.

The review shows that the majority of studies were conducted in the Southeast and South regions, which is consistent with the findings of a review of Brazilian literature in the field of pediatrics conducted by Gonçalves et al.²⁸. In this regard, the majority of Brazil’s higher education and research institutions are concentrated in these regions, facilitating their integration with services and the development of evaluation studies. The integration of education institutions with health services results in training and technical collaboration, fruits of joint research based on health service needs¹⁸.

A study conducted by Fracolli et al.³⁴ concluded that the PCAT is the most adequate tool for evaluating the quality of care services provided under the FHS and allows researchers to assess the extent to which primary care attributes are operationalized¹⁰. The Essential Score is made up of the attributes Strength of affiliation, First contact accessibility/utilization, Longitudinality, Coordination-integration of care and information systems, and Comprehensiveness of services available and provided. The Derived Score is made up of the attributes Family centeredness and Community orientation¹⁰. The analysis of each of these attributes enables health managers to tailor actions to the determinants of care quality in each service/setting. One of the advantages of the PCAT is that the instrument has been validated and applied in other countries^19,34,35. The tool has three editions (adult, child, and professional), which enables comparisons between different actors and types of primary care organizations^10,15.

Evaluation instruments should be able to identify service strengths and weaknesses and be recognized by the scientific community³⁴. The only evaluation instrument used in more than one study was the PCAT-CE. The other instruments used in the studies were mainly elaborated and validated by the researchers according to the specific objectives of each study. Although this might allow researchers to investigate the specific aspects of given services and settings, the use of individual tools limits comparison and the extrapolation of results. In this respect, cross-cultural adaptation and validation may not always be sufficient to allow the evaluation of different contexts and the peculiarities of different types of services.

Besides the evaluation of service performance, some of the studies compared different primary care models. Although it recognizes different models, the National Primary Health Care Policy¹⁰ states that the FHS is the basic care model underpinning primary care services in Brazil. The FHS differs from other models insofar as multiprofessional health teams are responsible for delivering a suite of integrated services to a specific number of families in a specific geographical area¹⁰, while traditional primary care centers emphasize clinical and health interventions¹⁰.

Although evidence shows that the FHS performs better in comparison to traditional care models, the review shows that this model has a number of limitations. In this respect, accessibility, understood as people’s ability to get care easily and conveniently³⁶, was one of the lowest-scoring attributes.

Although studies frequently highlighted the strength of affiliation with the FHS, geographic barriers, lack of adequate service structure, staff shortages, underprovision of care, appointment scheduling problems, and long wait times were shown to be major obstacles to improving access to high quality health services³⁷. These obstacles also affect child health indicators, highlighted as one of the main challenges of the PNAISC².

Lack of family and community centeredness was one of the most common limitations cited by the literature^22,38, suggesting that care is centered on the individual, with little emphasis on patient contextual factors. Social disadvantage, poor hygiene habits, unhealthy environments, and family violence are just some factors that adversely affect child health and therefore increase the demand for care services. The lack of health promotion and disease prevention actions culminates in not only increased demand for health services, but also a reduction in quality of care delivery^39,40. In this respect, training of care professionals is a decisive factor in tackling determinants of child health such as the domestic environment, family life, and antenatal care and care throughout labor and birth².

The quality of the patient-health care professional relationship, which encompasses access to health workers and information, the fact that patients know the professionals, receptiveness, and forming bonds with patients, was highlighted as a positive point^16,23,41. Improving the quality of the patient-health care professional relationship is one of the pillars of quality primary health care and one of the guidelines of the PNAISC^2,23 and, despite the organizational and structural challenges faced by the FHS, important strides have been taken in this direction.

One of the limitations of this study is the fact that screening for methodological quality was performed by only one reviewer; however, we adopted a rigorous study methodology, following the PRISMA statement for reporting systematic reviews and the STROBE and SRQR guidelines.

Conclusion

The findings show that the highest-scoring primary care attributes were strength of affiliation and the patient-health care professional relationship. Furthermore, despite the expansion of the FHS, accessibility remains a major obstacle to securing the necessary improvements in care quality. The lack of family centeredness and community orientation underscores the need for adequate staff training, extending the challenge beyond policy makers to include education institutions. In this respect, the latter need to prioritize the training of health professionals who have the necessary knowledge, practical skills, and commitment to strengthen primary health care.

Future research should investigate contexts outside the South and Southeast regions, focusing on vulnerable populations and more economically disadvantaged areas where it is difficult to attract and retain health workers, in order to help identify and respond to the challenges in improving primary child health care in Brazil.

Referências

• ¹
  Victora CG, Aquino EM, do Carmo Leal M, Monteiro CA, Barros FC, Szwarcwald CL. Maternal and child health in Brazil: progress and challenges. Lancet 2011; 377(9780):1863-1876.
• ²
  Brasil. Ministério da Saúde (MS). Secretaria de Atenção em Saúde. Departamento de Ações Programáticas. Política Nacional de Atenção Integral à Saúde da Criança: orientações para implementação Brasília: MS; 2018.
• ³
  Liu L, Oza S, Hogan D, Perin J, Rudan I, Lawn JE, Cousens S, Mathers C, Black RE. Global, regional, and national causes of child mortality in 2000-13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet 2015; 385(9966):430-440.
• ⁴
  Instituto Brasileiro de Geografia e Estatística (IBGE). Taxas de Mortalidade Infantil [Internet]. [acessado 2016 Abr 27]. Disponível em: http://brasilemsintese.ibge.gov.br/populacao/taxas-de-mortalidade-infantil.html
  » http://brasilemsintese.ibge.gov.br/populacao/taxas-de-mortalidade-infantil.html
• ⁵
  Figueiredo AM, Kuchenbecker RD, Harzheim E, Vigo Á, Hauser L, Chomatas ER. Análise de concordância entre instrumentos de avaliação da Atenção Primária à Saúde na cidade de Curitiba, Paraná, em 2008. Epidemiol Serv Saúde 2013; 22(1):41-48.
• ⁶
  Carvalho AL, Souza MD, Shimizu HE, Senra IM, Oliveira KC. A gestão do SUS e as práticas de monitoramento e avaliação: possibilidades e desafios para a construção de uma agenda estratégica. Cien Saúde Colet 2012; 17(4):901-911.
• ⁷
  Brousselle A, Champagne F, Contandriopoulos AP, Hartz Z. Avaliação: conceitos e métodos Rio de Janeiro: Fiocruz; 2011.
• ⁸
  Furtado JP, Vieira-da-Silva LM. A avaliação de programas e serviços de saúde no Brasil enquanto espaço de saberes e práticas. Cad Saúde Pública 2014; 30:2643-2655.
• ⁹
  Savassi LC. Qualidade em serviços públicos: os desafios da atenção primária. RBMFC 2012; 7(23):69-74.
• ¹⁰
  Brasil. Ministério da Saúde (MS). Secretaria de Atenção em Saúde. Manual do instrumento de avaliação da atenção primária à saúde: primary care assessment tool pcatool Brasília: MS; 2010.
• ¹¹
  Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009; 6(7):e1000100.
• ¹²
  Vandenbroucke JP, Von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M, Strobe Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med 2007; 4(10):e297.
• ¹³
  Malta M, Cardoso LO, Bastos FI, Magnanini MM, Silva CM. STROBE initiative: guidelines on reporting observational studies. Rev Saúde Pública 2010; 44(3):559-565.
• ¹⁴
  O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Academic Med 2014; 89(9):1245-1251.
• ¹⁵
  Sales ML, Ponnet L, Campos CE, Demarzo MM, de Miranda CT. Qualidade da atenção à saúde da criança na estratégia saúde da família. J Human Growth Dev 2013; 23(2):151-156.
• ¹⁶
  Perez LG, Sheridan JD, Nicholls AY, Mues KE, Saleme PS, Resende JC, Ferreira JA, Leon JS. Professional and community satisfaction with the Brazilian family health strategy. Rev Saúde Públ 2013; 47(2):403-413.
• ¹⁷
  Costa GD, Cotta RM, Reis JR, Ferreira MD, Reis RS, Franceschini SD. Avaliação da atenção à saúde da criança no contexto da Saúde da Família no município de Teixeiras, Minas Gerais, Brasil. Cien Saúde Colet 2011; 1:3229-3240.
• ¹⁸
  Silva CB, Paula CC, Lopes LFD, Harzheim E, Magnago TSBS, Schimith MD. Atenção à saúde de criança e adolescente com HIV: comparação entre serviços. Rev Bras Enferm 2016; 69(3):522-531.
• ¹⁹
  Marques AS, Freitas DA, Alves Leão CD, Oliveira M, Ketllin S, Pereira MM, Caldeira AP. Atenção Primária e saúde materno-infantil: a percepção de cuidadores em uma comunidade rural quilombola. Cien Saúde Colet 2014; 19(2):365-371.
• ²⁰
  Ferrer AP, Brentani AV, Sucupira AC, Navega AC, Cerqueira ES, Grisi SJ. The effects of a people-centred model on longitudinality of care and utilization pattern of healthcare services-Brazilian evidence. Health Policy Planning 2014; 29(2):107-113.
• ²¹
  Oliveira VB, Veríssimo MD. Assistência à saúde da criança segundo suas famílias: comparação entre modelos de Atenção Primária. Rev Esc EnfermUSP 2015; 49(1):30-36.
• ²²
  Leão CD, Caldeira AP, Oliveira MM. Atributos da atenção primária na assistência à saúde da criança: avaliação dos cuidadores. Rev Bras Saúde Matern Infant 2011: 11(3):323-334.
• ²³
  Caldeira AP, Oliveira RM, Rodrigues OA. Qualidade da assistência materno-infantil em diferentes modelos de Atenção Primária. Cien Saúde Colet 2010; 15(Supl. 2):3139-3147.
• ²⁴
  Machado MM, Bezerra Filho JG, Machado MD, Lindsay AC, Magalhães FB, Gama ID, Cunha AJ. Características dos atendimentos e satisfação das mães com a assistência prestada na atenção básica a menores de 5 anos em Fortaleza, Ceará. Cien Saúde Colet 2012; 17(11):3125-3133.
• ²⁵
  Luhm KR, Cardoso MR, Waldman EA. Cobertura vacinal em menores de dois anos a partir de registro informatizado de imunização em Curitiba, PR. Rev Saúde Publ 2011; 45(1):90-98.
• ²⁶
  Rocha AC, Pedraza DF. Acompanhamento do crescimento infantil em unidades básicas de saúde da família do município de Queimadas, Paraíba, Brasil. Texto Contexto-Enferm 2013; 22(4):1169-1178.
• ²⁷
  Figueiras ACM, Puccini RF, Silva EM, Pedromônico MR. Avaliação das práticas e conhecimentos de profissionais da atenção primária à saúde sobre vigilância do desenvolvimento infantil. Cad Saúde Públ 2003; 19(6):1691-1699.
• ²⁸
  Gonçalves E, Santos MI, Maia BT, Brandão RC, Oliveira EA, Martelli Júnior H. Scientific research in pediatrics produced at the CNPq. Rev Bras Educ Med 2014; 38(3):349-355.
• ²⁹
  Petherick A. High hopes for Brazilian science. Nature 2010; 465:674-675.
• ³⁰
  Strehl L, Calabró L, Souza DO, Amaral L. Brazilian Science between National and Foreign Journals: Methodology for Analyzing the Production and Impact in Emerging Scientific Communities. Bornmann L. PLoS ONE 2016; 11(5):e0155148.
• ³¹
  Almeida PF, Giovanella L. Assessment of Primary Health care in Brazil: mapping and analysis of research conducted and/or financed by the Ministry of Health from 2000 to 2006. Cad Saúde Pública 2008; 24(8):1727-1742.
• ³²
  Taquette SR, Minayo MCS, Rodrigues AO. Percepção de pesquisadores médicos sobre metodologias qualitativas. Cad. Saúde Pública 2015; 31(4):722-732.
• ³³
  Tanaka OY. Avaliação da atenção básica em saúde: uma nova proposta. Saúde Soc 2011; 20:927-934.
• ³⁴
  Fracolli LA, Gomes MF, Nabão FR, Santos MS, Cappellini VK, Almeida AC. Primary health care assessment tools: a literature review and metasynthesis. Cien Saúde Colet 2014; 19(12):4851-4860.
• ³⁵
  Malouin RA, Starfield B, Sepulveda MJ. Evaluating the tools used to assess medical home. Manag Care 2009; 18(6):44-48.
• ³⁶
  Donabedian A. An introduction to quality assurance in health care Oxford University: Press; 2002.
• ³⁷
  Silva SA, Fracolli LA. Avaliação da assistência à criança na Estratégia de Saúde da Família. Rev Bras Enf 2016; 69(1):47-53.
• ³⁸
  Araújo JP, Viera CS, Toso BR, Collet N, Nassar PO. Avaliação dos atributos de orientação familiar e comunitária na saúde da criança. Acta Paul Enferm 2014; 27(5):440-446.
• ³⁹
  Mesquita Filho M, Luz BS, Araújo CS. A Atenção Primária à Saúde e seus atributos: a situação das crianças menores de dois anos segundo suas cuidadoras. Cien Saúde Colet 2014; 19(7):2033-2046.
• ⁴⁰
  Modes PS, Gaiva MA. Satisfação das usuárias quanto à atenção prestada à criança pela rede básica de saúde. Esc Anna Nery Rev Enferm 2013; 17(3):455-465.
• ⁴¹
  Ribeiro LD, Rocha RL, Ramos-Jorge ML. Acolhimento às crianças na atenção primária à saúde: um estudo sobre a postura dos profissionais das equipes de saúde da família. Cad Saúde Pública 2010; 26(12):2316-2322.
• ⁴²
  Harzheim E, Hauser L, Pinto LF, Soranz D. Avaliação dos usuários crianças e adultos quanto ao grau de orientação para Atenção Primária à Saúde na cidade do Rio de Janeiro, Brasil. Cien Saúde Colet 2016; 21(5):1399-1408.
• ⁴³
  Modes PSSA, Gaíva MA. Structure of children's basic health units: descriptive study. Online Braz J Nur 2013; 12(3):471-481.
• ⁴⁴
  Leão CD, Caldeira AP. Avaliação da associação entre qualificação de médicos e enfermeiros em atenção primária em saúde e qualidade da atenção. Cien Saúde Colet 2011; 16(11):4415-4423.
• ⁴⁵
  Starfield B, Xu J, Shi L. Validating the Adult Primary Care Assessment Tool. J Fam Practice 2001; 50(2):161-175.

Publication Dates

• Publication in this collection
  05 Aug 2020
• Date of issue
  Aug 2020

History

• Received
  04 Mar 2018
• Accepted
  27 Nov 2018
• Published
  29 Nov 2018