evaluation of primary child health care in Brazil : a systematic review of methods and results

This systematic review analyzes the methods and instruments employed to evaluate primary child health care in Brazil and their main findings. The review was conducted in accordance with the recommendations of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyzes) statement. Searches of articles focusing on children aged between zero and five years published between 1994-2006 were conducted of the following databases: MEDLINE, LILACS, IBECS, BDENF, PubMed, PsycNET, Cochrane, and CINAHL. The searches yielded 3,004 articles. After initial screening and the application of the STROBE and SRQR criteria, 21 articles were included in the review. About 52% of the articles were conducted in the Southeast region and 95.2% were published as of 2010. The most commonly used evaluation tool was the Primary Care Assessment Tool Child Edition, adapted and validated for use in Brazil (52.4%). The quality of primary child care was inadequate. The main limitations included poor access to services, inadequate facilities, and underqualified health staff. There has been a significant increase in the number of evaluation studies conducted in Brazil in recent years. Despite advances in health care across the country, the findings point to the need for a more effective response to the challenges in ensuring comprehensive primary child care in Brazil.


introduction
Brazil has made significant strides in improving child health in recent decades as a result of improved living conditions, the advancement of children's rights, and advances in the country's health policy 1,2 .This is illustrated by a sharp fall in infant mortality rates 1,3 , from 85.6 per 1,000 live births in 1980 to 13.8 in 2015 4 , when Brazil achieved target 4 of the Millennium Development Goals (Reduce by two-thirds, between 1990 and 2015, the under-five mortality) 3 .
Brazil began to implement specific interventions targeting child health in the 1960s with the Maternal and Child Health Program.In the 1990s, initiatives began to incorporate policies designed to expand the coverage of public health services, as in the case of the family health and community health agent programs.With the restructuring of primary health care services, it was possible to improve the distribution of health care professionals across underserved areas and widen access to health services, positively impacting child health indicators 2 .The creation of the National Policy for Comprehensive Child Health Care (PNAISC, acronym in Portuguese), in 2015, was the culmination of interinstitutional efforts to enhance actions directed at young children and vulnerable groups, based on the principles of the right to life, equality, comprehensive humanized care, and participatory management 2 .
Brazil's Family Health Strategy (FHS) has consolidated primary health care, expanding service coverage across the country.However, numerous factors continue to limit coverage, including the coexistence of traditional and family care models in the same service, structural and organizational factors, and staff shortages, driving the institutionalization of evaluation processes to inform strategies to strengthen primary health care [5][6][7] .
Evaluation is a process in which, based on valid and legitimate information, value judgments are made about interventions, processes, and outcomes with a view to enhancing service management performance, facilitating change, decision-making, and the pursuit of quality 7 .The growing recognition of the importance of evaluation has led to the development of a suite of instruments and methods for evaluating the adequacy of public services in Brazil over the last three decades 8 .This process has been led by the Ministry of Health, with initiatives like the "Sala de Situação" or "Health Situation Room" (2002), the Assessment for Improving the Quality of the Family Health Strategy -AIQ (Avaliação para a Melhoria da Qualidade -AMQ) 9 , the Primary Care Assessment Tool 10 (2010), and the National Program for Improving Access and Quality in Primary Care (2011) 9 .In addition, research groups at universities across the country have also developed a number of initiatives 8 .
In view of the need to synthesize the knowledge accumulated in recent years, this review analyzes the methods and instruments employed to evaluate primary child health care in Brazil and their main findings.

Methods
We conducted a systematic review based on the recommendations set out in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement 11 .Searches were conducted of the databases Virtual Health Library, Brazil (MEDLINE, LILACS, IBECS, and BDENF), PubMed, PsycNET, Cochrane, and CINAHL, using the following search terms: "primary health care OR family health strategy AND evaluation AND child health OR child", based on the DeCS (Health Sciences Descriptors) and MeSH (Medical Subject Headings).
We included studies evaluating primary health care services for children aged 0 to 19 years (focusing on the zero to five-years age group) published in English, Spanish and Portuguese between January 1994 (the year in which the FHS was created) and September 2016.Literature reviews, theses, dissertations, editorials, case study reports, studies with data on non-Brazilian children, and studies that did not meet 80% of the required items of the methodological quality scales used by this study were excluded.
The methodological quality of observational studies was assessed in accordance with the STROBE 12 (Strengthening the reporting of observational studies in epidemiology) statement, consisting of a checklist of 22 items that should be addressed in articles.The version translated and validated for use in Brazil in 2008 13 establishes article quality categories.Articles that meet at least 80% of the items are classified as category "A" 12,13 .Qualitative studies were assessed using the Standards for Reporting Qualitative Research (SRQR) 14 .Studies that used both quantitative and qualitative methodologies were assessed using both instruments and included if they met at least 80% or higher of the items of one of the instruments.One independent reviewer assessed methodological quality and two reviewers evaluated thematic eligibility.

results
The searches yielded 3,004 articles, 538 of which were excluded because they were duplicates.A total of 2,333 articles were excluded after the analysis of the titles and abstracts because they failed to meet the study eligibility criteria.The full texts of the remaining citations were then examined, resulting in the exclusion of 99 articles because they failed to meet the study inclusion criteria.The remaining articles were then assessed against the STROBE and SRQR checklists.A total of 13 publications were excluded because they failed to meet at least 80% of the items, resulting in a final sample of 21 articles.The article selection process is shown in Figure 1.
Table 1 presents the general characteristics of the selected studies.Around 95% of the articles where published after 2009.All the studies used quantitative methodologies and two used a combination of quantitative and qualitative methodologies.We did not find any qualitative studies that met the inclusion criteria.The most commonly used evaluation instrument was the Primary Care Assessment Tool Child Edition (PCAT-CE) adapted and validated for use in Brazil (52.4%).The other studies used instruments developed by the researchers, except for a study conducted in Maceió in 2003 15 , which used the AIQ.
Only two studies 16,17 included both health care professionals and service users.In 85.7% of the studies, only parents and/or guardians evaluated services, in the majority of cases the children's mothers.In the other studies, doctors and nurses evaluated services.The studies evaluated children's and young people's services, with 61.9% of studies focusing specifically on the zero to five-year age group.Some studies focused on specific groups, such as children living with HIV 18 , quilombolas 19 , and children hospitalized for ambulatory care sensitive conditions 20 .
The studies were conducted in areas with varying population characteristics, ranging from state capitals to rural communities.Eleven of the 21 studies were conducted in the Southeast region, nine of which in the State of Minas Gerais (Figure 2).
Table 2 shows a synthesis of the methods and main conclusions of the selected articles.All the studies assessed the performance of family health teams, 52.4% investigated traditional health centers or a mixture of traditional and FHS centers, and 14.3% addressed other types of models.Almost all of the studies investigated service structure and work processes.The main aspects of care investigated were: primary care attributes (47.7%); specific actions developed by primary health care services, such as immunization programs, acolhimento or "receptiveness", and child health surveillance (33.3%); care structure (9.5%); and health care professional training and development (9.5%).
Only one study, conducted in the State of Paraná in 2015 21 using the PCAT-CE, showed that children's health care services provided under the FHS achieved the minimum cut-off score for service quality.In the studies that used this evaluation tool, the lowest-scoring primary care attributes were "first contact accessibility", "family centeredness", and "community orientation", while the highest-scoring attributes were "first contact utilization", "longitudinality", and "coordination-information system".The FHS obtained higher scores in four of the six studies that compared the performance of FHS centers and traditional health centers [20][21][22][23] .Furthermore, the findings of these studies also showed that the FHS was the preferred service among service users.
The studies that employed other instruments used different aspects of primary care to evaluate care quality.One of the highest-scoring aspects was the patient-health care professional relationship 16,24,25 .However, these studies also highlighted a number of problems, including: difficulties in accessing services, underqualified professional staff 16,17,26,27 , staff shortages 16,26 , poor structure and shortage of materials 26 , and an emphasis on disease-based curative care over health promotion and disease prevention 16,17,24,25 .

Discussion
The findings show that the most commonly used tool for evaluating primary child care in Brazil was the PCAT-CE.Furthermore, the studies mainly used quantitative methodologies, involved parents/guardians and health care professionals, covered all regions of Brazil, and concluded that care quality was inadequate.
The fact that the large majority of articles where published after 2009 reflects the trend in national scientific output in recent years [28][29][30] .In this respect, the quality of national publications has been the target for criticism, particularly due to the lack visibility of Brazilian research in international journals 28,30 .The large number of articles excluded from this review in the screening for methodological quality stage is a reflection of this situation.The rise in the number of publications in the field of health care evaluation is linked to the expansion of postgraduation programs and an increase in research funding in the country 1,8 .Over the last two decades, studies conducted by research groups in partnership with the Ministry of Health have driven the development of conceptual and methodological frameworks and the application of evaluation as management tool 8 .
The predominance of quantitative methods was also found by a review of evaluation studies conducted between 2000 and 2006 31 .In this respect, until recently, qualitative approaches were seen to have less methodological rigor, thus hindering their publication in high-impact journals 32 .The combination of different meth-odological approaches provides a broader and deeper understanding of the phenomena and processes of concern 33 .The fact that none of the articles used only qualitative methods suggests that the understanding of the underlying factors influencing the evaluations provided by service users and professionals may be limited.
The review shows that the majority of studies were conducted in the Southeast and South regions, which is consistent with the findings of a review of Brazilian literature in the field of pediatrics conducted by Gonçalves et al. 28 .In this regard, the majority of Brazil's higher education and research institutions are concentrated in these regions, facilitating their integration with services and the development of evaluation studies.The integration of education institutions with health services results in training and technical collaboration, fruits of joint research based on health service needs 18 .A study conducted by Fracolli et al. 34 concluded that the PCAT is the most adequate tool for evaluating the quality of care services provided under the FHS and allows researchers to assess the extent to which primary care attributes are operationalized 10 .The Essential Score is made up of the attributes Strength of affiliation, First contact accessibility/utilization, Longitudinality, Coordination-integration of care and information systems, and Comprehensiveness of services available and provided.The Derived Score is made up of the attributes Family centeredness and Community orientation 10 .The analysis of each of these attributes enables health managers to tailor actions to the determinants of care quality in each service/setting.One of the advantages of the PCAT is that the instrument has been validated and applied in other countries 19,34,35 .The tool has three editions (adult, child, and professional), which enables comparisons between different actors and types of primary care organizations 10,15 .
Evaluation instruments should be able to identify service strengths and weaknesses and be recognized by the scientific community 34 .The only evaluation instrument used in more than one study was the PCAT-CE.The other instruments used in the studies were mainly elaborated and validated by the researchers according to the specific objectives of each study.Although this might allow researchers to investigate the specific aspects of given services and settings, the use of individual tools limits comparison and the extrapolation of results.In this respect, cross-cultural adaptation and validation may not always be sufficient to allow the evaluation of different contexts and the peculiarities of different types of services.
Besides the evaluation of service performance, some of the studies compared different primary care models.Although it recognizes different models, the National Primary Health Care Policy 10 states that the FHS is the basic care model underpinning primary care services in Brazil.The FHS differs from other models insofar as multiprofessional health teams are responsible for delivering a suite of integrated services to a specific number of families in a specific geographical area 10 , while traditional primary care centers emphasize clinical and health interventions 10 .
Although evidence shows that the FHS performs better in comparison to traditional care models, the review shows that this model has a number of limitations.In this respect, accessibility, understood as people's ability to get care easily and conveniently 36 , was one of the lowest-scoring attributes.Primary care attributes* The attributes scored higher in the specialist service (6.4)** than in the primary health care services (6.1)**.Neither of the services achieved the minimum quality cut-off score (6.6)*** for "general items".The lowest-scoring attributes were "family centeredness" and "community orientation".All other attributes, except "services provided" (6.4)**, achieved the minimum quality cut-off score, indicating strong operationalization of the attributes related to access, longitudinality, comprehensiveness and coordination.Primary care services were not the principal source of care for 56% of the sample.

FHS and mixed health centers
Primary care attributes* There was no significant difference in scores between FHS and mixed model health centers for the "general" and "essential" items.Neither the FHS (6.1)** nor the mixed health centers (6.0)** achieved the minimum quality cut-off score***.The FHS centers scored higher than the mixed health centers.In both types of services, the highest-scoring attributes were "strength of affiliation" (7.6 and 7.4)** and "utilization" (7.9 and 7.9)**.The lowest-scoring attributes were "accessibility" (4.8 and 4.5)**, "family centeredness" (5.4 and 5.5)**, and "community orientation" (5.4 and 3.9)**.

Silva e
Fracolli (2016) 37 FHS in rural areas Primary care attributes* The "general score" (6.2)** and "essential score" (6.4)** failed to achieve the quality cut-off score***.The attributes "utilization" (8.0)**, "longitudinality" (6.7)**, "care integration" (6.9)**, and "information systems" (7.0)** achieved the quality cut-off score.The attributes "accessibility" (4.9), "available services" (5.2)**, "services provided" (6.5)**, "family centeredness" (5.1)**, and "community orientation" (5.7)** failed to achieve the minimum score.The study highlighted that limitations in access may be linked to same-day consultation scheduling problems and difficulties in obtaining guidance by telephone.The poor evaluation of "comprehensiveness" was related to the underprovision of primary care.Low scores for "family centeredness" and "community orientation" indicate that little regard is paid to the opinion of parents/guardians and characteristics of family and community groups.FHS was the main care source in 56.3% of the sample.

FHS and traditional care centers in urban areas
Primary care attributes* All attributes scored higher in the FHS centers than in the traditional health centers.FHS centers showed stronger operationalization (6.6)*** than traditional health centers (3.9)**.In the FHS centers, the attributes that failed to achieve the quality cut-off score were "accessibility" (5.1)**, "longitudinality" (5.4)**, "information systems" (6.3)**, "available services" (6.2)**, and "community orientation" (5.6)**.In the traditional health centers, none of the attributes achieved the minimum quality cut-off score.The factors that explained low scores in both types of services were: poor access and wait times, lack of availability of information registers, lack of access to child health care services, and insufficient knowledge about the family and the community.Despite satisfactory results for the FHS, the study highlighted the need to review child health care actions.

FHS and traditional care centers
Longitudinality of care* The attribute "longitudinality" failed to achieve the quality cut-off score (4.9)**.FHS services scored higher than traditional care centers for this attribute.Around 66% of FHS patients considered the attribute to be adequate.
(2014) 39 Primary health care services Primary care attributes* The study reported weak operationalization of primary care attributes.The "general score" was 5.2** and 85.0% of the evaluations were negative.The only attribute to achieve the quality cut-off score** was "longitudinality" (7.8)**.The lowest-scoring attributes were "accessibility" (4.7)** and "available services" (2.9)**, receiving low scores from 91.5% and 94.6% of service users, respectively.Coordination of primary health care scored higher among child FHS service users than in those registered in traditional primary care centers (PR = 0.34).Around 77.1% of service users reported that the FHS was their preferred service for child care.

Marques et
al. (2014) 19 FHS in a rural quilombola community Primary care attributes* The "general items" failed to achieve the quality cut-off score (6.4)**.The attributes "utilization" (9.1)** and "information systems" (6.9)** achieved the quality cut-off score.The lowest-scoring attributes were "accessibility" (4.8)** and "family centeredness" (3.8)**.The low primary care attribute scores suggest there is a gap between actual and desired actions.The FHS was the preferred service for child care for more than 80% of the service users.
Perez et al.

FHS
Service quality, health professional communication, and care problems Professionals gave higher scores for accessibility and care quality than service users.Parents/guardians and professionals identified similar weaknesses (lack of access, staff shortages, and underqualified professionals) and strengths (communication between community health agents, provision of educational material, and pediatric care).The most commonly reported problem was staff shortages.The majority of professionals highlighted insufficient training in terms of number, content, and methods.In general, both professionals and parents/guardians reported being satisfied with the services, despite the failings.Parent/guardian satisfaction/dissatisfaction tended to be linked to the health professionals and access to health promotion and disease prevention services, respectively.

FHS in urban and rural areas
Developmental follow-up structure and process The evaluation of care structure highlighted that the main limitation was staff shortages.With regard to process, 60% of the nurses had not received training for care actions and over 60% reported that consultations over the last three months were not recorded on the children's health cards.75% of care delivery was based on clinical protocols.The findings show a lack of consolidation of developmental follow-up in primary health care services.

Modes
e Gaíva (2013) 43 Primary health care services in urban and rural areas Structure of child care centers The results show that 61.5% of the care centers did not have reception or waiting rooms.However, all centers had nurse and doctor consulting rooms, vaccination rooms, and dispensing areas.Shortages of materials were reported in all centers.Availability of standard medications and SINAN and SISVAN forms was reported to be good.The findings showed a lack of SIAB forms and vaccination cards.Only 53.8% of the health professionals had computers without access to the internet.The care structure was not in line with the minimum standards set by the Ministry of Health.
chart 2. Results of the analysis of the articles included in the review, Belo Horizonte, Minas Gerais, Brazil, 2017.Child health care processes and outcomes With regard to process, 60.6% of children received developmental follow-up from doctors and nurses.The least tested parameter was blood pressure.The professionals that most used the child health card are nurses; however, the cards were not filled out properly.Around 75.6% of mothers understood the topics discussed with the professionals and 62.2% reported that they had the opportunity to raise their doubts and concerns.The evaluation of care quality and health professionals was fair and good, respectively.Level of satisfaction was associated with guidance on prevention of accidents, violence, and respiratory problem, the opportunity to discuss concerns during the consultation, and free-of-charge medication.

FHS
Access, care and satisfaction Access to health services: 91.1% of children had had consultations in the last 6 months; 77.4% used the health center closest to their home; 72.6% reported an average waiting time of at least two hours; and 51.7% received visits from a community health agent.Treatment: 74.0% reported that their last consultation was with a doctor; 57.7% knew the name of the health professional that saw them; and 63.7% of professionals called the mother/child by their name.Around two-thirds of the mothers reported being satisfied with the last consultation.The main associated factors were: the fact that the professional calls the mother/child by their name; same-day consultation; and provision of entertainment for children in the waiting room.One of the strengths of the services highlighted by the study was ease of access to services.Weaknesses included the adoption of a curative care model, with little emphasis on health promotion and disease prevention.Service user satisfaction was associated with the patient-health professional relationship.

FHS and other child care services
Primary care attributes* The FHS scored higher than other care services.The differences between the two types of service were significant.Neither the FHS (6.4)** nor the other services (5.7)** achieved the quality cut-off score for "general items"**.Both services obtained low scores for "accessibility", "available services", "family centeredness", and "community orientation".The FHS was the preferred service for child care for 77.7% of service users.Although studies frequently highlighted the strength of affiliation with the FHS, geographic barriers, lack of adequate service structure, staff shortages, underprovision of care, appointment scheduling problems, and long wait times were shown to be major obstacles to improving access to high quality health services 37 .These obstacles also affect child health indicators, highlighted as one of the main challenges of the PNAISC 2 .
Lack of family and community centeredness was one of the most common limitations cited by the literature 22,38 , suggesting that care is centered on the individual, with little emphasis on patient contextual factors.Social disadvantage, poor hygiene habits, unhealthy environments, and family violence are just some factors that adversely affect child health and therefore increase the demand for care services.The lack of health promotion and disease prevention actions culminates in not only increased demand for health services, but also a reduction in quality of care delivery 39,40 .In this respect, training of care professionals is a decisive factor in tackling determinants of child health such as the domestic environment, family life, and antenatal care and care throughout labor and birth 2 .
The quality of the patient-health care professional relationship, which encompasses access to health workers and information, the fact that patients know the professionals, receptiveness, and forming bonds with patients, was highlighted as a positive point 16,23,41 .Improving the quality of the patient-health care professional relationship is one of the pillars of quality primary health care and one of the guidelines of the PNAISC 2,23 and, despite the organizational and structural challenges faced by the FHS, important strides have been taken in this direction.
One of the limitations of this study is the fact that screening for methodological quality was performed by only one reviewer; however, we adopted a rigorous study methodology, following the PRISMA statement for reporting systematic reviews and the STROBE and SRQR guidelines.

conclusion
The findings show that the highest-scoring primary care attributes were strength of affiliation and the patient-health care professional relationship.Furthermore, despite the expansion of the FHS, accessibility remains a major obstacle to securing the necessary improvements in care quality.The lack of family centeredness and community orientation underscores the need for adequate staff training, extending the challenge beyond policy makers to include education institutions.In this respect, the latter need to prioritize the training of health professionals who have the necessary knowledge, practical skills, and commitment to strengthen primary health care.
Future research should investigate contexts outside the South and Southeast regions, focusing on vulnerable populations and more economically disadvantaged areas where it is difficult to attract and retain health workers, in order to help identify and respond to the challenges in improving primary child health care in Brazil.
The attributes "family centeredness" (4.4)** and "community orientation" (5.1)** failed to achieve the quality cut-off score**.The study highlighted that services adopt a predominantly traditional care approach centered on the individual rather than the family/ community.

it continues chart 2 .
Results of the analysis of the articles included in the review, Belo Horizonte, Minas Gerais, Brazil, 2017.
The findings demonstrated that the quality of child health care showed basic development patterns.The results also show that the health teams that had been working for less time showed the best results.
ranged from 95.3% at age 12 months to 90.3% at age 24 months.Around 98% of the immunization records were computerized.The dose underreporting and duplication rate was 11% and 20.6%, respectively.Coverage was greater among children permanently registered in the services and those who had had 3 or more consultations, and in centers with complete health teams.Vaccination coverage was high and even.Affiliation with services was an important factor in achieving Results of the analysis of the articles included in the review, Belo Horizonte, Minas Gerais, Brazil, 2017.fair for all domains.With regard to structure, the study highlighted limitations in physical facilities, material shortages, lack of protocols, among others.With regard to process, the findings highlight that care was limited and fragmented, a focus on curative care, and inadequate training for health promotion and disease prevention.The findings also highlighted advances in receptiveness, limitations with regard to the development of health promotion and disease prevention, and advances in the reorganization of child care.that the FHS was the preferred service for child care for 77.6% of respondents, principally due to the patienthealth care professional relationship.With respect to receptiveness, 74.2% of service users reported that they believed that the professionals understood what they said and asked; 79.2% responded questions according to their understanding; 77.2% were able to talk to the professionals whenever they needed to; and 73,2% felt comfortable talking to the professionals.Receptiveness was considered satisfactory.higher for all variables.Despite performing better than other services, the care FHS services were inadequate in terms of provision, quality, and new services.working in traditional health centers had better knowledge of child developmental surveillance than those in FHS centers.With respect to practices, only 21.8% of mothers reported having been asked about the development of their children, 27.6% stated that the professional had asked about or observed their child's development, and 14.4% had received guidance on how to stimulate their child.The study concludes that primary care professionals are not adequately trained for child developmental surveillance, meaning that surveillance is not carried out satisfactorily.Primary care attributes proposed by Starfield45; **Primary health care attribute score based on the PCAT-CE adapted and validated for use in Brazil;***Quality cut-off score proposed by the PCAT-CE is 6.6 -scores equal to or greater than the cut-off indicate strong operationalization of the attribute, while scores lower than the cut-off indicate weak operationalization; FHS (Family Health Strategy); SINAN (Brazil's notifiable diseases information system); SISVAN (Brazil's nutrition surveillance system); SIAB (Brazil's primary care information system); AIQ = Assessment for Improving the Quality of the Family Health Strategy; PCAT-CE = Primary Care Assessment Tool Child Edition.