Validity and reliability of the Brazilian Primary Care Assessment Tool: Oral Health of Adults

REV BRAS EPIDEMIOL 2020; 23: E200076 ABSTRACT: Objective: To test the factorial validity and reliability of the Primary Care Assessment Tool adapted to Oral Health, adult patient version, an instrument used to verify the presence and extent of attributes in Primary Health Care services. Methods: Population-based cross-sectional study using conglomerate random sample carried out in Primary Health Care Dental services in Porto Alegre between 2011 and 2013. We interviewed 407 adult patients who used Primary Health Care Dental services. Construct validity was tested through factorial validity and reliability of the Primary Care Assessment Tool, that comprises 81 items distributed throughout Primary Health Care attributes. Equamax orthogonal rotation method was used in the factorial analysis; and, in order to assess reliability of each component, we used the item-total correlation and the ratio of success of the scale. Results: In the factorial analysis, 10 factors were retained, explaining 53.3% of the total variation. This result demonstrates the multidimensional structure of the instrument. The reliability assessment showed Cronbach’s alpha values ranging from 0.39 to 0.89. For the success of the scale most of the results (eight of nine attributes) were greater than 85%. Conclusions: The instrument is valid for the assessment of oral health services in Primary Health Care from the perspective of adult patients, as well as for monitoring and evaluation of oral health services in Primary Health Care attributes and comparative studies.


INTRODUCTION
Primary Health Care is the first level of any health system.It is defined by its core attributes: "First contact access", "Continuity", "Comprehensiveness" and "Coordination of care", and ancillary domains: "Family-centeredness", "Community orientation" and "Cultural competence" 1 .Even though these are closely related to the practice of services, such attributes can be assessed separately 2 .Therefore, a primary care service is best assessed, in the quality of a primary care provider, when it presents the four core attributes and improves its capability of interaction with families and communities while also showing the ancillary domains 2 .
The attributes of Primary Health Care are acknowledged in international literature as the structuring axes of Primary Health Care, being associated with the quality of services 3 , effectiveness 4 , and efficiency 5 of their interventions.An instrument that measures their presence and extension, which is the general version of Primary Care Assessment Tool, was developed 6,7 , with versions for adults, children, health professionals and managers 8 .The general version of Primary Care Assessment Tool is an instrument capable of assessing the quality of primary care provided, based on the measurement of the range of the Primary Health Care attributes, hence producing a score for each attribute, a score for core attributes, and a general score on the quality of care.It is useful to make a comparative assessment of the degree of success of services when it comes to achieving excellence in the practice of Primary Health Care.
offer oral health care as part of their primary care systems, including the United Kingdom and Brazil 12, 13 .In Brazil, one of the strategies used for the restructuring of health assistance was the Family Health Program, currently known as Family Health Strategy 14 .From 2000 onwards, Oral Health Teams comprising a dentist and an dental assistant, with or without the presence of an dental technician, were included as part of the Family Health Strategy, consolidating oral health care for the population in the Primary Health Care setting 15 .
The first Oral Health Teams were implemented in March 2001 in 32 municipalities and, in July 2015, 24,444 Oral Health Teams were part of the Family Health Strategy, covering 38% of the population of the country 16 .Even though the number of cities covered by Oral Health Teams has increased noticeably, there was an absence of studies in the literature or validated instruments to assess the presence and extent of Primary Health Care attributes in these dental health services 17 .The objective of this study was to test the factorial validity, internal consistency and reliability of the Primary Care Assessment Tool adapted to Oral Health, adult patient version, an instrument used to verify the presence and extent of attributes in Primary Health Care services.

PARTICIPANTS
We interviewed adult patients (18 years of age or more), users of Primary Care Dentistry Services, that used the service at least once during the 24 months prior to the study.Patients who reported never having used the Primary Health Care service for dental appointments, were excluded.

SAMPLE
A conglomerate random sample was used.The first step was to define the Health Units of the three types of service that would participate in the study.The inclusion criteria used were having an Oral Health Team with a dentist and dental assistant or with a dentist, a dental assistant and a dental technician in the service for at least two years; have a dental office in the service, and the dentist working full time in the service.From all the Primary Health Care services in Porto Alegre, only five of the Family Health Strategy met the inclusion criteria and were automatically included.Of the 78 Traditional Health Services and 11 Primary Health Care Services of the Grupo Hospitalar Conceição that met the inclusion criteria, 5 of each were drawn, using a list that was randomly generated on the computer.During data collection, one dentist working in a Family Health Strategy was relocated and the service was removed and replaced by Primary Health Care Services of the Grupo Hospitalar Conceição service selected randomly.
Then, based on the maps of the territories assigned to the services, census tracts were drawn.Within each census sector, homes were drawn, with a random start, and a patient who fulfilled the inclusion criteria was interviewed.If there was more than one adult who met the inclusion criteria, one of them was drawn.In order to achieve the goals of validation, the sample was estimated at 405 individuals (5 respondents per item of the Primary Care Assessment Tool) 8 .
Throughout the data collection period, there was an attempt to contact 1,943 homes.Of these, 81 were visited 3 times and no one was found at home, 6 refused to participate and 1,449 had never accessed Primary Health Care services for dental care.Thus, the total sample comprised 407 individuals interviewed.

VARIABLES AND SOURCES OF DATA/MEASUREMENT
The interviews were conducted, as part of home visits, by 17 properly trained dental students, aged between 20 and 25 years, using a structured questionnaire.In order to assess the training, a pilot field trip was held, in pairs, in which one researcher applied the questionnaire to a patient, while the other was assessing him.Data from the pilot study were not used.
The adult patients general version of Primary Care Assessment Tool 8 comprises 87 items distributed throughout core domains ("First contact access", "Continuity", "Comprehensiveness" and "Coordination of care") and ancillary domains ("Family-centeredness" and "Community orientation"), in addition to items of "Affiliation with the Health Service".It was initially developed to measure the presence and extension of Primary Health Care services attributes.The answers to the items are presented in Likert-like scales, with values that vary from 1 = "certainly not" to 4 = "absolutely", and the option 9 = "I don't know/don't remember".Based on these responses, it becomes possible to calculate a score for each attribute of Primary Health Care, a score for core attributes and an overall score 18 .The items of the attribute cultural competence were not included in this study, since it has not been validated in Brazil on the versions for adult patients, children and health professionals.
The score for the attributes is the average of the scores of the core attributes and the degree of affiliation to the service.The overall score, in addition to these, includes the REV BRAS EPIDEMIOL 2020; 23: E200076 ancillary domains.The values of the scores are standardized to fit into a scale that ranges from 0 to 10.A high score is regarded as one whose value is equal or greater than 6.6 6 .

ADAPTATION OF THE PRIMARY CARE ASSESSMENT TOOL: ADULT PATIENT, ORAL HEALTH VERSION
As a basis for the adaptation of the attributes "First contact access", "Continuity", "Coordination of care", "Family-centeredness" and "Community orientation", a mirror of the general version of Primary Care Assessment Tool Brazil was used, changing the term "doctor/nurse" by "dentist/dental service".The only exception was the attribute "Comprehensiveness", since this is referent to the range of services available, and these are different and specific to the field of oral health.To address the need for the assessment of the presence and extension of the attributes of Primary Health Care, also with regard to "Comprehensiveness" in Oral Health, version for patients, this attribute was adapted by Fontanive 19 .
For the adaptation of the "Comprehensiveness" in Oral Health, version for patients, Delphi Technique 20 was used to reach a consensus among experts (36 dentists who work in different areas of Primary Health Care, assistance, research and management) of the five regions of the Brazil considering which items should be taken regarding "Comprehensiveness" in Oral Health Care.This methodology consists of, a systematic and repetitive process that differs from the collective methods of consensus, which use a face-to-face interaction, decreasing the effect of contamination between participants and providing feedback based on the statistics generated in each round of participation.It took a total of 5 rounds to achieve the stability of the 35 items that are part of the attribute 19 .The extension of "Comprehensiveness" ended up with 26 items in the subdimension "Available Services" and 9 items in the subdimension "Services Provided".The adaptation of the instrument comprised 81 items distributed in the 9 subscales attributes of Primary Health Care, plus the questionnaire of affiliation to the service (three items).

STATISTICAL ANALYSIS
To check the factorial validity, an exploratory factorial analysis with extraction of the main components was carried out using the Equamax orthogonal rotation method.A factorial load of 0.35 was considered as minimally acceptable 21 .The theoretical distribution of the items in the attributes (or components) of Primary Dental Care was contemplated in the definition when establishing the factorial loads.
The reliability of this instrument was observed using the internal consistency and the ratio of success of the scale.In order to assess internal consistency of each component, the item-total correlation was used, with items with a value above 0.30 being considered adequate and Cronbach's alpha coefficient, equal to or greater than 0.70 21 .Regarding the ratio of success of the scale, we verified the proportion between the number of times that correlations between the items of an attribute were higher than the correlations of each of those items with other attributes, and the total of correlations carried in this attribute.High values for this measurement indicate a greater discrimination of the attribute.The data analysis for this paper was carried out using SAS software.Copyright © 2009, SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc., Cary, NC, USA.

ETHICAL ASPECTS
This study was approved by the Ethics Committee of the Municipal Government of Porto Alegre and of the Grupo Hospitalar Conceição, Register: 554, Number: 001.043203.10.8.The interviews with the patients of the services were carried out after reading and signing the Informed Consent Form (ICF) and were performed according to the ethical principles established in the Declaration of Helsinki and Brazilian legislation.

RESULTS
The sample comprised 407 individuals interviewed in the areas assigned to Primary Health Care services of Porto Alegre (four Family Health Strategy, five Traditional Primary Health Services and six Primary Health Care Services of the Grupo Hospitalar Conceição) who reported as their preferred health service or professional of reference one of these services/professionals providers of Primary Dental Care.In relation to the access Primary Health Care services for dental care, 23.3% (n = 95) of the users reported Family Health Strategy; 37.1%, Primary Health Care Services of the Grupo Hospitalar Conceição (n = 151); and 39.6% (n = 161) reported that they used Traditional Health Services.
In relation to internal consistency, taken into account the item-total correlation, the attribute "First contact access -Accessibility" showed correlation values below the ones considered adequate for the items: about phone support when the service is closed, on the care provided by someone from the service during the weekends or at night, about a waiting time greater than 30 minutes to be seen by the dentist, and about having to be absent from work or school to go to a health service (Supplementary Table 1).
For the attribute "Continuity", questions related to the care provided by the same dentist at all times and advise by phone provided by the dentist had correlation values below the ones considered adequate.The three items of the Coordination -Information systemsthe items on tooth restoration, tooth extraction and application of fluoride -in the attribute "Comprehensiveness -Available services" and the dental examination for "Comprehensivenessservices provided" also presented the correlation values below the ones considered adequate.On the other hand, for the components "First Contact Access -Utilization", "Coordination REV BRAS EPIDEMIOL 2020; 23: E200076 of care", "Family-centeredness" and "Community Orientation", all items showed proper correlation values (Supplementary Table 1).Three items were excluded after the factorial analysis, for the attribute "Comprehensiveness -Available services", because they didn't present an adequate value correlation.The factor loadings for each item are presented in Supplementary Table 1.
In the factorial analysis, 10 factors were retained explaining 53,3% of the total variation (Table 1).These factors accounted for more homogeneous attributes, which allow the measurement of the presence and extent of Primary Health Care (Table 1).Moreover, the internal consistency, using as a parameter the Cronbach's alpha coefficient, presented an insufficient coefficient value that was observed for the Coordination-Information Systems (alpha = 0.39) and moderate for Family-centeredness (alpha = 0.58); on the other hand, all others attributes presented an adequate Cronbach's alpha coefficient (Table 2).For the ratio of the success of the scale, all results were greater than 85%, with the exception of the component "Coordination -Information Systems" (ratio of success of the scale = 56%) (Table 2).

DISCUSSION
This study addressed the validity and reliability of the Primary Care Assessment Tool -Brazil -Adult Oral Health version, in Primary Health Care services of the municipality of Porto Alegre.The results show that the instrument was capable of measuring the presence and extent of the attributes of Primary Health Care, hence being able to evaluate the quality of services.The attributes of Primary Health Care were retained in the Adult Oral Health version of the instrument, therefore representing the first developed and validated instrument, and one that allows an evaluation of the degree and presence of its attributes.The use of this scale may represent an extremely valuable instrument, both for evaluative research and for the organization of dental health services in the context of Primary Health Care.
Regarding the factorial validity, its 6 attributes were captured by 10 factors, hence identifying the multidimensional nature of the construct.This phenomenon is also found in the versions for adult patients, children and health care professionals of the general version of Primary Care Assessment Tool validated in Brazil 2,9,11 .The attribute "First contact access" was captured by factor 1, which also retained items of the attribute "Continuity", and, in turn, the attribute "Continuity" also had its items found in factor 8, which also captured some items of the attribute "Coordination".Such interrelationship of items obtained from different attributes, retained by common factors, proves that the complex and dynamic nature of the services of Primary Health Care is represented by the instrument, since, according to the concept of Primary Health Care, the attributes can be didactically divided, but are closely related to the practice of services 11 .In relation to the three items that were excluded after the factorial analysis, for the attribute "Comprehensiveness -Available services", the item "emergency care in cases of pain/ bleeding or trauma/accident" is actually an item that represents the access to the service.The other, "Advice on nutrition and diet", is represented by the item "Guidelines on habits that can be harmful to the mouth and teeth", present in the attribute "Comprehensiveness -Services provided".Lastly, the item "treatment and guidelines on oral health for disabled patients" was also excluded since Primary Health Care services provide universal access and cases of serious disability, which cannot be treated by the Primary Health Care, should be referred to Specialized Dental Care Services 15 .
The results provided by the analysis of reliability indicate a consistency of the attributes "First contact access", "Continuity", "Comprehensiveness" and "Community Orientation".On the other hand, internal consistency was regarded as moderate for "Family-centeredness" and insufficient for "Coordination".Nonetheless, a decision was made to keep the items, since they represent the concept of Primary Health Care of Starfield 1 , and to maintain the comparability with other versions of Primary Care Assessment Tool.The results of the ratio of success of the scale indicate that each item is properly allocated in its conceptual attribute, given that they are more related to the items of their original component than to those of other attributes, hence confirming the reliability of the instrument.
The general adult version of Primary Care Assessment Tool is a Primary Health Care evaluation tool already used in many countries, such as Canada 22 , China 23 , Argentina 24 , the United States 6 , Hong Kong 25 , among others.Different versions are developed to contribute to its use in the local management of services 26 .A study that utilized the general adult version of Primary Care Assessment Tool to assess the impact of the general practitioner team service in community health centers found that primary care policies, which include the attributes of Primary Health Care, coordinating care and longterm provider-patient relationships, could contribute to improving the quality of care in Primary Health Care 27 .
A recent study, using the general version of Primary Care Assessment Tool for adults users and dental professionals from two public oral health services in primary care setting, found better scores ascribed to Primary Health Care attributes in the Family Health Strategy model, both among users and dentists in comparison to "conventional" healthcare services 17 .However, it is important to highlight that this study used the general Primary Care Assessment Tool with questions related to physician and nurse care, and not to dental care as proposed in the current study.Considering this, the validation of the instrument Primary Care Assessment Tool -Oral Health of Adults provides a tool that contemplates several aspects of oral health care and from its results can contribute to the orientation and elaboration of oral health public policies aimed at the organization and improvements in the performance of oral health in the Primary Health Care setting.
The Primary Care Assessment Tool -Oral Health of Adults captured the key attributes of Primary Health Care in oral health services and presented acceptable measures

Table 1 .
Distribution of items and explained variation of attributes in factors obtained by factorial analysis.

Table 2 .
Reliability measures for the seven attributes of Primary Care Assessment Tool -Oral Health Brazil.