Portuguese version of the EUROPEP questionnaire: contributions to the psychometric validation

ABSTRACT OBJECTIVE To assess the construct validity and reliability of the Portuguese version of the European Task Force on Patient Evaluation of General Practice Care questionnaire. METHODS We applied the Portuguese version of the European Task Force on Patient Evaluation of General Practice Care to 392 users of 20 Family Health Units from the North of Portugal. The validity of the construct was evaluated by exploratory factor analysis, with the Principal Axis Factoring method, by orthogonal rotation (varimax procedure), by the Kaiser normalization criteria (eigenvalue ≥ 1). The factorability of the data matrix was verified by the Kaiser-Meyer-Olkin and Bartlett’s sphericity test. We estimated the reliability by the indicator of internal consistency Cronbach’s alpha. To analyze the correlations between satisfaction and loyalty, we used the Pearson correlations. The predictor effect of satisfaction on loyalty was analyzed by simple linear regression. RESULTS Satisfaction presented five robust and well individualized dimensions – medical care, nursing care, clinical secretariat services, accessibility, and organization of services – with alpha values between 0.86 and 0.97, good levels of internal consistency. The loyalty showed alpha value of 0.72, considered a reasonable internal consistency. The satisfaction was predictive of loyalty. CONCLUSIONS The Portuguese European Task Force on Patient Evaluation of General Practice Care questionnaire is a robust and reliable instrument to measure the satisfaction and loyalty of users of the Family Health Units.


INTRODUCTION
Quality in health is currently a requirement for all those involved in health care. The quality indicators have been developed in the hospital context and their use quickly spread to primary health care. From an initial focus primarily on scientific and technical aspects, they evolved into the consideration of relational and satisfaction aspects of users 20 . Their usefulness can extend to the own goals of health services. In fact, the received feedback allows to introduce actions of correction and improvement to increase the levels of satisfaction of users 13,22 .
Satisfaction is an individual perception linked to the achievement of expectations or satisfaction of a need, by the individual or with intervention of others, whose evaluation is carried out from the person's perspective. Regarding primary health care, the satisfaction is equivalent to the well-being of users, manifested in their opinion about the quality of the obtained services 24 . It is a dynamic process influenced by individual, psychological, and sociocultural factors, among others. Because it is a multidimensional concept, the satisfaction concerning primary health care can feature an assessment of several aspects, such as accessibility, user-health professional relationship, infrastructure, and health service results 7,9 . Thus, to improve the provided services, it is essential to approach the maximum satisfaction of the requirements and needs of users. For this, it is necessary to listen to the views of users on how the services contributed to improve or solve their health state 22,24 . This concept of involvement and active participation of users in the provision of health care came from the USA, being later adopted in Europe. Health professionals and users perceive differently the results of the services provided in health. For this reason, the contribution of the user on the quality of services provided is essential. As active agents, the users share their negative and positive experiences and their needs and expectations, contributing to a response system that controls, identifies, and corrects the health system deficits 4,27 . Additionally, users' satisfaction increasingly occupies an important place in assessing the quality, aware of their valorization as active agents and consumers, because it was found that their satisfaction is related to therapeutic adherence and with results achieved and effectiveness of interventions, contributing to change health behaviors 1,3,19,23 .
Thus, it is necessary to engage the population in the construction of instruments for measuring satisfaction, which will allow understanding the expectations of users. The evaluation of services must be multidimensional, considering specific aspects that influence the overall satisfaction. It is necessary to appeal to strict qualitative studies and methods that allow the inclusion of the users' perspective in the construction processes and their results (questionnaires, tests, interview guides etc.). And the possible negative consequences must also be contemplated, as negative rating or expression of negative expectations, i.e., the respondents must have the opportunity and own space to express their ideas freely 18,21,24,27 .
The European Task Force on Patient Evaluation of General Practice Care (EUROPEP) questionnaire is widely used to evaluate the satisfaction of users with the primary health care, focusing on their perspective of final consumers and active agents in the therapeutic process. This instrument has been prepared using a comprehensive approach, which includes analyses of other investigations and theoretical concepts (literature analysis), interviews with users, and validation studies. The questionnaire aims to evaluate five dimensions: (1) doctor-patient relationship (e.g.: ease with which patients feel comfortable to tell their problems); (2) medical care (e.g.: clinical examination done by the doctor); (3) information and support (e.g.: explanation of the medicines, treatments, and examinations prescribed); (4) organization of services (e.g.: preparation on what to expect of hospital care, other experts, or other care workers); (5) accessibility (e.g.: ease to make an appointment in the family health units (FHU)) 2,8,10,11,[14][15][16]26 .
In this context, the aim of this study was to assess the construct validity and reliability of the Portuguese EUROPEP questionnaire. Roque H et al. DOI:10.1590/S1518-8787.2016050006259

METHODS
The study was conducted at the FHU from the districts of Braga, Porto, and Viana do Castelo, which have 172 of the 353 FHU operating in Portugal 25 . From the invited FHU, 20 agreed to participate in this study.
Participants filled out the questionnaires in person in the waiting rooms of the FHU, while waiting for the appointment or after they have been met. The filling occurred in the self and hetero-report format, the latter only on users with low level of education or no education. The collection took place between October 2013 and January 2014. 432 questionnaires were collected, from which 40 were eliminated because they had blank answers or maximum quotation on all items, totaling 392 valid questionnaires.
The 392 users evaluated were aged between 14 and 84 years old, with average of 41.7 years (SD = 15.9). The majority was female (77.3%), married (57.4%), and has already used the FHU services more than once (98.2%). Last year, the participants used the FHU services, on average, 4.7 times (SD = 5.0). The evaluated users were enrolled at the FHU between zero and seven years, with average of 3.9 years (SD = 1.6). Only 13.0% of participants responded in the hetero-report format.
We used the Portuguese version of the EUROPEP, adapted by Ferreira et al. 11 This instrument features 37 items that assess the level of satisfaction from the perspective of users. The items are rated on likert type scale of five points (0 = bad; 4 = excellent). The EUROPEP questionnaire contains, still, three items related to the loyalty of the user toward the FHU 5,17 , rated from "0 = disagree a lot" to "4 = agree a lot".
The construct validity was evaluated by exploratory factor analysis, with the Principal Axis Factoring method, by orthogonal rotation (varimax procedure), by the Kaiser normalization criteria (eigenvalue ≥ 1). The factorability of the data matrix was verified by the Kaiser-Meyer-Olkin and Bartlett's sphericity test. We estimated the reliability by the indicator of internal consistency Cronbach's alpha. We used the Pearson correlations to analyze the correlations between satisfaction and loyalty, and simple linear regression to verify the predictive effect of satisfaction on loyalty.
This project was approved by the Ethics Committee of the Regional Administration of North Health, Portugal. All participants signed the informed consent form in accordance with the Helsinki Declaration and the Convention of Oviedo.

Satisfaction
The correlation matrix analysis showed significant correlations (p < 0.001) with values between r = 0.24 and r = 0.78. There were no items excessively or perfectly correlated, and it may be affirmed that there are no problems of multicollinearity and singularity, respectively 12 . The analysis of commonalities showed values between 0.46 and 0.87, with average of 0.65, reflecting that 65.0% of the variance associated to items is common or shared. We verified factorability in the data matrix, because the value of Kaiser-Meyer-Olkin (KMO) was greater than 0.60 (KMO = 0.96) and the Bartlett's sphericity test was significant (p < 0.001).
The internal consistency, estimated by the Cronbach' alpha, presented high value (0.97) to the full scale, and no items changed, significantly, this value.
In Table 1, we present the values of the factor saturation of each item, percentage of total variance, internal consistency of each factor, and the commonalities (h 2 ) regarding the satisfaction of users.
For the retention of items on each scale, the following criteria have been fixed: (1) saturation ≥ 0.40 of each item in the hypothetical factor and only in one single factor; (2) the final factor solution explaining at least 50.0% of the total variance; (3) consistency between the factor solution and items that constitute each factor; and (4) each factor being represented by at least three items 6,12 . We removed two items in the sequence of the retention criteria: "The respect with which you were treated and how your privacy was maintained" and "The quickness with which the urgent health problems have been resolved", because they had a factor loading < 0.40 and similar factor loadings in more than one factor.
In addition to the above procedure, we also performed the exploratory factor analysis by principal component method and maximum likelihood with oblique rotation. These showed similar results, with differences only in the order of the factors.

Loyalty
The correlation matrix analysis showed significant correlations (p < 0.001), with values between r = 0.35 and r = 0.62. There were no items excessively or perfectly correlated, and it may be affirmed that there are no problems of multicollinearity and singularity, respectively 12 .
The analysis of commonalities showed values between 0.25 and 0.76, with average of 0.50, reflecting that 50.0% of the variance associated to items is common or shared. We verified factorability in the data matrix, because the value of KMO was greater than 0.60 (KMO = 0.63) and the Bartlett's sphericity test was significant (p < 0.001).
The exploratory factor analysis extracted one factor -loyalty -that explains 50.2% of the variance with α = 0.72.
In Table 2, we present the values of the factor saturation of each item, percentage of total variance, internal consistency of the factor, and the commonalities (h 2 ) regarding the loyalty. For the retention of the items, we fixed the same criteria used in satisfaction. All items fulfilled the criteria.

Relation Satisfaction/Loyalty
The Pearson correlations show significant positive associations (p < 0.01) between satisfaction and its dimensions and loyalty. The highest value was in overall satisfaction, showing that high values of satisfaction are associated with high values of loyalty (r = 0.40; p < 0.01).
The simple linear regression showed that the overall satisfaction of users explains 16.0% of the variance of loyalty (R 2 aj = 0.16; p < 0.001) (F 1;390 = 74.07; p < 0.001), and, the higher the levels of overall satisfaction, the higher the levels of loyalty with the FHU.

DISCUSSION
The results show that the items are good indicators of the constructs intended to measure -satisfaction and loyalty -and that the factors are properly individualized. Regarding satisfaction, the results show five robust dimensions, which allow a fairly broad understanding of the users' satisfaction. Some of these dimensions -medical care, accessibility, and organization of services -find parallels in other validations of the EUROPEP questionnaire 2,8 . However, the remaining dimensions are not found in other studies, which can be attributed to the fact that the Portuguese version include more items than the original one. This increase of items meets the reality of the Portuguese primary health care. As for reliability, this instrument features good internal consistency. And, still, there were no problems about the singularity and multicollinearity.
Regarding loyalty, this dimension presented acceptable internal consistency and the items that constituted it compose a robust measure of the construct. Loyalty was positively associated with overall satisfaction and all its dimensions. And, as verified in other studies 5,17 , satisfaction is a predictor of loyalty and explains 16.0% of its variance.
In short, the Portuguese EUROPEP questionnaire is a robust and reliable instrument to measure the satisfaction of users by their own perspective.
This study had as its main limitation the apparent sample homogeneity -all FHU belong to the North of the Country -and future studies could appeal to a more heterogeneous sample, involving FHU from the North to the South of the Country. The results show that this is a robust instrument; but it contains many items, being a long questionnaire to apply. A shortened version would be more practical and economical to use in the FHU.