Satisfaction measurement instruments for healthcare service users: a systematic review

Instrumentos para mensuração de satisfação de usuários de serviços de saúde: uma revisão sistemática

Los instrumentos para medir la satisfacción del usuario de los servicios de salud: una revisión sistemática

Abstracts

Patient satisfaction surveys can be an interesting way to improve quality and discuss the concept of patient-centered care. This study aimed to conduct a systematic review of the validated patient satisfaction measurement instruments applied in healthcare. The systematic review searched the MEDLINE/PubMed, LILACS, SciELO, Scopus and Web of Knowledge. The search strategy used the terms: "Patient Satisfaction" AND "Patient centered care" AND "Healthcare survey OR Satisfaction questionnaire" AND "Psychometric properties". 37 studies were included and almost all studies showed that satisfaction is a multidimensional construct. In these studies, 34 different instruments were used and most surveys contained the dimension patient-healthcare professional interactions, physical environment and management process. The COSMIN score for methodological quality showed that most of them scored a good or fair average. We can conclude that there is not a gold standard instrument for patient satisfaction assessment but some dimensions are essential for this construct.

Quality of Health Care; Patient-Centered Care; Patient Satisfaction; Psychometrics

Questionários para satisfação dos pacientes podem ser uma boa maneira para melhorar a qualidade e discutir aspectos do cuidado centrado no paciente. O objetivo foi conduzir uma revisão sistemática referente a instrumentos já validados para mensuração de satisfação de pacientes em serviços de saúde. A revisão sistemática realizou buscas nas bases MEDLINE/PubMed, LILACS, SciELO, Scopus and Web of Knowledge. A estratégia de busca foi: "Patient Satisfaction" AND "Patient centered care" AND "Healthcare survey OR Satisfaction questionnaire" AND "Psychometric properties". Trinta e sete estudos foram incluídos e quase todos demonstraram que a satisfação é um construto multidimensional. Foram encontrados 34 diferentes instrumentos e a maioria deles possuía as dimensões paciente-profissional da saúde, ambiente físico e processos gerenciais. A pontuação COSMIN em qualidade metodológica dos artigos indicou que a maioria apresenta um valor bom ou médio neste aspecto. Não existe um padrão ouro para mensuração da satisfação do paciente, mas algumas dimensões são essenciais para este construto.

Qualidade da Assistência à Saúde; Assistência Centrada no Paciente; Satisfação do Paciente; Psicometria

Los cuestionarios para la satisfacción del paciente pueden ser una buena opción de mejorar la calidad y discutir la atención centrada en el paciente. El objetivo era llevar a cabo una revisión sistemática, en relación con los instrumentos previamente validados para medir la satisfacción de los pacientes. La revisión sistemática realizó búsquedas en las bases de datos MEDLINE/PubMed, LILACS, SciELO, Scopus y Web of Knowledge. La estrategia de búsqueda fue: "Patient Satisfaction" AND "Patient centered care" AND "Healthcare survey OR Satisfaction questionnaire" AND "Psychometric properties". Se incluyeron treinta y siete estudios y casi todos han demostrado que la satisfacción es multidimensional. Se encontraron 34 instrumentos y la mayoría contaba con las dimensiones: interacción paciente-profesional de la salud, entorno físico y procesos administrativos. La puntuación COSMIN en calidad metodológica indica que la mayoría de los estudios tiene un valor bueno o medio. No existe un estándar clave para la medición de la satisfacción del paciente, pero algunas dimensiones son esenciales para este constructo.

Calidad de la Atención de Salud; Atención Dirigida al Paciente; Satisfacción del Paciente; Psicometría

Background

In the past decade, several programs for quality improvement were developed in many health services and systems ^{1 , 2}. Leatherman et al. ² reported that there are publications on this subject in various regions of the world, including India, South America, Asia and North America. More specifically, on the topic of how to increase quality, many institutions have been trying to establish guidelines and parameters for quality assessment and improvement.

Patient-centered care has been accepted as one of the pillars for health services to achieve quality of care ³. This issue must involve the patients' needs and preferences in decision-making about care, as well as previous experience within these services. Thus, the literature shows a trend towards studying what actually is relevant for the patient's experience within healthcare ^{3 , 4}.

Considering this goal, patient satisfaction studies can be a useful way of measuring a part of this complex construct, because it is an important concern related to quality of healthcare and patient centeredness, both in the public and private sectors ⁵. The concept of satisfaction is well known and applied in management and especially in marketing. Evrard ⁶ defines satisfaction as a psychological state, appearing after consumption (of a service) and compared to an initial baseline (services' expectation). Olivier ⁷ and Fournier & Mick ⁸ expressed satisfaction as the subjective comparison between expectations and perceptions of service performance. The satisfaction is subjective, yet measurable, provided that the consumer has an initial point of reference. Satisfaction on healthcare must consider the "emotional" aspect related to a medical action that may have a significant impact on the patients' own health. In this way, understanding all dimensions and factors related to patient satisfaction is important in order to improve the quality of healthcare ^{6 , 7 , 8}.

Recent publications on quality of care have discussed the challenge of improving quality ⁹ and most of them have stressed patient-centered care as one of the core goals ^{10 , 11}. In this context, the user satisfaction construct has already been scrutinized by many different study designs in different areas ^{12 , 13 , 14}. The validation and cross-cultural adaptation of surveys are the most common study designs, but, up to now, this area has needed more investigation ^{15 , 16}.

Choosing the appropriate instrument for a patient-related outcome measurement is difficult; it depends on the proposed use and can probably have an impact on the accuracy of the results. The discussion about which are the essential psychometric properties for healthcare instruments validation is still lacking in the literature ^{14 , 15 , 16 , 17}. It is therefore quite important to know if there is some reference instrument to measure patient satisfaction with healthcare services and if there is a similar scope in the structure of these instruments after the validation process. Given this concern, this study aims to perform a systematic review of validation studies of patient's satisfaction measurement instruments applied in healthcare services. It thus will seek to identify the kinds of tools and psychometric properties analyzed and the quality of the instruments.

Methods

Study design and information sources

A systematic review of patient satisfaction surveys was undertaken. Electronic databases were chosen based on their indexed journals profile. MEDLINE/PubMed, LILACS (Virtual Health Library; Bireme/PAHO), SciELO (Scientific Electronic Library Online), Scopus and Web of Knowledge were used as reference material in this study. LILACS and SciELO were included due the fact that both have Latin American studies in their collections. All databases were screened in December, 2013. The search strategy was oriented only for validation studies of patient satisfaction instruments and for this reason other terms that could be relevant for quality of care or other pillars of this complex construct were not included. In this way, terms that could have other concepts inputted were not included, such as patient preference, patient needs and consumer satisfaction. In addition, the term "validation study" was not used, because it was only introduced as a topic in MEDLINE in 2008. After testing several words and MeSH terms, the search strategy used the terms: "Patient Satisfaction[MeSH Terms]" AND "Patient centered care[MeSH Terms]" AND "Healthcare survey OR Satisfaction questionnaire" AND "Psychometric properties".

Inclusion and exclusion criteria

The research assumed the following eligibility criteria: (a) only studies published in English, Spanish, French and Portuguese; (b) studies published between 2002 and 2013; (c) studies which have used a patient satisfaction tool to investigate only the opinion of health service users; (d) studies that had evaluated and described at least one step of the COSMIN check list ¹⁷ were considered. The temporal limitation was justified in the report Crossing the Quality Chasm: A New Health System for the 21st Century ³, published in 2001, where patient-centered care as a dimension of quality of care has become more widespread and highlighted a renewed place for the patient satisfaction. No types of health service were excluded, in order to guarantee different patient profiles. The definition of health service used was "any center that provides diagnosis, treatment or prevention for more than one kind of disease, illness or injury in the human body". However, instruments used to measure patient satisfaction regarding just one specific treatment or illness or clinical condition were excluded, such as diabetes or chemotherapy, for instance. Systematic review and metanalysis studies were excluded.

Study selection

Two independent reviewers applied the inclusion and exclusion criteria to the selected articles. All titles and abstracts were screened after the removal of duplicates and based on this examination, a list of references for full text reading was prepared. After this, both reviewers selected the papers to be included in the next step (qualitative synthesis of the studies). The search results and papers screened were discussed by e-mail and also during some meetings. Any disagreements on the records included in this review were submitted to a third reviewer.

Data extraction

A form was prepared to collect priority information; the following items were extracted from each study: year, authors, study design (instrument validation, adaptation - changes suggested from the original, and cross-cultural adaptation), healthcare setting (hospital, clinic), satisfaction instrument utilized, dimensions of the instrument and psychometric properties analyzed.

Evaluation of methodological quality

It is necessary to emphasize that the methodological quality of the study is as important as the quality of a measurement instrument. The psychometric properties of an instrument are merely one part of the validation process and, thus, other steps must be demonstrated during this process ^{14 , 15}. The COSMIN checklist is one of the guidelines to evaluate the methodological quality of measurement instruments in different areas, and many systematic reviews have already proved its usefulness ^{17 , 18 , 19}. Considering this, the present study chose the COSMIN checklist to evaluate the methodological quality of the studies included in the reviewing process. This checklist is a standardized tool to assess studies on measurement properties. It contains nine boxes and each one has a different measurement property, with 5-18 items per box on design aspects and statistical methods. The boxes refer to: internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, and responsiveness ¹⁷.

Each measurement property is evaluated separately. This means that if multiple measurement properties are assessed in one study, several COSMIN boxes need to be completed. The content evaluated in each box can be judged as: "excellent"; "good"; "fair"; "poor". The methodological quality score per box is obtained by taking the lowest rating of any item in the box ("worse score counts") ¹⁷. The checklist includes one additional box for studies that used item response theory (IRT). Two additional boxes are recommended (interpretability and generalizability of results), but, as they do not provide a score, they were not used here. Here the COSMIN checklist was applied for each selected study by the principal reviewer and all doubts were discussed.

Results

The initial search strategy found 1,398 articles, and after exclusion of duplicates, the total was 1,286. After the inclusion and exclusion criteria were applied, there were only 49 articles remaining (Figure 1). All these studies were retrieved for reading and 12 were excluded in this phase. By the end of this phase, 37 articles were included in the qualitative synthesis of the reviewing process (Figure 1).

Figure 1
Flow of information through the different phases of the systematic review.

General data

The results showed that satisfaction measurement instruments have been applied all around the world (Europe, Asia, North America, Australia, and Africa). The sample size in the studies screened showed a wide variation. The number of people included ranged from 80 ²⁰ to 7,093 ²¹; almost all studies collected some kind of cultural and socio demographic variables of the patients (age, gender, family income, educational level and health area of treatment).

It was difficult to identify which health services were public or private, most studies did not clearly state this ^{20 , 21}. Only five studies ^{22 , 23 , 24 , 25 , 26} clearly stated the ownership of the service studied. Service complexity level showed major differences because three studies utilized university hospitals to measure patient satisfaction, 10 used only clinics and another 20 studies measured patient satisfaction in general hospitals (Table 1). There are many different instruments being used, whether in hospitals or in clinics.

Table 1
Characteristics of the studies.

Study (year)	N	Health care setting	Study design	Instrument used	Construct dimensions (*)	Measurement proprieties **	Psychometric values	COSMIN scores **
Meehan et al. 45 (2002)	356	Hospital	Validation	Inpatient Evaluation of Service Questionnaire (IESQ)	Multidimensional (3)	1-Internal consistency 2-Structural validity 3-Hypotheses testing 4-Content validity	α > 0.86; correlation with overall satisfaction measure (r = 0.78)	1-Fair 2-Fair 3-Fair 4-Excellent
Hendriks et al. 50 (2004)	728	Hospital	Adaptation	Satisfaction with Hospital Care Questionnaire	Multidimensional (12)	1-Internal consistency 2-Structural validity	α > 0.6; inter-scales correlations (r = 0.3-0.7)	1-Fair 2-Fair
Scholle et al. 31 (2004)	1,202	Clinic	Validation	Primary Care Satisfaction Survey for Women (PCSSW)	Multidimensional (3)	1-Structural validity 2-Internal consistency 3-Hypotheses testing 4-Content validity	α > 0.80; correlation between scales (r = 0.65-0.69); correlation with other instruments (r > 0.42)	1-Good 2-Good 3-Good 4-Excellent
Lam et al. 34 (2005)	711	Hospital	Trans cultural adaptation	Medical Interview Satisfaction Scale (C-MISS)	Bidimensional	1-Internal consistency 2-Structural validity 3-Cross-culrural validity 4-Hypotheses testing 5-Reliability	α > 0.84; inter-item and item-total correlations (r > 0.3) ; sub-scales to total correlation (r > 0.87); inter-scales (r = 0.58)	1-Good 2-Good 3-Fair 4-Fair 5-Fair
Gonzalez et al. 42 (2005)	1,910	Hospital	Validation	In-patient Satisfaction Questionnaire	Multidimensional (6)	1-Internal consistency 2-Structural validity 3-Hypotheses testing 4-Content validity	α > 0.60; inter-scales correlations (r = 0.16-0.62)	1-Excellent 2-Excellent 3-Fair 4-Excellent
Wilde & Hendriks 36 (2005)	262	Clinic	Adaptation	Client Satisfaction Questionnaire (CSQ)	Unidimensional	1-Structural validity 2-Internal consistency 3-Hypotheses testing	α = 0.92;  correlations between other instruments (r > 0.46)	1-Good 2-Excellent 3-Good
Gerkensmeyer & Austin 22 (2005)	278	Clinic Children Mental Health Service	Validation	Parent Satisfaction Scale (PSS)	Unidimensional	1-Internal consistency 2-Structural validity 3-Hypotheses testing	α > 0.70; correlation with other instruments (r = 0.86)	1-Fair 2-Fair 3-Fair
Otani et al. 53 (2005)	8,465	Hospital	Adaptation	Patient Satisfaction Survey	Multidimensional (3)	1-Internal consistency 2-Structural validity 3-Measurement error	α > 0.8; RMSEA = 0.12	1-Fair 2-Fair 3-Fair
Poinsot et al. 40 (2006)	416	Hospital (Oncology Center)	Validation	Out-Patsat35	Bidimensional	1-Structural validity 2-Internal consistency	α > 0.90; inter-tem correlations (r = 0.33-0.91); correlation with other instrument (r = 0.38)	1-Poor 2-Poor
Tso et al. 25 (2006)	344	Hospital	Validation	Concise Outpatient Department User Satisfaction Scale	Bidimensional	1-Internal consistency 2-Structural validity 3-Hypotheses testing	α > 0.90; Guttman split-half reliability (0.93); Spearman-Brown reliability (0.94)	1-Fair 2-Fair 3-Fair
Gagnon et al. 47 (2006)	873	Clinic	Validation	Health Care Satisfaction Questionnaire (HCSQ)	Multidimensional (3)	1-Structural validity 2-Internal  consistency 3-Content validity 4-Reliability	α > 0,74; KMO (0.95); CFI = 0.93; NFI = 88; AGFI = 0.87; RMSEA = 0.6; correlations between factor (r > 0.6); ICC (> 0.45)	1-Excellent 2-Excellent 3-Excellent 4-Fair
Campbell et al. 49 (2007)	1,250	Out-of-our health service	Validation	Out-of-hours Patient Questionnaire (OPQ)	Multidimensional (8)	1-Internal consistency 2-Structural validity 3-Reliability 4-Content validity	α = 0.70, inter-item and total item correlation                   (r > 0.20)	1-Fair 2-Fair 3-Fair 4-Fair
Moret et al. 28 (2007)	2,684	Hospital	Adptation	Echelle de Qualité des Soins en Hospitalisation     (EQS-H scale)	Bidimensional	1-Internal consistency 2-Structural validity 3-Content validity                          4-Measurement error	α > 0.80; correlation inter-item (r > 0.40) and item-total (r < 0.40); RMSEA = 0.063	1-Good 2-Good 3-Fair 4-Good
Granado de la Orden et al. 44 (2007)	651	University hospital	Validation	Satisfacción de los Usuarios de las Consultas Externas (SUCE)	Bidimensional	1-Internal consistency 2-Structural validity 3-Hypotheses testing 4-Content validity	α > 0.90; correlation with other instrument (r = 0.85); item-total correlation (r = 0.52-0.73)	1-Good 2-Good 3-Fair 4-Excellent
Shea et al. 37 (2008)	3,215	Private clinic	Adaptation	The Consumer Assessment of Healthcare Providers and Systems Surveys (CAHPS)	Multidimensional (7)	1-Internal consistency 2-Hypotheses testing 3-Content validity	α = 0.39-0.90	1-Fair 2-Poor 3-Fair
Kristensson & Ekwall 27 (2008)	128	Hospital	Adaptation	Consumer Emergency Care Satisfaction Scale (CECSS)	Multidimensional (3)	1-Structural validity 2-Internal consistency	α > 0.85	1-Fair 2-Fair
Clark et al. 51 (2008)	1,415	Clinic and hospital     (mental health services)	Validation	Consumer Perceptions of Care (CPC)	Multidimensional (4)	1-Internal consistency 2-Structural validity 3-Reliability 4-Hypotheses testing 5-Content validity	α > 0.70; correlations between instruments and different moments (r = 0.34-0.56)	1-Good 2-Good 3-Fair 4-Fair 5-Good
Coluccia et al. 39 (2009)	1,804	Hospital	Adaptation	Questionnaire of Quality Perception (QQP)	Multidimensional (4)	1-Structural validity	Correlations between factors 0.56-0.87; GFI = 0.92; RMSEA = 0.18; CFI = 0.96; NFI = 0.96	1-Excellent
Arraras et al 20 (2009)	80	Hospital	Adaptation	European Organization for Research and Treatment of Cancer In-patient Questionnaire (EORTC IN-PATSAT 32)	Multidimensional (13)	1-Internal consistency 2-Hypotheses testing	α > 0.70; correlations with other instruments > 0.40	1-Fair 2-Fair
Orden et al. 24 (2010)	296	Hospital emergency	Validation	Satisfacción del Usuario con el Área de Urgencia Hospitalaria (SUAUH)	Bidimensional	1-Internal consistency 2-Structural validity 3-Hypotheses testing 4-Content validity	α > 0.83; correlation with other instrument and global satisfaction measures (r = 0.37-0.51); item-total correlation (r = 0.56-0.71)	1-Fair 2-Fair 3-Fair 4-Excellent
Danielsen et al. 38 (2010)	225	Clinic	Validation	Users’ experiences and satisfaction with casualty clinics	Multidimensional (4)	1-Internal consistency	α > 0.82	1-Fair
Ekwall & Davis 26 (2010)	157	Hospital	Transcultural adaptation	Consumer Emergency Care Satisfaction Scale (CECSS)	Multidimensional (3)	1-Structural validity 2-Internal consistency 3-Cross-cultural validity	α > 0.77	1-Fair 2-Fair 3-Fair
Charalambous 33 (2010)	194	Hospital (oncology center)	Transcultural adaptation	Risser Patient Satisfaction Scale (RPSS)	Multidimensional (3)	1-Structural validity 2-Internal consistency 3-Reliability 4-Cross-cultural validity	α = 0.89; Kappa = 0.84; inter-item (r = 0.45) and item-total correlations (r = 0.51); sub-scale correlation (r = 0.55-0.70)	1-Fair 2-Fair 3-Fair 4-Fair
Erci & Ciftcioglu 30 (2010)	381	Clinic (primary care)	Transcultural adaptation	Primary Care Satisfaction Survey for Women (PCSSW)	Multidimensional (3)	1-Internal consistency 2-Structural validity 3-Content validity 4-Cross-cultural validity 5-Reliability	α > 0.80; item-total correlations (r = 0.43-0.73); test-retest correlation (r = 0.85)	1-Fair 2-Fair 3-Fair 4-Fair 5-Fair
Soufi et al. 29 (2010)	214	University hospital	Adaptation	Echelle de Qualité des Soins en Hospitalisation     (EQS-H scale)	Bidimensional	1-Internal consistency 2-Structural validity 3-Content validity 4-Cross-cultural validity	α > 0.88; GFI = 0.96; RMSEA = 0.090; CFI = 0.93	1-Fair 2-Fair 3-Fair 4-Fair
Schröder et al. 46 (2011)	1,340	Hospital	Validation	Quality in Psychiatric Care – Outpatient (QPC-OP)	Multidimensional (7)	1-Internal consistency 2-Structural validity	α = 0.66; CFI = 1.0, RMSEA = 0.036; SRMR = 0.053; inter-scales correlations (r = 0.35-0.67)	1-Excellent 2-Excellent
Webster et al. 41 (2011)	486	Hospital	Validation	Patient Assessment of Healthcare for Inpatient Care (I-PAHC); Patient Assessment of Healthcare for Outpatient Care (O-PAHC)	Multidimensional (5)	1-Internal consistency 2-Structural validity 3-Content validity	α > 0.70; inter-scales and item correlations (r = 0.2-0.6)	1-Good 2-Good 3-Poor
Sjetne et al. 48 (2011)	1,324	Hospital	Validation	Generic Short Patient Experiences Questionnaire         (GS-PEQ)	Multidimensional (7)	1-Content validity	Theorical approach	-
Latour et al. 52 (2012)	441	University hospital (neonatal intensive care)	Validation	Empowerment of Parents in the Intensive Care-Neonatology (EMPATHIC-N)	Multidimensional (5)	1-Internal consistency 2-Structural validity 3-Reliability 4-Hypotheses testing 5-Content validity	α > 0.82; correlation with other instrument and global satisfaction measures (r = 0.37-0.51)	1-Good 2-Good 3-Good 4-Fair 5-Good
Evans et al. 23 (2012)	397	Clinic (mental health)	Validation	Views on Inpatient Care (VOICE)	Multidimensional (7)	1-Structural validity 2-Internal consistency 3-Content validity 4-Hypotheses testing 5-Reliability validity	α > 0.92; correlation with other instrument (r = 0.82); Kappa (ρ = 0.88)	1-Poor 2-Poor 3-Fair 4-Fair 5-Fair
Manaf et al. 43 (2012)	424	Clinic	Validation	Oupatient satisfaction instrument	Bidimensional	1-Internal consistency 2-Structural validity 3-Hypotheses testing 4-Content validity	α > 0.94 ; correlation with other  instruments and overall satisfaction measure (r = 0.59-0.62)	1-Excelent 2-Excellent 3-Good 4-Excellent
Milutinović et al. 32 (2012)	235	Hospital	Transcultural adaptation	Patient Satisfaction Nursing Care Quality Questionnaire	Unidimensional	1-Structural validity 2-Internal consistency 3-Cross-cultural validity	α > 0.90; inter-item correlation and item total correlation (r =0,52-0.92); GFI (AUC = 0.82)	1-Good 2-Good 3-Good
Ottonello et al. 21 (2012)	7,093	Hospital (emergency care)	Adaptation	SAT-16	Bidimensional	1-Structural validity 2-Internal consistency	α > 0.99; correlation between item (r < 0.30); Rasch reliability values (Item separation index: 18.1 and 20.7; item separation reliability: 1; person separation index: 0.70)	1-Excellent 2-Excellent
Castillo et al. 54 (2012)	483	Hospital	Adaptation	Satisfaction Scale with Primary Health Care Services	Multidimensional (4)	1-Structural validity 2-Internal consistency	α > 0.75; RMSEA = 0,09; GFI = 0.92; AGFI = 0.87; NFI = 0.94; CFI = 0.95	1-Fair 2-Fair
Dyer et al. 55 (2012)	21,318	Hospital/Clinic	Adaptation	Consumer Assessment of Healthcare Providers and Systems (CAHPS-CG)	Multidimensional (3)	1-Structural validity 2-Internal consistency 3-Reliabilty	α > 0.75; RMSEA = 0.05; SRMR = 0.04; CFI = 0.97; ICC = 0.01-0.11	1-Good 2-Good 3-Good
Vanti et al. 35 (2013)	317	Hospital	Transcultural adaptation	Physical Therapy Patient Satisfaction Questionnaire    (PTPSQ-I)	Multidimensional (3)	1-Structural validity 2-Internal consistency 3-Cross-cultural validity 4-Hypotheses testing	α > 0.90; correlations between item and global perceive measures (r = 0.1-0.42)	1-Fair 2-Fair 3-Good 4-Fair
Vanti et al. 56 (2013)	354	Hospital	Transcultural adaptation	Physical Therapy Patient Satisfaction Questionnaire    (PTPSQ-I)	Multidimensional (4)	1-Structural validity 2-Internal consistency 3-Cross-cultural validity 4-Reliability 5-Hypotheses testing	α > 0.75; ICC = 0.62-0.93; correlations between item and global perceive measures (r = 0.28-0.42)	1-Fair 2-Fair 3-Good 4-Fair 5-Fair

• ^*
  Number of dimensions in the instrument (construct representation);
• ^**
  The methodological classification scores for each item of the Measurement Properties column are demonstrated by the respective number in the COSMIN column. a: Cronbach's alpha; AGFI: adjusted goodness of index; AUC: area under the ROC curve; CFI: comparative fit index; GFI: goodness of fit index; ICC: intra-class correlation; NFI: normed fit index; RMSEA: root mean square error of approximation; SRMR: standardized root mean residual.

34 different instruments for satisfaction measurement were found and few of them were utilized by more than one study: Consumer Emergency Care Satisfaction Scale (CECSS) ^{26 , 27} - twice; Echelle de Qualité des Soins en Hospitalisation (EQS-H scale) ^{28 , 29} - twice; Primary Care Satisfaction Survey for Women (PCSSW) ^{30 , 31} - twice (Table 1). It seems that there is not a gold standard or a standardized instrument for satisfaction measurements in health services.

Methodological quality

The most frequent study design observed was instrument validation and only six cross-cultural adaptations were found ^{26 , 30 , 32 , 33 , 34 , 35} (Table 1).

An analysis of the psychometric properties indicates that most studies used the classical test theory (CTT) to evaluate the instruments' structure. Only one paper used the IRT ¹⁹. The main properties assessed were structural validity and internal consistency. Just five papers were found not to approach these two proprieties together ^{18 , 36 , 37 , 38 , 39}. Content validity (face validity), hypotheses testing, reliability (test-retest) and cross-cultural validity were other properties analyzed in many studies (Table 1).

The COSMIN checklist scores demonstrated that most papers achieved a medium score ("good" or "fair") on the methodology quality of the psychometric analysis. Only four studies demonstrated low scores ("poor") ^{23 , 37 , 40 , 41} and eleven studies had at least one property categorized as excellent ^{21 , 24 , 31 , 36 , 39 , 42 , 43 , 44 , 45 , 46 , 47}. 37 studies ^{20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56} presented the Cronbach α value as an internal consistence measure (ranged from 0.39 to 0.99); most of them showed correlations between items, sub-scales and other satisfaction measures (Table 1). The main statistical methods utilized were the exploratory factorial analysis (EFA) and confirmatory factorial analysis (CFA) to evaluate the instruments' structure.

Satisfaction construct

One can argue that there is almost a consensus between studies on the multidimensional characteristic of the satisfaction construct (Table 1). Only three studies showed a unidimensional structure for the tested instrument ^{22 , 32 , 48}. However, there is not a clear pattern of what the essential dimensions (factors) that must be included in a satisfaction measurement are. The number of dimension/factors in the instruments scanned ranged from 1 to 13.

It was also possible to observe a trend showing that more than one dimension should be used to develop or adapt an instrument (Table 1). The main dimensions were: patient-healthcare professional interactions, physical environment and internal management process ^{22 , 23 , 32 , 48}. Only services with some different characteristics (home care, out-of-hours, psychiatric and pediatric services, and more multidisciplinary complex services) usually utilized other specific dimensions ^{22 , 23 , 49}.

Discussion

Considering the focus on patient-centered care has been proposed all around the world, the development or adaption of satisfaction surveys has been the most common method to evaluate patient satisfaction. However, the very high specificity of the different types of care makes the development of a single measurement scale transposed into all services more complex. The findings of this revision agree with previous studies ^{11 , 12 , 13 , 57 , 58} showing that most surveys demonstrated "user satisfaction in health care services" as a multifactorial construct, but until now there is not a reference or gold standard instrument for the measurement of such a concept.

Previously, Boyer et al. ¹¹ showed, in their systematic review about patient satisfaction surveys in psychiatric services, a wide variation in the number of dimensions of the instruments scanned. Even though our review had screened instruments applied to different healthcare settings, the results support these findings, and the number of dimensions in the instruments scanned ranged from 1 to 13.

A good theory-based approach is the first step to define any construct. When it comes to user satisfaction, it has already been shown that the essential aspects are: patient-professional interactions, physical environment and internal management processes ^{17 , 18 , 22 , 23}. However, services with some different characteristics (home care, out-of-hours, psychiatric and pediatric services, and more multidisciplinary complex services) usually utilized other specific elements, such as phone contact or clear information to the caregiver ^{18 , 24 , 25 , 26}.

The complexity of this construct can make it difficult to develop measurement instruments and to define their real contribution for healthcare quality and management. It must be understood that the concept of satisfaction can change across many cultures, and different variables may be related depending on patient profile or service ^{57 , 59 , 60}. Almost all clinicians and patients agree that clinical skill, rapport and health-related communication behavior are key elements for quality and satisfaction ^{58 , 60}. However, our findings demonstrated that all multidimensional instruments utilized sub-scales that were linked with these key elements.

The methodological quality of the questionnaires and their validation process are important topics that should be highlighted ⁶¹. All instruments found were developed in the English language and there were only six cross-cultural adaptation papers ^{26 , 30 , 32 , 33 , 34 , 35}. The adaptation process to other languages and cultures is not simple and the scrutiny made by different researches on distinct sociocultural contexts would help to develop a reliable instrument ^{61 , 62 , 63}.

The psychometric properties of the questionnaires are just one point of the validation or adaptation process and other steps should be taken ^{62 , 63}. Our results showed that structural validity and internal consistency were the main measurement properties described by the studies, but none of them had already undergone all other steps proposed by the COSMIN checklist for validation: reliability, measurement error, content validity, hypotheses testing, criterion validity, cross-cultural validity and responsiveness ^{14 , 15 , 16 , 17}.

Discussion about what drives users' satisfaction is necessary to guide the development of new surveys and classic confirmatory analysis. Assessing this issue is quite important to choose an instrument that has already been scrutinized by several authors and crossed many important validation steps ^{15 , 61}. However, this approach has still proved to be insufficient in the field of quality of care, especially on patient satisfaction questionnaires.

Unfortunately, until now there are few guidelines to evaluate instruments' validation processes and help to define the most important steps for the development of a new instrument ^{14 , 15 , 16}, which led to the large quantity of instruments found. This study used the COSMIN guide to check the quality of each measurement property described on instruments validation process and psychometric properties ¹⁷. This analysis demonstrated that most of the instrument achieved a medium score ("good" or "fair"). The comparison with other studies is difficult because no other systematic review with similar characteristics was found in the literature.

Even though patient satisfaction has been widely studied in healthcare, no gold standard instrument or even a validated instrument was found for many different contexts ^{20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56}. The option to develop new instruments by the healthcare organizations can reflect the specificity of each culture or health settings, but, at the same time, this phenomenon can show a gap in state-of-the-art approaches in this area.

It should be highlighted that users' satisfaction is just one facet of quality of care concept and the dimension related to this topic is the patient centered care. Satisfaction studies can help some local health policies because they can show a tendency of presence of quality on care. It had already been shown that improvement in actions to understand patient preferences or patient-physician communication can impact outcomes ⁶⁴.

Rathert et al. ⁶⁵ reported in their systematic review that patient centered care actions may affect patients over time in ambulatory settings. Moreover, to understand how these concepts are related to patient health outcomes can help to determine the effectiveness of patient centered care.

This review presents limitations and even through some search equations had been tested, the final search strategy assumed restricted headings and only papers published since 2002. The search strategy used the headings merging the terms "patient centered care" and "patient satisfaction". In addition, the term 'psychometric properties' was chosen to capture all kinds of possible analyses. This choice could lead to excluding articles classified as validity or reliability, although previous tests have shown that it were included. The temporal limitation was included because the literature on patient centered care as a dimension of quality in health care has become more widespread recently ^{3 , 4}. Hence, the development of instruments for patient satisfaction based has also proved to be more recent. Another limitation of the present study is caused by access problems to bibliographic databases, whereby important databases such as EMBASE were not screened. Furthermore, the search strategy faced some difficulties to find studies from different areas of healthcare, such as physiotherapy, odontology or nutrition. This can give rise to some difficulties when it comes to the interpretation of the satisfaction dimensions for all health fields and the structure of the main instruments.

Conclusion

This review showed that there is not a consensus about a reference instrument (gold standard) for patient satisfaction measurement in healthcare. However, the main dimensions of many instruments utilized are patient-professional interactions, physical environment and internal management processes. The measurement instruments analyzed have shown a medium score of methodological quality by the COSMIN checklist. This construct presents many options of measurement, but the utilization of an instrument to measure patient satisfaction of health services should consider the quality of the validation process of each questionnaire. Finally, it seems clear that the "patient satisfaction construct" should be measured considering a multidimensional approach, because even the patient's clinical profile, such as health service environment or professional behavior, can be determinants for patient satisfaction. Furthermore, within the context of shared decision-making and preference sensitive care, it is important to highlight the need to better understand the role of patient experience and satisfaction as a measure of quality of care ^{12 , 66}.

Acknowledgments

The authors would like to thank the National School of Public Health, Oswaldo Cruz Foundation (ENSP/Fiocruz) for their structural and financial support in carrying out the research and for help in finding all papers.

References

• ¹
  Shaw C, Groene O, Mora N, Sunol R. Accreditation and ISO certification: do they explain differences in quality management in European hospitals? Int J Qual Health Care 2010; 22:445-51.
• ²
  Leatherman S, Ferri TG, Berwick D, Omaswa F, Crisp N. The role of quality improvement in strengthening health systems in developing countries. Int J Qual Health Care 2010; 22:237-43.
• ³
  Committee on Quality of Healthcare in America, Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press; 2001.
• ⁴
  Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013; 3:e001570.
• ⁵
  Machado NP, Nogueira LP. Evaluation of physical therapy service user satisfaction. Braz J Phys Ther 2008; 12:401-8.
• ⁶
  Evrard Y. La satisfaction des consommateurs: état des recherches. Revue Française du Marketing 1993; 144:53-65.
• ⁷
  Oliver RL. A cognitive model of the antecedents and consequences of satisfaction decision. J Mark Res 1980; 44:460-7.
• ⁸
  Fournier S, Mick DG. Rediscovering satisfaction. J Mark 1999; 63:5-23.
• ⁹
  Paim J, Travassos C, Almeida C, Bahia L, Macinko J. The Brazilian health system: history, advances, and challenges. Lancet 2011; 377:1778-97.
• ¹⁰
  Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Med Care Res Rev 2010; 67:275-93.
• ¹¹
  Boyer L, Baumstarck-Barrau K, Cano N, Zendjidjian X, Belzeaux R, Limousin S, et al. Assessment of psychiatric inpatient satisfaction: a systematic review of self-reported instruments. Eur Psychiatry 2009; 24:540-9.
• ¹²
  Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med 2012; 172:405-41.
• ¹³
  Hekkert KD, Cihangir S, Kleefstra SM, van den Berg B, Kool RB. Patient satisfaction revisited: a multilevel approach. Soc Sci Med 2009; 69:68-75.
• ¹⁴
  Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010; 19:539-49.
• ¹⁵
  Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010; 63:737-45.
• ¹⁶
  Mokkink LB, Terwee CB, Gibbons E, Stratford PW, Alonso J, Patrick DL, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist. BMC Med Res Methodol 2010; 10:82.
• ¹⁷
  Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res 2011; 21:651-7.
• ¹⁸
  Mutsaers JH, Peters R, Pool-Goudzwaard AL, Koes BW, Verhagen AP. Psychometric properties of the Pain Attitudes and Beliefs Scale for Physiotherapists: a systematic review. Man Ther 2012; 17: 213-8.
• ¹⁹
  Janaudis-Ferreira T, Beauchamp MK, Goldstein RS, Brooks D. How should we measure arm exercise capacity in COPD? A systematic review. Chest 2012; 141:111-20.
• ²⁰
  Arraras JI, Vera R, Martínez M, Hernández B, Laínez N, Rico M, et al. The EORTC cancer in-patient satisfaction with care questionnaire: EORTC IN-PATSAT32. Clin Transl Oncol 2009; 11:237-42.
• ²¹
  Ottonello M, Franchignoni F, Giordano A, Benevolo E. Patient satisfaction with hospital rehabilitation: validation of the SAT-16 questionnaire through Rasch analysis. Minerva Med 2012; 103: 1-11.
• ²²
  Gerkensmeyer JE, Austin JK. Development and testing of a scale measuring parent satisfaction with staff interactions. J Behav Health Serv Res 2005; 32:61-73.
• ²³
  Evans J, Rose D, Flach C, Csipke E, Glossop H, McCrone P, et al. VOICE: developing a new measure of service users' perceptions of inpatient care, using a participatory methodology. J Ment Health 2012; 21:57-71.
• ²⁴
  Orden SG, Garcia AC, Flavia L, Gijón R, Rieiro CR, De Blas CS, et al. Development and validation of a questionnaire to assess satisfaction with hospital emergency care. Emerg Med J 2010; 28:770-4.
• ²⁵
  Tso IF, Ng SM, Chan CLW. The development and validation of the concise outpatient department user satisfaction scale. Int J Qual Health Care 2006; 18:275-80.
• ²⁶
  Ekwall A, Davis BA. Testing a Swedish version of the consumer emergency care satisfaction scale in an emergency department and 2 observation wards. J Nurs Care Qual 2010; 25:266-73.
• ²⁷
  Kristensson J, Ekwall A. Psychometric properties of the consumer emergency care satisfaction scale: tested on persons accompanying patients in emergency department. J Nurs Care Qual 2008; 23:277-82.
• ²⁸
  Moret L, Nguyen JM, Pillet N, Falissard B, Lombrail P, Gasquet I. Improvement of psychometric properties of a scale measuring inpatient satisfaction with care: a better response rate and a reduction of the ceiling effect. BMC Health Serv Res 2007; (7):197.
• ²⁹
  Soufi G, Belayachi J, Himmich S, Ahid S, Soufi M, Zekraoui A, et al. Patient satisfaction in an acute medicine department in Morocco. BMC Health Serv Res 2010; 10:149.
• ³⁰
  Erci B, Ciftcioglu S. Psychometric evaluation of the primary health-care satisfaction scale in Turkish women. Int J Qual Health Care 2010; 22:500-6.
• ³¹
  Scholle SH, Weisman CS, Anderson RT, Camacho F. The development and validation of the primary care satisfaction survey for women. Womens Health Issues 2004; 14:35-50.
• ³²
  Milutinovic D, Simin D, Brkic N, Brkic S. The patient satisfaction with nursing care quality: the psychometric study of the Serbian version of PSNCQ questionnaire. Scand J Caring Sci 2012; 26:598-606.
• ³³
  Charalambous A. Validation and test-retest reliability of the Risser patient satisfaction scale in Cyprus. J Nurs Manag 2010; 18:61-9.
• ³⁴
  Lam WW, Fielding R, Chow L, Chan M, Leung GM, Ho EY. The Chinese Medical Interview Satisfaction Scale-revised (C-MISS-R): development and validation. Qual Life Res 2005; 14:1187-92.
• ³⁵
  Vanti C, Monticone M, Ceron D, Bonetti F, Piccarreta R, Guccione AA, et al. Italian version of the physical therapy patient satisfaction questionnaire: cross-cultural adaptation and psychometric properties. Phys Ther 2013; 93:911-22.
• ³⁶
  Wilde EF, Hendriks VM. The Client Satisfaction Questionnaire: psychometric properties in a Dutch addict population. Eur Addict Res 2005; 11:157-62.
• ³⁷
  Shea JA, Guerra CE, Weiner J, Aguirre AC, Ravenell KL, Asch DA. Adapting a patient satisfaction instrument for low literate and Spanish-speaking populations: comparison of three formats. Patient Educ Couns 2008; 73:132-40.
• ³⁸
  Danielsen K, Bjertnaes OA, Garratt A, Forland O, Iversen HH, Hunskaar S. The association between demographic factors, user reported experiences and user satisfaction: results from three casualty clinics in Norway. BMC Fam Pract 2010; 11:73.
• ³⁹
  Coluccia A, Cioffi R, Ferretti F, Lorini F. Patient perception of the quality of healthcare services: a psychometric study on the Questionnaire of Quality Perception. Gazz Med Ital 2009; 168:179-87.
• ⁴⁰
  Poinsot R, Altmeyer A, Conroy T, Savignoni A, Asselain B, Leonard I, et al. Multisite validation study of questionnaire assessing out-patient satisfaction with care questionnaire in ambulatory chemotherapy or radiotherapy treatment. Bull Cancer 2006; 93:315-27.
• ⁴¹
  Webster TR, Mantopoulos J, Jackson E, Cole-Lewis H, Kidane L, Kebede S, et al. A brief questionnaire for assessing patient healthcare experiences in low-income settings. Int J Qual Health Care 2011; 23:258-68.
• ⁴²
  Gonzalez N, Quintana JM, Bilbao A, Escobar A, Aizpuru F, Thompson A, et al. Development and validation of an in-patient satisfaction questionnaire. Int J Qual Health Care 2005; 17:465-72.
• ⁴³
  Manaf NHA, Mohd D, Abdullah K. Development and validation of patient satisfaction instrument. Leadersh Health Serv 2012; 25:27-38.
• ⁴⁴
  Granado de la Orden S, Rodríguez Rieiro C, Olmedo Lucerón MC, Chacón Garía A, Vigil Escribano D, Rodríguez Pérez P. Design and validation of a questionnaire of the hospital outpatient clinics in Madrid, Spain. Rev Esp Salud Pública 2007; 81: 637-45.
• ⁴⁵
  Meehan T, Bergen H, Stedman T. Monitoring consumer satisfaction with inpatient service delivery: the Inpatient Evaluation of Service Questionnaire. Aust N Z J Psychiatry 2002; 36:807-11.
• ⁴⁶
  Schroder A, Ahlström G, Wilde-Larsson B, Lundqvist LO. Psychometric properties of the Quality in Psychiatric Care - Outpatient (QPC-OP) instrument. Int J Ment Health Nurs 2011; 20:445-53.
• ⁴⁷
  Gagnon M, Hébert R, Dubé M, Dubois MF. Development and validation of the Health Care Satisfaction Questionnaire (HCSQ) in elders. J Nurs Meas 2006; 14:190-204.
• ⁴⁸
  Sjetne IS, Bjertnaes OA, Olsen RV, Iversen HH, Bukholm G. The Generic Short Patient Experiences Questionnaire (GS-PEQ): identification of core items from a survey in Norway. BMC Health Serv Res 2011; 11:88.
• ⁴⁹
  Campbell JL, Dickens A, Richards SH, Pound P, Greco M, Bower P. Capturing users' experience of UK out-of-hours primary medical care: piloting and psychometric properties of the Out-of-hours Patient Questionnaire. Qual Saf Health Care 2007; 16:462-8.
• ⁵⁰
  Hendriks AA, Vrielink MR, van Es SQ, De Haes HJ, Smets EM. Assessing inpatients' satisfaction with hospital care: should we prefer evaluation or satisfaction ratings? Patient Educ Couns 2004; 55: 142-6.
• ⁵¹
  Clark C, Young MS, Jackson E, Graeber C, Mazelis R, Kammerer N, et al. Consumer perceptions of integrated trauma-informed services among women with co-occurring disorders. J Behav Health Serv Res 2008; 35:71-90.
• ⁵²
  Latour JM, Duivenvoorden HJ, Hazelzet JA, van Goudoever JB. Development and validation of a neonatal intensive care parent satisfaction instrument. Pediatr Crit Care Med 2012; 13:554-9.
• ⁵³
  Otani K, Kurz RS, Harris LE. Managing primary care using patient satisfaction measures. J Healthc Manag 2005; 50:311-24.
• ⁵⁴
  Castillo AF, Vilchez-Lara MJ, Sada-Lázaro E. Satisfaction with hospitalization scale: adaptation and psychometrics properties. Behavioral Psychology 2012; 20:383-400.
• ⁵⁵
  Dyer N, Sorra JS, Smith SA, Cleary PD, Hays RD. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Adult Visit Survey. Med Care 2012; 50 Suppl:28-34.
• ⁵⁶
  Vanti C, Bonetti F, Ceron D, Piccarreta R, Violante FS, Guccione A, et al. Cross-cultural adaptation and validation of the Physical Therapy Outpatient Satisfaction Survey in an Italian musculoskeletal population. BMC Musculoskelet Disord 2013; 14:125.
• ⁵⁷
  Rozenblum R, Lisby M, Hockey PM, Levtzion-Korach O, Salzberg CA, Efrati N, et al. The patient satisfaction chasm: the gap between hospital management and frontline clinicians. BMJ Qual Saf 2013; 22:242-50.
• ⁵⁸
  Rashid WE, Jusoff K. Service quality in health care setting. Int J Health Care Qual Assur 2009; 22: 471-82.
• ⁵⁹
  Rozenblum R, Lisby M, Hockey PM, Levtizion-Korach O, Salzberg CA, Lipsitz S, et al. Uncovering the blind spot of patient satisfaction: an international survey. BMJ Qual Saf 2011; 20:959-65.
• ⁶⁰
  Levine R, Shore K, Lubalin J, Garfinkel S, Hurtado M, Craman K. Comparing physician and patient perceptions of quality in ambulatory care. Int J Qual Health Care 2012; 24:348-56.
• ⁶¹
  Reichenheim ME, Moraes CL, Oliveira ASD, Lobato G. Revisiting the dimensional structure of the Edinburgh Postnatal Depression Scale (EPDS): empirical evidence for a general factor. BMC Med Res Methodol 2011; 11:93.
• ⁶²
  Herdman M, Fox-Rushby J, Badia X. A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual Life Res 1998; 7:323-35.
• ⁶³
  Herdman M, Fox-Rushby J, Badia X. "Equivalence" and the translation and adaptation of health-related quality of life questionnaires. Qual Life Res 1997; 6:237-47.
• ⁶⁴
  Lee YY, Lin JL. Do patient autonomy preferences matter? Linking patient-centered care to patient-physician relationship and health outcomes. Soc Sci Med 2010; 71:1811-8.
• ⁶⁵
  Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Med Care Res Rev 2012; 70:351-79.
• ⁶⁶
  Wennberg JE. Tracking medicine a researcher's quest to understand health care. New York: Oxford University Press; 2010.

Publication Dates

• Publication in this collection
  Jan 2015

History

• Received
  19 Feb 2014
• Reviewed
  14 Oct 2014
• Accepted
  20 Oct 2014