Cross-cultural adaptation of safety culture tool for Primary Health Care

Abstract

Objective

Translate, adapt and validate the Medical Office Survey on Patient Safety Culture (MOSPSC).

Methods

Methodological study for the cross-cultural adaptation of the MOSPSC, elaborated by the Agency for Healthcare and Research in Quality. The following steps were undertaken: translation, back-translation, expert analysis, target population group and pretest, in a sample of 37 professionals.

Results

In the expert analysis, the tool reached a general content validity score of 0.85. Six professionals performed the assessment by the target population group, and the adaptation suggestions were analyzed and modified by consensus. The pretest involved 37 professionals, who assessed the tool as easy to understand. Cronbach’s alpha coefficient corresponded to 0.95.

Conclusion

The tool was translated and adapted to Brazilian Portuguese with a satisfactory content validity and high reliability.

Nursing research; Primary care nursing; Public health nursing; Patient safety; Organizational culture

Resumo

Objetivo

Traduzir, adaptar e validar o instrumento de pesquisa Medical Office Survey on Patient Safety Culture (MOSPSC).

Métodos

Estudo metodológico de adaptação transcultural do instrumento MOSPSC elaborado pela Agency for Healthcare and Research in Quality. Seguiram-se as etapas de tradução, retrotradução, análise de especialistas, grupo de população meta e pré-teste com amostra de 37 profissionais.

Resultados

Na análise dos especialistas, o instrumento atingiu índice de validade de conteúdo geral de 0,85. A avaliação pelo grupo de população meta foi realizada por seis profissionais, e as sugestões de adaptação foram analisadas e modificadas por consenso. O pré-teste foi realizado com 37 profissionais, que avaliaram o instrumento como de fácil compreensão. O coeficiente alfa de Cronbach foi de 0,95.

Conclusão

O instrumento foi traduzido e adaptado para a língua portuguesa do Brasil com nível satisfatório de validade de conteúdo e alta confiabilidade.

Pesquisa em enfermagem; Enfermagem de atenção primária; Enfermagem em saúde pública; Segurança do paciente; Cultura organizacional

Introduction

The importance of discussing the patient safety culture and establishing safer care that does not cause harm is beyond doubt, with a view to establishing a safety culture at health institutions and offering quality care. Patient safety is neither an individual’s nor a professional category’s problem, but a process that involves an institutional transformation.

The patient safety culture is a multifactorial structure, intended to promote an approach of the system to prevent and reduce damage for the patients, essentially referring to a culture in which all stakeholders are aware of their role and contribution to the organization, being responsible for the consequences of their actions.(¹,²) The frailty aspects include the existence of the culture of fear that errors are registered in their job records, the clear lack of communication and the reporting culture of adverse events.(³)

Adverse event management involves identification, registering, analysis, discussion and prevention, in a culture of accountability instead of blame.(¹) Cultural transformation is thus a complex process with multiple factors that influence its success.(⁴)

The assessment of the safety culture is considered the starting point to understand the current scenario and start the planning of actions aimed at changes to reduce the incidence of adverse events.(⁵) It permits the identification and prospective management of relevant safety issues in the work routines, aiming to guarantee safe health care in general practice.(⁶)

It is important for the organizational culture to support learning and development since, if it is based on punishment and guilt, it can cause the omission of adverse events, hampering the construction of an institutional culture aimed at patient safety.(³) The careful analysis of error triggers evidences a series of incidents that, even in a safe practice, when influenced by the work environment and organizational culture, can produce bad results. Hence, human error should be faced in two different ways: the individual mode and the organizational mode.(¹) The factors that knowingly affect the patient safety outcomes include the number of nurses at a service, the nurses’ level of education and a favorable work environment.(⁷)

Thus, it is fundamental to adopt solutions applicable to all members of the organizations, which should be easily integrated in the routine and work flow, in order to increase the adherence and sustainability.(⁴) Teamwork should be strengthened as a basic core, since is may contain a decisive potential for the efficacy of the current Primary Health Care model.(⁸) The institutions should critically reflect on the role the managers should play, as their strategic decisions include personnel management, demanding professional encouragement and training for the effective prevention, reporting and management of these risks during the performance and assessment of the care delivered, the planning of the facilities, the elaboration of the operating procedures, the choice of the equipment and all other decisions that define the structure of the system.(³)

In recente years, patient safety research in primary care has considerably evolved.(⁹) Adverse events are common also in Primary Care, where most services are provided. Therefore, there has been increasing interest in patient safety factors also beyond the hospital contexto.(¹⁰) Thus, the safety culture needs to be explored from the perspective of the multidisciplinar teams, inserted in an organizational context, to constitute a body of knowledge, identifying the professionals’ view on patient safety and, thus, arousing debate and reflection on the theme to support the implementation of actions that improve the safety culture and the quality of care at Primary Health Care services.

The qualification of Primary Care in the Unified Health System (SUS), adopted by the Brazilian government, intends to rescue the universal tone of the Declaration of Alma-Ata, emphasizing the reorientation role of the care model towards a universal and integrated health care system. In that sense, the Family Health Strategy is one of the proposals for the reorganization of Primary Care, being considered an alternative action to achieve the objectives of universalization, equity and integrality.(¹¹)

Based on the above, the objective in this study was to translate, adapt and validate the research tool Medical Office Survey on Patient Safety Culture (MOSPSC) to assess the patient safety culture in Primary Health Care in Brazil.

Methods

A methodological study was undertaken for the translation and cultural adaptation of the MOSPSC,(¹²) an assessment tool the Agency for Health Care Research and Quality (AHRQ) developed in 2007. This tool has shown its usefulness as a form of scientific research. The cross-cultural adaptation and validation was undertaken for use in Primary Health Care in Spain.(¹³) It was also validated in Arabic and recently applied in a study in Al Mukalla, Yemen.(¹⁴) In a study developed by the LINNEAUS project, published in November 2015, in which 15 experts from the United Kingdom, the Netherlands, Denmark, Germany, Poland and Austria analyzed the tool, the results show that it is useful and applicable to assess the patient safety culture at Primary Health Care services in Europe.(¹⁵)

The original assessment tool consists of 51 questions that measure 12 dimensions of the patient safety construct, including: (1) open communication; (2) error communication; (3) information exchange with other sectors; (4) work process and standardization; (5) organizational learning; (6) general perceived patient safety and quality; (7) management support in patient safety; (8) follow-up of patient care; (9) aspects of patient safety and quality; (10) team training; (11) teamwork; and (12) pressure at work and rhythm.

To achieve the study objective, initially six steps were followed (Figure 1), strictly monitored and registered in reports, according to a guideline for validation studies in health, which are: Step 1 – translation of original tool to target language by two independent translators, (versions T1 and T2), native speakers of Brazilian Portuguese and bilingual in English/Portuguese, experienced in this method and knowledgeable about the research objective; Step 2 - in synthesis I, versions T1 and T2 were compared with the original version of the tool and summarized; Step 3 – back translation by two independent translators (R-T1 and R-T2), bilingual native North Americans, who were unfamiliar with the original version of tool and not knowledgeable about the study objectives; Step 4 - in Synthesis II, the cross-cultural adaptations were undertaken after assessing for discrepancies; Step 5 - Content validity and semantic analysis undertaken in two phases.

Figure 1
Steps of the method; CVI – Content Validity Index; IRA – inter-rater agreement; PHC – Primary Health Care

In Phase 1 of Step 5, the expert analysis was undertaken. Six experts participated, selected based on the following criteria: be a researcher (M.Sc. or Ph.D.) and author of scientific research on the theme patient safety or methodological advice for tool adaptation. Curricula were analyzed on the Lattes Platform of the Brazilian Scientific and Technological Development Council and, beyond the criteria established, five out of six experts have expertise in collective health.

The invitation and instructions were forwarded to the experts by e-mail. For the sake of analysis, the research program SurveyMonkey^® was used, which contained the items for individual assessment in terms of clarity, pertinence and content form, using a Likert scale with the following scores: 1 unclear, 2 hardly clear, 3 clear and 4 very clear. At the end of each item, there was space for suggestions and observations.

To measure the proportion or percentage of inter-rater agreement, the collected data were analyzed concerning the Content Validity Index,(¹⁶,¹⁷) according to the following formula:

• Number of “3” and “4” answers divided by total number of answers.

In addition, the inter-rater agreement level was calculated for each section, using the following formula:

• Number of items with Content Validity Index ≥80% divided by total number of items in each section.

The inter-rater agreement serves to assess the extent of the experts’ reliability in the assessments of the items in view of the study context.(¹⁸) Finally, the Content Validity Index of the tool was calculated by adding up the Content Validity Indices, divided by the total number of items.

Phase 2 of Step 5 referred to the assessment by a group from the target population, aiming to verify whether all items were understandable to the target population of the tool. In this phase, six professionals from the target population participated, representing the main professional categories active in Primary Health Care, ranging from the lowest to the highest education level. Each item was assessed interactively and its understanding was analyzed during a brainstorming.(¹⁹) For the questions whose interpretation aroused doubts, suggestions were requested for adaptations, which were registered in a report and later analyzed.

In Step 6, the pretest was applied, which was aimed at assessing whether the questionnaire was understandable to a larger number of people in the target population and the reliability analysis of the tool was processed.

The data were collected in March and April 2005 in a sample of 37 multidisciplinary team professionals from three health centers and one family clinic (service with eight family health teams) in a regional health department of the State Health Secretary in the Federal District. The understanding was scored on a five-point Likert scale, as follows: 1 I did not understand; 2 I hardly understood; 3 I understood more or less; 4 I understood; and 5 I fully understood. To analyze the reliability, Cronbach’s alpha coefficient was used.

The study was registered in Brazil under the Platform Presentation of Certificate number to Ethics Assessment (CAEE) 31787314.0.0000.5553.Initially, tbe AHRQ was consulted, which authorized the study.

Results

The adaptations of the assessment tool started with the title, “Medical Office Survey on Patient Safety”, originally translated as “Pesquisa de Consultório Médico sobre Segurança do Paciente”. The goal was to adapt a tool that could be used in the different forms of Primary Health Care. Therefore, the title was modified to “Pesquisa sobre Cultura de Segurança do Paciente para Atenção Primária” (“Research on Patient Safety Culture for Primary Health Care”). In addition, the term “medical office” was changed to “health service”, and the term “provider” to “physician”. The professionals included in the health team were also adapted, as there is no equivalent for some professional categories in Brazil.

Attachment 1 displays the final version of the translation and cross-cultural adaptation applied in the pretest.

Expert analysis

The items in the assessment tool that received more than 20% of score 1 (unclear) or 2 (hardly clear) were considered unsatisfactory. They were then modified based on the suggestions while maintaining the general concept. Hence, the minimum inter-rater agreement level was set at 80% to avoid any need for adaptations.

In section A, only three items (questions 1, 3 and 8) reached a Content Validity Index of 0.8; questions 4, 5 and 6 reached index 0.7 and the remainder below 5. In section B, the Content Validity Index was unsatisfactory because the alternative answers were maintained in a single question. In Section C, most items reached an index between 0.8 and 1.0; the index was inferior to 0.7 in only three items (questions 3, 12 and 14), requiring adaptations. In section D, only one item (question 6) did not reach a satisfactory Content Validity Index; the remainder continued between 0.8 and 1.0. In sections E, F, G and H, all questions obtained indices superior to 0.8.

As regards the inter-rater agreement, the coefficients obtained in sections A (0.3) and B (0.0) were unsatisfactory, demanding further adaptations. In section C, the inter-rater agreement corresponded to 0.8; in D, 0.9; and the remaining sections reached a score of 1.0.

Although some sections required further adaptations, the calculation of the Content Validity Index for the general tool was satisfactory, corresponding to 0.85.

Assessment by target population group

In this pahse, six professionals participated in the research, one from each category: nurse, physician, dentist, nursing technician, community health agent and oral health technician. As for education, one held a secondary education degree, one was taking a higher education program and one held a higher education degree, two held a specialization degree and one a Master’s degree.

The researchers analyzed the group’s suggestions and the items considered relevant were modified by consensus. In Section A, question number 8 was included, suggested by the group and accepted, so that this section consisted of ten questions (Attachment 1).

Pretest

Fifty-two questionnaires were distributed, 37 of which were returned (71%). In the distribution by professional category, the collection was done to include the widest possible range, as follows: nursing technicians (n=11; 29.7%), nurses (n=7; 19.0%), physicians (n=4; 10.8%), dentists (n=3; 8.1%), administrative team (n=3; 8.1%), laboratory technicians (n=2; 5.4%), nutritionist (n=1; 2.7%), oral health technician (n=1; 2.7%), administrator (n=1; 2.7%), manager (n=1; 2.7%), head nurse (n=1; 2.7%), head of registration sector (n=1; 2.7%), and community health agent (n=1; 2.7%).

The Portuguese version of the MOSPSC showed a Cronbach’s alpha coefficient of 0.95, expressing high reliability.

Discussion

This study was limited by the fact that, to reach a satisfactory level with only six experts in the calculation of the Content Validity Index, five experts had to assess the item with a minimum score of 3 (clear) or 4 (very clear). Another aspect was that, in section B, the subitems were maintained in a single question, which interfered in the assessment, because the experts suggested that the answered should be divided. Despite these limitations, the Content Validity Index of the general tool reached a satisfactory level (0.85). To check the validity of new tools in general, some authors suggest a minimum agreement coefficient of 0.80.(¹⁷)

In the internal consistency analysis, Cronbach’s alpha coefficient corresponded to 0.95, similar to the coefficient found in the validation of the tool for Spanish (0.96).(¹³)

The use of a consistent method for the translation, cross-cultural adaptation and validation of a research tool is essential.(¹²) In this study, the steps were carefully monitored, analyzed and documented to achieve a better consistency level. A robust and well-developed tool, with validity and reliability of the data in the original version, strictly adapted and translated in different languages, permits the comparison of the results on an international scale, in different cultures.(²⁰)

The research steps were used to support the conceptual, semantic and content equivalence of the tool to be translated. Experts performed the content analysis step. According to the methodological framework used, six experts are sufficient to obtain content validity.(¹³,¹⁹) Therefore, they were carefully selected to guarantee the quality of the evaluation.

The semantic analysis was undertaken through the target population group, assessing the understanding and face validation. The items were reviewed and modified to enhance the understanding and clarity in accordance with the suggestions. A project to translate, adapt and validate a cross-cultural research tool can take several years and is normally developed based on more than one study as a methodological framework. The initial target can be defined as the translation, adaptation and validation of a tool using steps 1 to 5, followed by the full psychotechnical test of the prefinal version of the translated tool in a sample of the target public.(¹⁶)

The largest number of suggested adaptations were related to section A, in the expert analysis phase as well as in the assessment by the target population. One factor that contributed to reduce the Content Validity Index of the section was one expert’s assessment, who scored all items as “hardly clear”, not related to the question, but to the answer frequency. Replacing the alternative answers “weekly” or “monthly” by “at least once a week” and “at least once a month” was suggested, respectively.

In section H, the professional categories active at Primary Health Care services in Brazil were adapted, considering that the theme covers the organizational structure and the multiprofessional team. Similarly, another instrument translation and validation study for use in hospitals developed this adaptation.(⁶)

None of the items was excluded from the assessment tool; on the opposite, item 8 was added in section A, suggested by the target population group. The item refers to the access to medical exams as, in the Brazilian reality, the necessary exams are not always offered to the patients freely and in due time. Access is defined as the user’s capacity to obtain health care whenever necessary, easily and conveniently,(¹¹) and this dimension of quality in health interferes directly in the safety, as it makes appropriate diagnosis and treatment impossible. The large number of questions in the tool may interfere in the adherence to the research and, in another study, it was equally suggested that a lesser number of items would be more appropriate.(¹³)

Hence, the challenge of adapting a tool cross-culturally, and the limitations of assessing a complex theme like the patient safety culture, containing items that target managers, the administrative team and the multiprofessional team, its reliability can be confirmed. Nevertheless, subsequent steps need to be pursued, aimed at the operational and measuring equivalence, in order to prove its psychometric properties.(⁶) The application of a validated questionnaire in studies compared among different contexts in the country and among other countries from different continents will permit understanding the multifaceted phenomenon of safety culture at several Primary Health Care services.

Conclusion

The research tool Medical Office Survey on Patient Safety Culture was translated, cross-culturally adapted and validated, including the semantic analysis and assessment of the clarity and understanding of the items. In the expert analysis, the results demonstrated satisfactory content validity. In the pretest, the Brazilian Portuguese version showed high reliability, according to Cronbach’s alpha, and was considered easy to understand by the target population.

Attachment 1. Adapted and validated tool

https://minio.scielo.br/documentstore/1982-0194/NKwYbxLthmqg6m9zYKTvYMy/ac3c2f47baa93133ac253d9de4498ef284b29754.pdf

Referências

• ¹
  Mendes CM, Barroso FF. Promoting a culture of safety in primary health care. Rev Port Saúde Pública. 2014; 32(2):197-205. Portuguese.
• ²
  Ammouri AA, Tailakh AK, Muliira JK, Geethakrishnan R, Al Kindi SN. Patient safety culture among nurses. Int Nurs Rev. 2015; 62(1): 102-10.
• ³
  Françolin L, Gabriel CS, Bernardes A, Silva AE, Brito MF, Machado JP. Patient safety management from the perspective of nurses. Rev Esc Enferm USP. 2015; 49(2):277-83.
• ⁴
  Thomas L, Galla C. Building a culture of safety through team training and engagement. BMJ Qual Saf. 2013; 22(5):425-34.
• ⁵
  Reis CT, Martins M, Laguardia J. [Patient safety as a dimension of the quality of health care-a look at the literature]. Cien Saude Colet. 2013; 18(7):2029-36. Portuguese.
• ⁶
  Reis CT, Laguardia J, Martins M. [Translation and cross-cultural adaptation of the Brazilian version of the Hospital Survey on Patient Safety Culture: initial stage]. Cad. Saúde Pública. 2012; 28(11):2199-210. Portuguese.
• ⁷
  Kirwan M, Matthews A, Scott PA. The impact of the work environment of nurses on patient safety outcomes: a multi-level modelling approach. Int J Nurs Stud. 2013; 50(2):253-63.
• ⁸
  Rodríguez-Cogollo R, Paredes-Alvarado IR, Galicia-Flores T, Barrasa-Villar JI, Ruiz SC. [Patient safety culture in family and community medicine residents in Aragon]. Rev Calid Asist. 2014; 29(3):143-9. Spanish.
• ⁹
  Gaal S, Verstappe W, Wensing M. What do primary care physicians and researchers consider the most important patient safety improvement strategies? BMC Health Serv Res. 2011; 11:102.
• ¹⁰
  Bondevik GT, Hofoss D, Hansen EH, Deilkås EC. The safety attitudes questionnaire - ambulatory version: psychometric properties of the Norwegian translated version for the primary care setting. BMC Health Serv Res. 2014; 14:139.
• ¹¹
  Oliveira MA, Pereira IC. Primary Health Care essential attributes and the Family Health Strategy. Rev. Bras. Enferm. 2013; 66(spe):158-64. Portuguese.
• ¹²
  Agency for Healthcare Research and Quality (AHRQ). Medical Office Survey on Patient Safety Culture [Internet]. [cited 2015 Jul 7]. Available from:http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/medical-office/index.html
  » http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/medical-office/index.html
• ¹³
  Torijano-Casalengua ML, Olivera-Cañadas G, Astier-Peña MP, Maderuelo-Fernández JÁ, Silvestre-Busto C. [Validation of a questionnaire to assess patient safety culture in Spanish Primary Health Care professional]. Aten Primaria. 2013; 45(1):21-37.
• ¹⁴
  Webair HH, Al-Assani SS, Al-Haddad RH, Al-Shaeeb WH, Bin Selm MA, Alyamani AS. Assessment of patient safety culture in primary care setting, Al-Mukala, Yemen. BMC Fam Pract. 2015; 16(1):136.
• ¹⁵
  Parker D, Wensing M, Esmail A, Valderas JM. Measurement tools and process indicators of patient safety culture in primary care. A mixed methods study by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract. 2015 Nov; 21 Supp 1:26-30.
• ¹⁶
  Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract. 2011; 17(2):268-74.
• ¹⁷
  Alexandre NM, Coluci MZ. [Content validity in the development and adaptation processes of measurement instruments]. Ciênc. saúde coletiva. 2011; 16(7):3061-8. Portuguese.
• ¹⁸
  Bellucci JJ, Matsuda LM. [Construction and validation of an instrument to assess the Reception with Risk Rating]. Rev Bras Enferm. 2012; 65(5):751-7. Portuguese.
• ¹⁹
  Pasquali L. [Principles of elaboration of psychological scales]. Rev. psiquiatr. clín. [Intenet]. 1998; 25(5):206-13. Portuguese.
• ²⁰
  Tuthill EM, Burler LM, McGrath JM, Cursson RM, Maklware, Gable RK, et al. Cross-cultural adaptation of instruments assessing breastfeeding determinants: a multi-step approach. Int Breastfeed J. 2014; 9:16.

Publication Dates

• Publication in this collection
  Jan-Feb 2016

History

• Received
  20 July 2015
• Accepted
  18 Jan 2016