The Normalization Measure Development (NoMAD) tool is used to determine the contextual determinants in the process of implementing complex health interventions. The aim of this study is to translate and culturally adapt NoMAD to Brazilian Portuguese.
The cross-cultural adaptation was performed in five steps: 1) translation of the questionnaire into Portuguese; 2) synthesis and creation of the first version; 3) back-translation of the instrument into the source language; 4) review of the instrument by a group of experts and target professionals; and 5) pretesting. A final version of the questionnaire was answered by users of a clinical monitoring system in specialist care services for people living with HIV/AIDS, and the internal consistency of the questionnaire was assessed using Cronbach's alpha.
The questionnaire was answered by 188 health professionals, of which 87.7% were female, and the average age was 45.2 years. For the final version of the questionnaire, Cronbach's alpha was over 0.70 for the construct's coherence (0.74), collective action (0.70), cognitive participation (0.71), and reflexive monitoring (0.81).
The NoMAD questionnaire was cross-culturally adapted and can be used to evaluate the implementation of complex health care interventions.
Implementation Science; Implementation process; NoMAD; Cross-cultural adaptation
O questionário Normalisation Measure Development (NoMAD) tool busca identificar os fatores contextuais determinantes no processo de implementação de intervenções complexas em saúde. Este artigo tem por objetivo traduzir e adaptar culturalmente o NoMAD para o português do Brasil.
A adaptação transcultural foi realizada em cinco etapas: 1-Tradução do questionário para o português; 2-Síntese e criação da primeira versão; 3-Retro-tradução do instrumento para a língua de origem; 4-Revisão por grupo de especialista e profissionais-alvo do instrumento; e 5-Pré-teste. A versão final do questionário foi respondida por usuários do sistema de monitoramento clínico em serviços de assistência especializada às pessoas vivendo com HIV/AIDS e sua consistência interna foi avaliada por meio de alpha de Cronbach.
O questionário foi respondido por 188 profissionais, 87,7% eram do sexo feminino e média de idade de 45,2 anos. A versão final do questionário apresentou α de Cronbach superiores a 0.70 nos construtos coerência (0.74), ação coletiva (0.70), participação cognitiva (0,71) e monitoramento reflexivo (0.81).
O questionário NoMAD foi adaptado transculturalmente e pode ser utilizado para avaliar a implementação de intervenções complexas no cuidado em saúde.
Ciência da implementação; Inquéritos e Questionários; Promoção da Saúde/organização & administração
One aim in implementing new health interventions is to maximize the intended benefits with the provision of services and patient care. However, this process is generally designed and executed without considering the characteristics of the intervention's context, which could lead to underuse of the intervention, regardless of its potential to produce better health outcomes11. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327-50.–33. May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11:141.. The aim of implementation science is to understand the contextual factors that influence the implementation of health interventions. This field is defined as the “scientific study of methods to promote the systematic incorporation of research results and other evidence-based practices into routine practice and, consequently, improve the quality and effectiveness of health services” (Eccles & Mittman, 2006. p.1)44. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1.. To instrumentalize studies developed from this perspective, researchers in the area have developed theories, models, and approaches with three main objectives: 1) describing or guide the process of translating evidence into practice; 2) understanding or explaining the facts that influence the results of the implementation; and 3) evaluating the implementation55. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53..
Normalization Process Theory (NPT) is an implementation theory that seeks to explain the contextual factors that influence the implementation of complex health interventions. It is concerned with the social organization of work (implementation), integrating routine elements of everyday life into practice (incorporation), and sustaining practices incorporated into social contexts (integration). The theory assumes that a successful implementation of new health care practices is dynamic, non-linear, and dependent on the collective and coordinated behavior of individuals who work within the limits of health care contexts66. Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8:63.–99. May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13:80..
NPT explains four constructs that determine collective behavior for the incorporation of complex interventions in practice: coherence, participation or cognitive engagement, collective action, and reflective monitoring10, 11. The coherence construct evaluates how a new practice to be implemented is evaluated by the participants, whether it can bring about changes in the work processes performed, and the users' perception of its purpose. The cognitive participation construct assesses the individual and collective involvement of professionals with the new intervention and their motivation to ensure the day-to-day implementation of the service.
The collective action construct assesses professionals' perception of the intervention's implementation process in pre-existing routine while considering the capacity and interaction with other professionals, the availability of resources, and technical and administrative support provided by the service coordination. The fourth and final construct, reflective monitoring, assesses how participants evaluate the new practice or intervention, whether it can be improved, and whether it causes changes in work processes in the day-to-day services 10, 11.
The Normalization Measure Development (NoMAD) tool is a developed, validated, NPT-based questionnaire11, 12. The questionnaire evaluates contextual factors that are seen as barriers or facilitators for the implementation of interventions in day-to-day services by professionals. The aim of this study is to present the process of cross-cultural adaptation of the NoMAD questionnaire and a final version in Brazilian Portuguese.
The NoMAD questionnaire
NoMAD is available for free1313. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. NoMAD: Implementation measure based on Normalization Process Theory. [Measurement instrument]. Retrieved from http://www.normalizationprocess.org
http://www.normalizationprocess.org... and consists of 23 items that are divided into three sections, as shown in Figure 1. It starts with section A, which consists of two questions about the participant, followed by section B, which has three items that provide an overall assessment of participants' expectations and their experience with the intervention's implementation process. The items in section B are answered on two 10-point Likert scales ranging from “Not at all” to “Completely” and from “Still feels very new” to “Feels completely familiar.”
ORIGINAL AND BRAZILIAN PORTUGUESE FINAL VERSION OF THE NOMAD QUESTIONNAIRE AFTER CROSS-CULTURAL ADAPTATION
Section C contains 20 specific items on the intervention, which correspond to the four constructs of the NPT: Coherence and Cognitive Participation with four items each, seven items for Collective Action, and five items for Reflective Monitoring. The items in part C are answered using a 5-point Likert scale ranging from “Strongly disagree” to “Strongly agree.” If participants consider that any of the 20 items is not relevant to expressing their experience with the intervention, they can respond to one of three “Neutral” options: “Not relevant to my role;” “Not relevant at this stage;” and “Not relevant to the intervention” (Figure 1).
NoMAD does not offer specific instructions for scoring or creating measurements for each construct. However, when assessments are required at the level of the four NPT constructs, it is recommended that the average of the items for each construct be calculated with the aim of creating “‘scores” that can be compared between constructs, groups, or locations if appropriate for the purpose of the study1212. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135..
The cross-cultural adaptation was carried out by following the methodological steps used in the translation of NoMAD into seven other languages within the scope of the European collaboration project “ImpleMentAll” for the implementation of e-health interventions (https://www.implementall.eu/). The steps were made available by the questionnaire authors in a methodological guide and validation articles 11, 12. In step 1, the questionnaire was translated by two Brazilian translators independently. This step provided versions T1 and T2 versions of the Portuguese questionnaire.
In step 2, T1 and T2 were evaluated and compared with the original version by researchers who have good knowledge of English, which gave rise to version T3. Version T3 was back-translated into English by two independent and native English translators in step 3, which generated versions T4 and T5. A committee of experts reviewed these versions to reach a consensus on the differences found and prepare a pre-test version. The committee was composed of two methodologists, two health professionals, and authors of the questionnaire. The committee members were also fluent in the English language and native speakers of Brazilian Portuguese. None of them participated as translators of the questionnaire. The authors responsible for developing the NoMAD questionnaire collaborated directly in the translation and back-translation process.
Before applying the pre-test, 30 health professionals were invited to assess the clarity and understanding of the questionnaire items through three questions: 1) “Write in your words how you understood the previous question;” 2) “Did you fully understand this questionnaire item (yes/no)?” and, 3) “How would you rewrite this question to make it more understandable?” The answers to the first question were evaluated to ensure that the adapted version maintained its equivalence with the original version of the questionnaire. If a participant declared that the item in the second question was not understood, an open field was made available (if not, the participant was asked to write the word or phrase that was not understood) so that the participant could signal which word or expression had not been understood. Items not understood by more than 90% of the participants were restructured, translated, and evaluated by the committee.
The pre-test was answered online by health professionals who participated in the implementation of the Clinical Monitoring System (SIMC) for people living with HIV/AIDS (PLWHIV). SIMC is an individual management support system that was developed by the Ministry of Health in 2013. The system identifies PLWHIV with difficulties in the continuum of care: a) PLWHIV who perform CD4 tests and viral load tests and are not starting antiretroviral treatment; b) PLWHA under treatment who had a detectable viral load after six months of treatment; c) PLWHIV in loss of follow-up; and d) pregnant women with HIV and a detectable viral load1414. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Doenças de Condições Crônicas e Infecções Sexualmente Transmissíveis. Sistema de monitoramento clínico das pessoas vivendo com HIV. [cited 2020 Apr 3]. Available from: https://simc.aids.gov.br
The choice of this sample was due to the low implementation of SIMC in the state of São Paulo. Analysis of the use of SIMC in September 2019 showed that among the 10,561 patients in the treatment gap in the state of São Paulo, 89.3% (n = 9,151 patients) were cases not analyzed by the services. Therefore, 409 professionals with an approved login to use the system in the state were invited to answer NoMAD on an online platform between May and July 2019.
The characterization of the participants and their responses to NoMAD were described using simple and relative frequencies. The mean and standard deviation of the set of respondents in each item of the four constructs were also calculated. The calculation of the average considered the responses on a 5-point Likert scale ranging from “1 - Strongly disagree” to “5 - Strongly agree”.
The internal consistency of the items of each construct was assessed using Cronbach's alpha and considered satisfactory when the values were ≥0.7. All analyses were conducted using the software Stata/IC 14.0®. Ethical approval was obtained from the Research Ethics Committee of the Faculty of Medicine of the University of São Paulo (opinion number 3.033.064).
Figure 1 presents the items from the original questionnaire and the culturally adapted version. The 23 items of the questionnaire and two other questions for the characterization of the respondent were maintained (Chart 1). Of the 30 professionals invited to assess the clarity and understanding of the questionnaire items, 29 participated.
Items C01, C03, AC4, and MR1 were not understood by 90% of the health professionals. The participants reported doubts about the meaning of the expression, “nature of my own work,” and proposed replacing it with the expression, “essential activities of my own work.” The expression, “appropriate skills,” was replaced by “adequate capacity,” and there was a need to modify the phrase “differs from usual ways of working” to “brings changes in the current work routine.” Suggestions for changes were also made to replace the phrase “about the effects of” with “regarding the impact.” After restructuring, the items were answered by 14 other health professionals and were understood by more than 90% of the participants.
Considering SIMC as the implemented intervention, the NoMAD questionnaire was answered by 188 professionals in specialized care services who used the clinical monitoring system. Most participants were female (87.2%) and had higher education with specialization (54.8%) (Table 1). The average age was 45 years, and 28.2% of the participants reported working for more than 15 years in health services. Managers of health services and nurses were the most frequent professions (25% and 22.9%, respectively). The professionals reported little familiarity and low use of the system in their routine (average=2.5 and 2.9, respectively). On the other hand, professionals believed that the system would become routine in their work (average=7.1) (Table 1).
DEMOGRAPHIC CHARACTERISTICS AND RESPONSES OF PARTICIPANTS TRAINED TO USE SIMC (N = 188)
Table 2 shows the answers to the items corresponding to the four questionnaire constructs. In the coherence dimension, the averages varied between 3.5 and 4.1. The lowest average was found for item CO2. This item assessed whether there is a common understanding among the service employees about the purpose of the SIMC. 30.3% of the participants responded that the disagreed with the statement, and another 50.7% neither disagreed nor agreed. However, the participants were aware of the possibilities of SIMC offering to bring changes in the work routine (C01=4.1), in the performance of essential activities (C03 = 3.9), and in the improvement of work (C04=3.9).
RESPONSES OF PROFESSIONALS INVOLVED IN THE IMPLEMENTATION OF SIMC TO THE NOMAD QUESTIONNAIRE
In the cognitive participation construct, only item PC1 had an average lower than 4.0. This item assesses whether there are professionals in the service who decisively encourage others to use SIMC. However, items PC3 and PC4 refer to the participant's willingness to contribute to colleagues and individually support the system. More than 90% of respondents agreed with these items, presenting the highest averages among all NoMAD items in this sample (average=4.2) (Table 2).
The collective action construct demonstrated that 76% of the participants disagreed that the system can cause possible losses in the working relationships between service professionals (average=2.0). The proportion of professionals who trust the ability of others to use SIMC was also above 70% (average=3.8). Less than 50% agreed that the available resources are sufficient to support the use of SIMC, and only 41% reported that employees receive training to enable them to implement SIMC (Table 2).
The reflexive monitoring construct showed that 70.4% of the participants agreed that the system is worthwhile, and almost 80% of the professionals believed they are able to modify their work processes using the system (items MR1 and MR5). The lowest average observed in this construct (MR1=3.5) refers to the professional's perception of the reports made by other colleagues about the impact of using the system. The evaluation of internal consistency indicated Cronbach's α greater than 0.70 in all constructs: coherence (α: 0.74), collective action (α: 0.70) reflective monitoring (α: 0.81), and cognitive participation (α: 0.71). The constructs “reflective monitoring and collective action” (α: 0.87) and “coherence and collective action” (α: 0.81) showed a strong correlation.
This study presents the cross-cultural adaptation of the NoMAD questionnaire and provides a questionnaire to assess the implementation of complex health care interventions in a Brazilian context. The methodology used to carry out this study was carefully chosen to guarantee reliable results and the maintenance of the original meaning of the questionnaire built from the NPT. The translation and back-translation process, the participation of an expert committee, and the pre-testing are steps that have been used and recommended in the literature since 19931515. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417-32.. The questionnaire showed good reliability in the internal consistency test. All four theoretical constructs of the NoMAD questionnaire showed good reliability, with α values between 0.70 and 0.81. The internal consistency result was similar to that found in the validation study of the original instrument, which had α values between 0.78 and 0.831212. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135..
This study provides a questionnaire that is capable of assessing involvement, the perception of individual and collective participation, and the reflection of professionals participating in the implementation of complex interventions based on evidence in the context of health care. NoMAD can be used before or after the implementation of an intervention and can assist managers and professionals who are conducting an implementation process in understanding contextual factors that interfere with the implementation1313. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. NoMAD: Implementation measure based on Normalization Process Theory. [Measurement instrument]. Retrieved from http://www.normalizationprocess.org
http://www.normalizationprocess.org... . Confirmatory factor analysis was not performed due to a sample size of less than 10 cases for each variable, which is recommended as the minimum1616. Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas. 2013;76(6):913-34.. However, Acuardo et al. showed a consensus in the literature regarding the completion of the cross-cultural adaptation process after the pre-test stage1717. Acquardo C, Conway K, Hareendran A, Aaronson N; European Regulatory Issues and Quality of Life Assessment (ERIQA) Group. Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008;11(3):509-21..
This article study has provided a Brazilian Portuguese adaptation of the NoMAD questionnaire to evaluate the implementation of complex interventions in health care. The 23 items from NoMAD contribute to the identification and evaluation of contextual factors involved in the social organization of the work of professionals participating in the implementation.
1Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327-50.
2Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217-26.
3May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11:141.
4Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1.
5Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.
6Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8:63.
7May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4:29.
8Finch TL, Rapley T, Girling M, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: measure development based on normalization process theory (NoMAD): study protocol. Implement Sci. 2013;8:43.
9May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13:80.
10Finch T, Girling M, May C, Mair F, Murray E, Treweek S, et al. Can we measure implementation progress using Normalization Process Theory? Development and validation of the NoMAD survey tool. 48th Congress of the International Society of Paediatric Oncology; 2016. p.135.
11Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, et al. Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):133.
12Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135.
13Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. NoMAD: Implementation measure based on Normalization Process Theory. [Measurement instrument]. Retrieved from http://www.normalizationprocess.org
14Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Doenças de Condições Crônicas e Infecções Sexualmente Transmissíveis. Sistema de monitoramento clínico das pessoas vivendo com HIV. [cited 2020 Apr 3]. Available from: https://simc.aids.gov.br
15Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417-32.
16Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas. 2013;76(6):913-34.
17Acquardo C, Conway K, Hareendran A, Aaronson N; European Regulatory Issues and Quality of Life Assessment (ERIQA) Group. Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008;11(3):509-21.
Publication in this collection
06 Nov 2020
Date of issue
22 Apr 2020
23 May 2020