Training of evidence-based assessment and intervention approaches in cross-cultural contexts: challenges and solutions

Treinamento de estratégias de intervenção e avaliação baseadas em evidências em contextos transculturais: desafios e soluções

UPDATING

Training of evidence-based assessment and intervention approaches in cross-cultural contexts: challenges and solutions

Treinamento de estratégias de intervenção e avaliação baseadas em evidências em contextos transculturais: desafios e soluções

Cheryl Y C So^I; Jack S F Hung^I; José J Bauermeister^II; Peter S Jensen^III; Doa Habib^IV; Paulo Knapp^V; Orit Krispin^VI; Integrated Services Program Task Force^*

^IKwai Chung Hospital, Hong Kong

^IIBehavioral Sciences Research Institute, University of Puerto Rico, Puerto Rico

^IIICenter for the Advancement of Children's Mental Health, Columbia University, New York (NY), United States

^IVChild Mental Health Administrator & Psychiatrist Independent Researcher, Egypt

^VUniversidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre (RS), Brazil

^VIFeinberg Department of Psychiatry, Schneider's Children's Medical Center, Petach Tikvah, Israel

^{Correspondence}

ABSTRACT

Dissemination of evidence-based assessment and intervention approaches for child and adolescent with behavioral and/or emotional problems is now a priority in the field worldwide. However, developing staff competence in evidence-based assessment and intervention approaches in different countries is complicated by some environmental and economic constraints. In this paper a distance training/supervision model is discussed. We describe seven specific challenges encountered and solutions used for overcoming the obstacles in order to implement evidence-based assessment and intervention approaches in different sites in Brazil, Egypt, Israel, and Lebanon.

Keywords: Evidence-based medicine; Process assessment (Health care); Remote consultation; Supervision; Inservice training

RESUMO

A disseminação de estratégias de intervenção e avaliação baseadas em evidências para crianças e adolescentes com problemas comportamentais e/ou emocionais é hoje uma prioridade mundial. No entanto, o desenvolvimento de equipes capacitadas para implementação de estratégias de intervenção e avaliação baseadas em evidências nos diferentes países é limitado por restrições ambientais e econômicas. Neste artigo, discute-se um modelo de treinamento/supervisão à distância. Em seguida, são descritos sete desafios específicos encontrados e as soluções utilizadas para superar os obstáculos para implementação de estratégias de intervenção e avaliação baseadas em evidências em diferentes localidades no Brasil, Egito, Israel e Líbano.

Descritores: Medicina baseada em evidência; Avaliação de processos (Cuidados da saúde); Consulta remota; Supervisão; Capacitação em serviço

The World Psychiatric Association (WPA) Presidential Global Programme on Child Mental Health is the first international collaboration to develop and disseminate evidence-based assessment and intervention approaches (EBAIs) for child and adolescent. As described by Bauermeister et al.,¹ the Integrated Services Program Task Force ("Taskforce") developed two manual-guided, flexible interventions to address internalizing ("Helping Children and Youth with Fears, Sadness, Depression, and Risk for Self-Harm") and externalizing problems ("Helping Challenging Children"). Four sites in Brazil, Egypt, Israel and Lebanon volunteered to implement the program and to participate in a small feasibility study.

This paper describes a distance training/supervision model. We first note the implementation challenges faced by the Taskforce in implementing EBAIs across these widely dispersed geographic sites, and we then describe how each of these challenges were met. The challenges encountered included: 1) difficulties providing training and supervision across distant geographical sites; 2) the lack of close interpersonal relationships usually required in psychotherapy supervisory relationships; 3) the need for a credible EBAI program that could address training needs beyond those of clinicians alone; 4) the need for cultural adaptation of manuals and for ensuring treatment integrity; 5) the implementation of two new service programs simultaneously; 6) the logistic difficulties of conducting international supervisory teleconference across widely spaced time zones and languages; and 7) the need for the program to provide a feasible and sustainable system to continue EBAIs after the withdrawal of the Taskforce. We describe each of these challenges below, detail how these challenges were addressed, and consider implications for future efforts to disseminate EBAIs worldwide.

1. Challenge 1

Difficulties providing training and supervision to four sites across distant geographical regions.

Solution: The Taskforce concluded that the training/supervision model, if it were to be ultimately disseminable, must combine some face-to-face contact with electronically mediated exchanges (e.g., e-mail, web conference) and telecommunication. A one and half day face-to-face training on "Helping Challenging Children" Program for supervisors and therapists was conducted at the initial phase of the WPA Program at a central location (Berlin). Structured training activities included lecture on the theoretical underpinnings and principles of behavioral therapy (BT), modeling, behavioral rehearsals, and feedback.^2-3 Since each site included the Strengths and Difficulties Questionnaire (SDQ)⁴ in their systematic evaluation of children, training was provided on using SDQ and other appropriate clinical information to determine level and type of care based upon treatment-decision algorithms and needs of local culture. Training also included brainstorming discussion on flexible application of the manuals in each site,⁵ based on the availability of health care and school resources, the nature and severity of their clients' problems, as well as the preferences and cultural factors within their communities. After face-to-face training, some supervisors conducted training of therapists in their sites. About 30 clinicians in 4 sites received training either by the Taskforce or site supervisors (Table 1).

After initial training, the Taskforce held monthly international supervision teleconferences with supervisors and therapists. More frequent teleconferences (i.e., twice a month) were arranged at the initial stage to focus on issues of: 1) screening and recruitment of participants for the program; 2) translation and modification of the program contents in view of cultural differences; and 3) training of therapists and non-mental health (non-MH) professionals in each site. After therapy sessions were initiated, discussion shifted to therapy implementation issues, such as how to engage parents to attend therapy sessions and how to modify treatment contents or process to match characteristics of children and families. Once the implementation process of each site had become somewhat smoother, teleconferences were faded to once a month.

While the Taskforce held only monthly teleconferences for progress review and problem-solving discussion, local sites' supervisors provided weekly face-to-face supervision to their therapists in their home countries. Within each site, supervision provided by an experienced mental health professional with clinical experience in the same culture was crucial. They could conduct role-plays with therapists and refine their skills. Most importantly, they could advise on case management based on cultural considerations. Although this type of face-to-face supervision was preferable, it was not feasible for some sites. For example, the "Helping Challenging Children" Program was implemented in a Bedouin Village, which is in the north of Israel. Due to long traveling, on-site supervision was held only once every two months. Therefore, Israel created a new position of a sub-supervisor and assigned it to one of the therapists who served as a "middle-man" to meet with the supervisor once a week.

2. Challenge 2

Personal relationships that are important during supervision process may be compromised in the international supervision teleconference.

Solution: Face-to-face training at the initial stage created an open, supportive environment in which problem-solving discussion and collaboration between each site and the Taskforce were emphasized.^6-7 Thorough discussion on how each clinician provided mental health services within the larger context of their culture was held during training. This helped the Taskforce to have a heightened awareness of, and sensitivity to, cultural influences on mental health services provision in each site. These initially-formed personal relationships, resultant group cohesion and better understanding of the cultural issues assisted the Taskforce to supervise and support each site via the later teleconference, despite the lack of subsequent face-to-face contact.

3. Challenge 3

To provide optimal services for children and families, a credible EBAI program must address training needs beyond those of clinicians alone, such as those of clinical staff and administrators.

Solution: To address these needs, both manuals included multiple training components for supervisors, therapists, and non-MH professionals. Although non-MH professionals were not directly involved in providing therapy to children and families, their understanding of mental health problems in youngsters and their attitude towards the program might impede or facilitate its implementation.^8-9 There are two versions of training, namely, a shorter version for community leaders or school administrators and a full version for teachers or paraprofessionals. Once they had been trained, each sites' therapists were encouraged to use the designated sections of the manuals and materials to reach out to provide psychoeducation to schools, parents, and community leaders.

4. Challenge 4

Cultural adaptation may be necessary since the manuals have been developed principally in the United States for populations of limited ethnic diversity. However, ensuring treatment integrity is also important.^10-12

Solution: The Taskforce involved each site in the development of the treatment manuals and constantly addressed the needs of cultural adaptation¹ Several steps to ensure treatment integrity were performed.¹³ First, for both manuals the Taskforce outlined the treatments with clear specifications of procedures, techniques, tasks, and therapist and child characteristics that should define the treatment for behavioral or emotional problems in children. Second, a uniform training experience was provided to supervisors and therapists by the initial face-to-face training. Each site also received a video compact disc¹⁴ showing major therapeutic skills described in the "Helping Challenging Children" manual. Third, the Taskforce provided continued 1-2 times monthly phone supervision to all participants. Group feedback during international teleconferences was intended to ensure treatment integrity across different sites.

To improve treatment adherence and competence of therapists, the manuals included a checklist to facilitate systematic supervision. According to the level of skill shown by the therapist in delivering the treatment, the supervisor could provide more role-play of certain therapeutic techniques and discuss how to apply these techniques in a culturally appropriate manner, based on the suggestions listed on the checklist. Supervisors could also use the checklist to assess whether the major therapeutic activities of a treatment session were performed. Supervisors were encouraged to consider all relevant aspects of the therapeutic context and respond to these contextual variables appropriately.^8,15 Important contextual variables included cultural, political and socioeconomic factors, attitude towards psychological treatment, and level of improvement already achieved by children or their families. For example, to match the needs of child and family, Lebanon shortened some sessions on parent training for a particular family, considering that the experienced mother had already mastered some positive parenting skills.

5. Challenge 5

Providing training of two programs simultaneously could be overwhelming for the sites.

Solution: Considering the differential complexity of therapeutic skills required for the two service programs, the Taskforce decided that training for children's behavioral (externalizing) problems should be provided first, and that sites should have 3-4 months' experience with the first manual before learning the second manual for emotional (internalizing) problems.¹⁶ The actual timing was tailored to each site, depending upon the availability of mental health personnel and their previous experience with EBAIs.

6. Challenge 6

Poor quality of the call (e.g., disconnected calls during conference, poor sound quality) and participants who use different mother tongues and stay in different time zones sometimes impeded smooth communication during the teleconference.

Solution: To facilitate full attendance, all teleconferences were scheduled in odd hours (i.e., mid-night, early morning, and mid clinic hour for different sites) that required dedication of all participants. They were encouraged to send questions or case vignettes to the Taskforce via email beforehand. This strategy enabled participants to have time to examine thoroughly the materials and then participate in the teleconference actively. Regularly tracking some difficult or illustrative cases during each teleconference also helped therapists develop a more thorough understanding of the application of EBAIs appropriately and flexibly. The Taskforce also conducted training for supervisors via web conference, utilizing procedures where trainers and trainees could jointly view the same materials via the web.

7. Challenge 7

The training/supervision process should provide a feasible and sustainable system to continue EBAIs after the withdrawal of the Taskforce.

Solution: Treatment manuals, translations of treatment materials, and training curricula (e.g., training workshop protocols) can be adopted quickly by other service settings, which in turn can facilitate dissemination of EBAIs worldwide.¹⁷ Uploading the treatment manuals on the WPA website (www.wpanet.org/sectorial/bulletin/message2.htlm) should also make this program more likely to be adopted and sustained by other sites. Notably, with the availability of Arabic translation of the manuals and parents handouts, some site supervisors were able to conduct large-scale training for clinicians from peripheral areas of different countries in the Middle East.

Conclusion

To the best of our knowledge, the World Psychiatric Association (WPA) Presidential Global Programme on Child Mental Health represented the first international collaboration to develop and disseminate EBAIs in different countries. Consistent with previous studies,^18-21 providing training and supervisory services to clinicians in different sites was complicated by some environmental and economic constraints, such as distance and budget. Our distance training/supervision model provided a blueprint for future international training programs. As discussed by Hoagwood et al.,²² high ratings were found on therapist's satisfaction towards training and consultation, treatment acceptability, and the likelihood that they would use the treatment in the future. Nevertheless, some limitations to this study should be noted. First, we did not measure the effects of training on therapist competence, in terms of knowledge and the level of skill in delivering the treatment.^12,23 Second, although we included a checklist of treatment adherence to facilitate supervision, we did not measure treatment adherence systematically. It will be invaluable to develop valid adherence measures in the future.²⁴ The study of training effects on therapeutic outcomes remains limited in the therapy literature.²⁵ Future studies will be needed to determine that such distance training/supervision approaches are sufficient to ensure adequate treatment integrity to EBAI methods, and to demonstrate the ultimate effectiveness of such methods for children and their families across different cultural and geographical settings.

Acknowledgements

See complete acknowledgements in the Introduction to the Special Section, in this issue. The authors also thank the special efforts and dedication of the clinicians and therapists who contributed their time and expertise to this project: Lisiane Lyzkowski and Maria G. Machado (Brazil); Wafaa Mohamed Saad and Mohamed Osman Ebrahim (Egypt); Muhammed El Heib and Sami Hamdan (Israel); Lynn Farah and Youmna Cassir (Lebanon).

Submitted: 21 September, 2005

Accepted: 20 December, 2005

Financing: World Psychiatric Association (WPA)

Conflict of interests: None

• 1. Bauermeister JJ, So CY, Jensen PS, Krispin O, Seif El Din A, Integrated Services Program Task Force. Development of adaptable and flexible treatment manuals for externalizing and internalizing disorders in children and adolescents. Rev Bras Psiquiatr 2006;28(1):67-71.
• 2. Ford JD. Research on training counselors and clinicians. Rev Ed Res 1979;49(1):87-130.
• 3. Lambert MJ, Arnold RC. Research and the supervisory process. Prof Psychol Res Pr 1987;18(3):217-24.
• 4. Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997;38(5):581-6.
• 5. Brannon D. Adult learning principles and methods for enhancing the training role of supervisors. Clin Supervisors 1985;3(2):27-41.
• 6. Bruch H. Learning psychotherapy: rationale and ground rules. Cambridge, MA: Harvard University Press; 1974.
• 7. Albott WL. Supervisory characteristics and other sources of supervision variance. Clin Supervisor 1984;2(4):27-41.
• 8. Weisz J. Psychotherapy for children and adolescents: evidence-based treatments and case examples. Cambridge, UK: Cambridge University Press; 2004.
• 9. Tolan P, Dodge K. Children's mental health as a primary care and concern: a system for comprehensive support and service. Am Psychol 2005;60(6):601-14.
• 10. Yeaton WH, Sechrest L. Critical dimensions in the choice and maintenance of successful treatments: strength, integrity, and effectiveness. J Consult Clin Psychol 1981;49(2):156-67.
• 11. Vermilyea BB, Barlow DH, O'Brien GT. The importance of assessing treatment integrity: An example in the anxiety disorders. J Behav Assess 1984;6(1):1-11.
• 12. Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol 1993;61(4):620-30.
• 13. Kazdin AE. Research design in clinical psychology. 4th ed. Needham Heights, MA: Allyn and Bacon; 1998. p. 476.
• 14. So CY, Leung WL, Hung SF. Enhancement of learning behavior project: cooperation between schools, families, and community. 2nd ed. Hong Kong: Kwai Chung Hospital & The Chinese University of Hong Kong; 2004.
• 15. Kazdin AE. Psychotherapy for children and adolescents: directions for research and practice. New York: Oxford University Press US; 2000. p. 320.
• 16. Kratochwill T, Lepage K, McGivern J. Child and adolescent psychotherapy supervision. In: Watkins CE, editor. Handbook of psychotherapy supervision. New York: Wiley and Sons; 1997. p. 347-65.
• 17. Chorpita B. The frontier of evidence-based practice. In: Kazdin AE, Weisz JR, editors. Evidence-based psychotherapies for children and adolescents. New York: Guilford; 2003. p. 42-59.
• 18. Yellowlees P, Kennedy C. Telemedicine applications in an integrated mental health service based at a teaching hospital. J Telemedicine Telecare 1996;2(4):205-9.
• 19. Stamm BH. Clinical applications of telehealth in mental health care. Prof Psychol Res Pr 1998;29(6):536-42.
• 20. Schopp L, Johnstone B, Reid-Arndt S. Telehealth brain injury training for rural behavioral health generalists: supporting and enhancing rural service delivery networks. Prof Psychol Res Pr 2005;36(2):158-63.
• 21. Wood J, Miller T, Hargrove D. Clinical supervision in rural settings: a telehealth model. Prof Psychol Res Pr 2005;36(2):173-9.
• 22. Hoagwood K, Kelleher K, Murray L, Integrated Services Program Task Force. Implementation of evidence-based practices for children in four countries: a project of the World Psychiatric Association. Rev Bras Psiquiatr 2006;28(1):59-66.
• 23. Margison FR, Barkham M, Evans C, McGrath G, Clark JM, Audin K, Connell J. Measurement and psychotherapy: evidence-based practice and practice-based evidence. Br J Psychiatry 2000;177:123-30.
• 24. Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: issues and research recommendations. Clin Psychol Sci Prac 2005;12(4):365-83.
• 25. Lambert MJ, Ogles BM. The efficacy and effectiveness of psychotherapy. In: Lambert MJ, editor. Bergin and Garfield's handbook of psychotherapy and behavior change. 5th ed. New York: Wiley and Sons; 2005. p. 139-93.

*

Members of the ISP Task Force are listed in the Introduction to the Special Section: Peter S Jensen, The ISP Task Force. Disseminating child & adolescent mental health treatment methods.

Rev Bras Psiquiatr. 2006;28(1):1-2.

Publication Dates

• Publication in this collection
  24 Mar 2006
• Date of issue
  Mar 2006

History

• Accepted
  20 Dec 2005
• Received
  21 Sept 2005