An adaptation of the Interpersonal Problem Areas Rating Scale: pilot and interrater agreement study

Adaptação da Escala de Áreas Problema da Psicoterapia Interpessoal: estudo piloto e avaliação de concordância

BRIEF REPORT

An adaptation of the Interpersonal Problem Areas Rating Scale: pilot and interrater agreement study

Adaptação da Escala de Áreas Problema da Psicoterapia Interpessoal: estudo piloto e avaliação de concordância

Ana Claudia Fontes de Andrade^I; Ellen Frank^II; Francisco Lotufo Neto^III; Patricia R Houck^II

^IClinical psychologist

^IIWestern Psychiatric Institute and Clinic, School of Medicine, University of Pittsburgh, Pittsburg, USA

^IIIDepartment of Psychiatry, Universidade de São Paulo (USP), São Paulo (SP), Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: This article describes the adaptation of a rating scale of interpersonal psychotherapy problem areas to include a fifth problem area appropriate to bipolar disorder and an interrater agreement study in identifying interpersonal problem areas and selecting a primary treatment focus if patients were to engage in treatment.

METHOD: Five research interpersonal psychotherapists assessed nine audiotapes of a single interview with five bipolar and four unipolar patients in which the interpersonal inventory and identification of problem areas were undertaken.

RESULTS: Raters agreed on presence and absence of problem areas in seven tapes. Kappas for identification of problem areas were 1.00 (grief), 0.77 (role dispute), 0.61 (role transition), 0.57 (interpersonal deficits) and 1.00 (loss of healthy self). Kappa for agreement on a primary clinical focus if patients were to engage in interpersonal psychotherapy treatment was 0.64.

CONCLUSIONS: The adaptation of the original scale to include an area pertinent to bipolar disorder proved to be applicable and relevant for use with this population. The results show substantial interrater agreement in identifying problem areas and potential treatment focus.

Descriptors: Psychotherapy; Interpersonal relations; Problems and exercises; Imagery (psychotherapy); Scales

RESUMO

OBJETIVO: Este artigo descreve a adaptação de uma escala de avaliação de áreas problema da psicoterapia interpessoal que inclui uma área própria ao transtorno bipolar e um estudo de concordância em identificar áreas problema e selecionar um foco primário de tratamento caso os pacientes fossem participar de tratamento.

MÉTODO: Cinco terapeutas interpessoais avaliaram nove audiotapes de uma única entrevista com cinco pacientes bipolares e quatro pacientes unipolares em que o inventário interpessoal e identificação de áreas problema foram empreendidos.

RESULTADOS: Os avaliadores concordaram na presença e ausência de áreas problema em sete fitas. Kappas para identificação de áreas problema foram 1,00 (luto), 0,77 (disputa de papel), 0,61 (transição de papel), 0,57 (déficits interpessoais) e 1,00 (perda do self sadio). Kappa para concordância num foco de tratamento clínico foi 0,64.

CONCLUSÕES: A adaptação da escala original para incluir área pertinente ao transtorno bipolar mostrou-se relevante para o uso com tal população. Os resultados demonstram uma concordância substancial entre avaliadores na identificação de áreas problema e foco de tratamento.

Descritores: Psicoterapia; Relações interpessoais; Problemas e exercícios; Imagens (psicoterapia); Escalas

Introduction

Interpersonal psychotherapy (IPT), a time-limited psychotherapy that was initially developed and manualized for unipolar depression by Klerman et al.,¹ was adapted to treat bipolar disorder by the second author of this paper and named interpersonal and social rhythm therapy (IPSRT).²

IPT treatment strategies focus on four specific interpersonal problem areas: grief, interpersonal role disputes, role transitions, and interpersonal deficits. In IPSRT a fifth problem area, "loss of healthy self", was added to describe the symbolic loss patients experience as a consequence of the illness.^2,3

There have been important reviews on both IPT^4-6 and IPSRT⁷ outcome studies, but only a few reviewed their basic processes for clinical formulation and focus of treatment.⁸ Markowitz et al., in an attempt to study IPT components, developed a scale designed to assess interpersonal problem area identification and choice of treatment foci (IPT Problem Area Rating Scale - IPARS).⁹ In a reliability study of three research psychotherapists rating 16 audiotapes from a treatment study of dysthymic disorder, therapists agreed closely on ratings, providing empirical support for potential therapist consistency in problem area selection.

Our objective in this study was to assess interrater agreement of IPT therapists in identifying problem areas and potential treatment focus from audiotapes of unipolar and bipolar patients using a modified version of the IPARS. We hypothesized that our findings would confirm prior positive results using the original scale, and that our adaptation for bipolar disorder and redefinition of interpersonal deficits would not affect reliability.

Adaptation of the Interpersonal Problem Areas Rating Scale

The first two authors modified the original Interpersonal Problem Areas Rating Scale, expanding it to include a fifth problem area, "loss of healthy self", appropriate to the treatment of patients with bipolar disorder (IPARS-M - ^Appendix).

Another change was made on how interpersonal deficits were rated. The original scale has deficits in relation to personality traits and/or lack of social skills. The objective was to remain close to the initial definition of this problem area, which included patients that 1) are socially isolated, 2) have an adequate number of relationships, but find them unfulfilling and/or have difficulty in sustaining them, and 3) have lingering symptoms, untreated or inadequately treated in the past, that interfere with relationships. A detailed explanation of our conceptualization of this important problem area was presented in a previous paper.¹⁰

A question about the patient's perception of the primary problem was also added. This addition stems from the clinical observation that patient and therapist will occasionally disagree on the therapeutic task, requiring a compromise on the primary focus of treatment.

Method

IPT/IPSRT problem areas were assessed through a set of questions established as a structured interview guide. The questions were focused on the time period within the prior year of the interview, although the time frame can be adjusted to meet the aim of any treatment.

Training of all raters was performed before use of the IPARS-M, in which a videotaped IPT initial session was rated as a case example. Following the training, the first author interviewed all patients and completed the IPARS-M. Four experienced IPT research therapists, blind to the interviewer's assessment, rated the tapes using the same scale, yielding 18 ratings that could be paired for comparison.

The raters, three psychologists and two social workers, were trained in IPT at the University of Pittsburgh, had a mean of 17.6 years of clinical experience and 10.8 years of IPT experience. They were selected to participate in the study based on availability to participate in the training portion of the study and rating of all tapes as well as sound IPT experience. The agreement between the five raters was evaluated using a Kappa statistic^11,12 on the presence of each interpersonal problem area and separately on the selection of primary treatment focus if patients were to engage in IPT treatment.

Patients were recruited while they were at the clinic for medication visits. Only recovered patients that had not been treated with IPT or IPSRT prior to the interview were included.

Results

Eleven patients, six with bipolar disorder and five with unipolar disorder, from ongoing research studies at the University of Pittsburgh took part in the study. One patient denied any interpersonal problem and one tape was inaudible; therefore, data from nine patients, five bipolar and four unipolar, were analyzed for this study. Five patients were male and four female, with mean (± SD) age of 36.2 ± 14.9 years, ranging from 21 to 57 years. In terms of clinical characteristics, the median number of prior depressive episodes was four, and the median number of manic episodes for bipolar subjects was two, with the number of depressive or manic episodes ranging from one to six.

The paired ratings agreed on presence and absence of IPT/IPSRT problem areas in seven tapes and disagreed in two. In one case, the disagreement was over interpersonal deficit. In the other case, raters classified the same issue into different problem areas – role transition and interpersonal deficit. Grief was identified in five tapes, but only one case was classified as complicated bereavement. Role dispute was identified in six tapes. Role transition was identified in eight tapes (one disagreement). Interpersonal deficits were identified in six tapes (two disagreements). Loss of healthy self, the fifth problem area added to the scale, was identified in four tapes of the five bipolar patients interviewed.

The interrater reliability rates for identification of problem areas were 1.00 (grief), 0.77 (role dispute), 0.61 (role transition), 0.57 (interpersonal deficits) and 1.00 (loss of healthy self). Kappa for agreement on primary focus of treatment was 0.64. The problem area most frequently chosen as primary treatment focus was role transition (four agreements, and two disagreements), followed by loss of healthy self (two agreements). Raters agreed on role dispute as primary focus in one tape and disagreed in another. Grief was not classified as primary treatment focus by any rater and only one rater classified interpersonal deficits as the primary clinical focus.

In the field "main problem according to the patient," only one patient's perception of primary problem was different from that of the rater, yielding a kappa of 0.82 ± 0.16.

Discussion

This study confirms prior positive results using the original interpersonal problem area assessment scale. The modified version described here expanded the original study to include bipolar disorder without affecting reliability.

Substantial overall therapist agreement on problem area identification and choice of treatment foci suggests that formulation of the therapeutic task can be effectively grounded in the problem areas proposed in IPT and IPSRT.

The fifth problem area added to the scale, "loss of healthy self," had high prevalence in the bipolar patients studied, demonstrating its applicability and relevance to the assessment of interpersonal problems in the bipolar population. Four out of five bipolar patients were identified with this problem area and, in two cases, it would be the primary treatment focus. Frank described in her book on bipolar disorder treatment,² that most bipolar patients have a tendency to divide their lives in two: the person that he/she had been before developing the illness and the person he/she was now. She goes on to say that they are often frustrated by the limitations posed (or apparently posed) by the illness and frequently need to compromise their ideals to make certain that their needs, as determined by the illness, are met. Evidence for these statements is apparent in the high prevalence of this problem area in our sample and highlights the importance of identifying symbolic and real losses that, if left untreated, can diminish the patient's well-being, lead to non-adherence and impact social functioning.

Not surprisingly, interpersonal deficits had the lowest reliability score. This problem area is the least well-defined in IPT, causing discrepancies in its identification and treatment. Future research needs to focus on developing clearer definition and clinical strategies for different types of interpersonal deficits to avoid the risk of having this problem area overlooked and undertreated.

Almost perfect agreement between raters and patients on what would be the clinical focus if patients were to engage in psychotherapy strongly suggests that the IPT/IPSRT rationale would be well received and make personal sense to most patients even for those who are not undergoing treatment.

The major limitation of this study is the small sample size of patients and raters. It is not possible to know how well these results would generalize to larger samples of patients and raters or how well the scale would perform in them.

Clinical and research implications

Our data suggest that therapist adherence and competence can be evaluated and enhanced, at least for the initial phase of treatment, by use of the modified version of the IPARS-M.

Since treatment approach integrity is central to the interpretation of results from research on psychotherapy, adding such an instrument to the protocol would be a definite advance.

Although evaluating the validity of a structured questionnaire to identify problem areas was not part of this study's objective, its use proved efficacious to produce the information needed for a potential case formulation in a single interview. Further work needs to be carried out on the standardization of questionnaires that function as interview guides in IPT/IPSRT. Results of this pilot study suggest that its use might rapidly produce the information needed for therapeutic task formulation.

For clinical training purposes, such instruments might prove helpful training/supervising tools. Frank and Scocco developed a model of IPT training for clinicians, where Italian trainees, after identification of problem areas, filled out an Italian translation of the modified IPARS-M version described in this article.¹³ The authors concluded that the use of various forms of supervision, including a more structured tool like the IPARS-M, can accomplish positive results.

Finally, although outcome research studies are a necessary path to evidence-based treatment, they have not been sufficient for IPT/IPSRT dissemination into clinical practice, which depends not only on the information that these approaches work, but on how they work. Instruments like those discussed here might play an important role in IPT/IPSRT training and further increase in their use.

Acknowledgements

This work was supported by Grant #00/02306 from Fundação de Amparo à Pesquisa do Estado de São Paulo, FAPESP, (Ms. Andrade), Brazil, NIMH Grants 29618, 49115 and 30915 (Dr. Frank) and Departmental funds from the Western Psychiatric Institute and Clinic, Department of Psychiatry.

Submitted: February 15, 2008

Accepted: July 3, 2008

^{Clique aqui para apliar}

• 1. Klerman GL, Weissman MM, Rounsaville BJ, Chevron ES. Interpersonal Psychotherapy of Depression New York: Basic Books; 1984.
• 2. Frank E. Treating Bipolar Disorder: a Clinician's Guide to Interpersonal and Social Rhythm Therapy New York: Guilford Press; 2005.
• 3. Frank E, Kupfner DJ, Ehlers CL, Monk TH, Cornes C, Carter S, Frankel D. Interpersonal and social rhythm therapy for bipolar disorder: integrating interpersonal and behavioral approaches. Behav Ther 1994;17(7):143-9.
• 4. Jarrett RB, Rush AJ. Short-term psychotherapy of depressive disorders: current status and future directions. Psychiatry 1994;57(2):115-32.
• 5. Balslev Jorgensen M, Dam H, Bolwig TG. The efficacy of psychotherapy in non-bipolar depression: a review. Acta Psychiatr Scand 1998;98(1):1-13.
• 6. de Mello MF, de Jesus Mari J, Bacaltchuk J, Verdeli H, Neugebauer R.. A systematic review of research findings on the efficacy of interpersonal therapy for depressive disorders. Eur Arch Psychiatry Clinl Neurosci 2005;255(2):75-82.
• 7. Frank E, Kupfer DJ, Thase ME, Mallinger AG, Swartz HA, Fagiolini AM, Grochocinski V, Houck P, Scott J, Thompson W, Monk T. Two-year outcomes for interpersonal and social rhythm therapy in individuals with bipolar I disorder. Arch Gen Psychiatry 2005;62(9):996-1004.
• 8. Markowitz JC, Swartz HA. Case formulation in interpersonal psychotherapy of depression. In: Handbook of Psychotherapy Case Formulation New York: Guilford; 1997. p. 192-222.
• 9. Markowitz JC, Leon AC, Miller NL, Cherry S, Clougherty KF, Villalobos L. Rater agreement on interpersonal psychotherapy problem areas. J Psychother Pract Res 2000;9(3):131-5.
• 10. Scocco P, Barbieri I, Frank E. Interpersonal problem areas and onset of panic disorder. Psychopathology 2007;40:8-13.
• 11. Fleiss JL. Statistical methods for rates and Proportions New York: Wiley; 1981.
• 12. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-74.
• 13. Frank E, Scocco P. IPT training and supervision in Italy: a model for the clinical psychiatrist across geographic and linguistic boundaries. Inter Soc Interpersonal Psychotherapy Bull 2003;2(1).

Appendix

Publication Dates

• Publication in this collection
  17 Dec 2008
• Date of issue
  Dec 2008

History

• Accepted
  03 July 2008
• Received
  15 Feb 2008