Reliability of the Brazilian version of the Camberwell Assessment of Needs (CAN) in first-episode psychosis cases in São Paulo, Brazil

Confiabilidade da versão brasileira da escala Camberwell de Avaliação de Necessidades (CAN) em casos incidentes de psicoses em São Paulo, Brasil

BRIEF REPORT

Reliability of the Brazilian version of the Camberwell Assessment of Needs (CAN) in first-episode psychosis cases in São Paulo, Brazil

Confiabilidade da versão brasileira da escala Camberwell de Avaliação de Necessidades (CAN) em casos incidentes de psicoses em São Paulo, Brasil

Ana Cristina Belizia Schlithler^I; Marcia Scazufca^I, ^II; Geraldo Busatto^I; Letícia Maria Silva Coutinho^{II, III}; Paulo Rossi Menezes^{II, III}

^IDepartment of Psychiatry, Medical School, Universidade de São Paulo (USP), São Paulo (SP), Brazil

^IISection of Epidemiology, University Hospital, Universidade de São Paulo (USP), São Paulo (SP), Brazil

^IIIDepartment of Preventive Medicine, Medical School, Universidade de São Paulo (USP), São Paulo (SP), Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: To translate and adapt the Brazilian version of the Camberwell Assessment of Needs schedule for research, and to assess its inter-rater reliability in a sample of first-episode psychosis cases in São Paulo, Brazil.

METHOD: A sub-sample of participants included in a study of first-episode psychosis in São Paulo was assessed utilizing the 22 domains of the Brazilian version of the Camberwell Assessment of Needs. The Camberwell Assessment of Needs was applied, tape-recorded and rated by research assistants. An independent rater gave scores to the same participants, based on the recorded interviews. The kappa coefficient and the intraclass correlation coefficient were used to assess inter-rater reliability.

RESULTS: Fifty-two subjects were included. For seven domains the inter-rater reliability was almost perfect to total (range: 0.81 to 1.0). The domains with the lowest agreement were transport and benefits. The inter-rater reliability for the total number of needs was excellent (ICC = 0.95).

CONCLUSIONS: The inter-rater reliability of the Camberwell Assessment of Needs was similar to what has been found in previous studies. The Camberwell Assessment of Needs showed to be easy to use and reliable with first-episode psychosis individuals. The use of standardized instruments to assess needs of care in Brazil will contribute to the assessment of the effectiveness of treatment and to the planning of individualized care for individuals with mental illnesses.

Descriptors: Needs assessment; Mental disorders; Psychotic disorders; Reproducibility of results; Psychiatric status rating scale

RESUMO

OBJETIVO: Traduzir e adaptar a versão para pesquisa da escala Camberwell de Avaliação de Necessidades e avaliar sua confiabilidade entre avaliadores em uma amostra de casos incidentes de psicoses funcionais em São Paulo, Brasil.

MÉTODO: Uma amostra de participantes incluídos em um estudo de casos incidentes de psicoses funcionais em São Paulo foi avaliada utilizando-se os 22 domínios da versão brasileira da escala Camberwell de Avaliação de Necessidades. A escala Camberwell de Avaliação de Necessidades foi aplicada, gravada e pontuada por assistentes de pesquisa. Um avaliador independente pontuou os mesmos participantes, baseando-se nas gravações das entrevistas. O coeficiente kappa e o coeficiente de correlação intraclasse foram utilizados para avaliar a confiabilidade entre avaliadores.

RESULTADOS: Cinqüenta e dois indivíduos foram incluídos. Para sete domínios, a confiabilidade entre avaliadores foi de quase perfeita a total (k = 0,81 a 1,00). Os domínios com a menor concordância entre avaliadores foram transporte e benefícios. O coeficiente de correlação intraclasse para o total de necessidades foi excelente

(ICC = 0,95).

CONCLUSÕES: A confiabilidade entre avaliadores da escala Camberwell de Avaliação de Necessidades foi similar aos resultados de estudos anteriores. A escala Camberwell de Avaliação de Necessidades mostrou-se confiável e fácil de aplicar em indivíduos com psicoses incidentes. O uso de instrumentos padronizados para a avaliação de necessidades de cuidado no Brasil contribuirá para a avaliação da efetividade de tratamentos e para o planejamento de cuidados personalizados para indivíduos com transtornos mentais.

Descritores: Determinação de necessidades de cuidados de saúde; Transtornos mentais; Transtornos psicóticos; Reprodutibilidade de resultados; Escalas de graduação psiquiátrica

Introduction

Individuals with psychotic illnesses usually face changes and limitations in their daily life activities. Their needs are complex and encompass a range of types and levels of formal and informal care.¹ In several countries, the systematic assessment of needs related to the living conditions, routine tasks, social relationships and disabilities associated with the psychotic illness is considered essential for service planning and for the establishment of treatment goals.^2-3 The Camberwell Assessment of Needs (CAN) was developed by the Health Service Research Department of the Institute of Psychiatry of London (UK), aiming to evaluate a wide range of needs of people with severe mental illnesses, and has been one of the schedules most commonly used worldwide. Studies conducted in countries with different languages and cultures have shown that the CAN is a valid, reliable and easily-applied instrument.^1,4-6

In Brazil, the assessment of needs is still limited. The present study aimed to assess the inter-rater reliability of the Brazilian version of the CAN for research (CAN-R), in a sample of first-episode psychosis cases in São Paulo, Brazil.

Method

1. Sample

The present study is part of an epidemiological investigation entitled "Brazilian First Contact Psychosis Study". Eligible individuals were those who had had a first contact with any mental health service in their lives due to a psychotic episode, according to DSM-IV⁷ criteria. They were aged between 18 and 64 years old, and had lived for at least six months in pre-defined areas of São Paulo. Subjects included in the reliability study were selected from the sample of the incidence study, between May 2003 and February 2005. Selection was based on the availability of the tape-recorder for the interviewer in the day scheduled for the assessment of the incidence study, having no relationship with the participants' characteristics.

2. Research instruments

The CAN assesses needs in 22 domains, encompassing several areas of care (Table 1). Each domain is divided into four sections. Section 1 assesses whether there is a need in that domain, and is rated 0 (no need) if there is no problem in the domain, 1 (met need) if the problem is present but effective help has been provided, 2 (unmet need) if the problem is serious and help is not being provided or is not effective, and 9 (not known) if the interviewee does not know or does not want to answer the question. The other three sections are assessed only if a need is present in the domain. Section 2 assesses the amount of help given by informal sources, such as friends and relatives. Section 3 assesses the amount of help given by formal services. Section 4 assesses whether users believe they are getting the right type of help and whether they are satisfied with the amount of help that they are receiving.² For each rating, anchor points and guidelines are provided. The CAN allows the assessment of needs according to the client's point of view or according to the perception of the mental health professional involved with his/her treatment. ^{Appendix 1} shows the questionnaire for the "accommodation" domain. The complete Brazilian version of the CAN is available, under request, from the corresponding author of this paper.

The present study assessed the reliability of section 1 of the CAN. Ratings for each of the 22 domains were given according to the participants' point of view. The period assessed was the month prior to the interview.

3. Procedures and ethical aspects

The original version of the CAN was translated and adapted into Brazilian Portuguese by a group of bilingual mental health professionals, after permission from the authors of the schedule.¹ The applicability of the CAN was tested in a pilot study with 21 participants. As a consequence, further minor adaptations were carried out, to improve the understanding of some questions.

Training of the group of research assistants started with the discussion of the schedule and its application. Research assistants were all mental health professionals with clinical experience (psychiatrists, psychologists, nurses, occupational therapists, social workers). Then, each research assistant conducted an interview with a patient with psychosis, while the remaining research assistants established their own ratings. The research team then discussed problems related to the application and the ratings.

The assessment of needs was accomplished by the research assistants, when participants were assessed with the protocol of the study of incident psychosis in São Paulo. Most interviews were performed at patients' homes. After that, one of the investigators (ACBS) carried out the second assessment of all interviews, being blind to the research assistants' ratings. All participants were asked to sign an informed consent, with a special authorization for tape-recording the interview.

This study was approved by the Comissão de Ética para Análise de Projetos de Pesquisa – (CAPPesq) of the clinical direction of the Hospital das Clínicas and the School of Medicine of the Universidade de São Paulo (Research protocol no.985/00).

4. Statistical analysis

The kappa coefficient was used to measure inter-rater agreement for the ratings of each one of the 22 domains, as assessed by the research assistants and by the independent investigator, using scores 0, 1, 2, 9. The intraclass correlation coefficient (ICC) was used to measure the agreement for the total number of needs.⁸ For each reliability estimate 95% Confidence Intervals were calculated. The results of the coefficients were interpreted using the criteria of Landis & Kock,⁹ whereby an agreement between 0.21 and 0.40 is considered discrete, between 0.41 and 0.60, moderate, between 0.61 and 0.80, substantial, and between 0.81 or plus, almost perfect.

Results

Fifty-two subjects were included in the study, of whom 27 (52%) were women. The mean age of participants was 32.7 years (s.d: 10.6). Thirty three subjects met DSM-IV criteria for schizophrenia or other disorders of the schizophrenia spectrum. Socio-demographic and clinical characteristics of participants of the present study were similar to those of the sample of the study of incident psychosis in São Paulo.

Table 1 shows the comparison of the ratings of each of the 22 domains of the CAN performed by the research assistants and by the independent investigator. There was total agreement (k = 1.00) in the "looking after home" and "telephone" domains. The agreement was almost perfect (k = 0.82-0.90) for the "physical health", "psychological distress", "safety to self", "safety to others", "intimate relationships", "sexual expression", and "basic education" domains. For the domains "accommodation", "food", "self-care", "daytime activity", "psychotic symptoms", "information", "alcohol", "drugs", "company", and "money", the agreement was substantial. The domains with the lowest agreement were "transport" and "benefits". The ICC for the total number of needs was 0.95 (95%CI: 0.90-1.00).

Discussion

To the best of our knowledge, this is the first study that evaluated the reliability of the CAN in a sample of subjects with first-contact psychosis. Although the number of participants included in the present study is similar to previous reliability studies of the CAN, which included patients with long duration of psychotic symptoms,^1,10-11 the data only allowed the assessment of the reliability of the first section of each domain. For several domains, the frequency of positive answers for the presence of needs (met and unmet) was low, possibly because of the clinical characteristics of the participants of the study. As a result, the remaining sections were only applied to few participants. The low frequency of positive answers also decreased the precision of the reliability estimates. A larger sample would allow the assessment of the reliability for all sections of each domain and would increase the precision of the reliability estimates for domains with low frequency of positive answers.¹² The use of tape-recorded interviews in reliability studies can also affect the agreement between raters, because the independent investigator cannot observe the presentation, body language and physical and mental state of study participants. However, the good results of the present study suggest that inter-rater agreement might be even better if both raters observe the interview simultaneously.

The results of the present study are similar to results of other studies of inter-rater reliability that established the scores according to the users' point of view.^1,10-13 However, for some domains, results are conflicting. In the study by Phelan et al., the worst agreement was in the domain "safety to others" (k = 0.65), while in the present study the agreement for this domain was almost perfect (k = 0.83).¹ This difference may have been due to the small number of participants who reported problems in this area in the present study. The agreement for "transport" and "benefits" was very high in the study by Phelan et al. (k = 0.93 for both domains), and in Yeh's study (k = 1.00 for both domains), whereas in the present study these were the domains with the worst agreement.^1,13 One possibility for such low agreement in these domains may be related to the study participants' lack of information about eligibility criteria for receiving transport and other welfare benefits, which may have led to difficulties in answering these questions and consequently for interviewers to understand the answers. The kappa coefficient for the domain "childcare" was zero, because the agreement observed was the same as the agreement expected by chance alone. Only a few participants answered that domain, because participants with children aged 18 years or over were rated without needs.

The CAN showed good applicability and reliability according to the clients' point of view, in first-episode psychosis. The use of the CAN in individuals with mental disorders, in different levels of mental health care in varied communities, can help improve the understanding of the determinants of needs of care, and guide the planning of services and programs. The use of this instrument can also contribute to assess the effectiveness of interventions and to tailor individualized care.

Acknowledgements

We are grateful to all research assistants for helping with data collection. We would like to thank Profs. R. Murray and P. McGuire for their participation in the first-contact psychosis study in Brazil, and the Wellcome Trust and CNPq for their financial support.

Submitted: May 18, 2006

Accepted: October 10, 2006

Financing: the Wellcome Trust, UK. MS and PRM are partially funded by the Conselho Nacional de Desenvolvimento Tecnológico (CNPq-Brazil).

Conflict of interests: None

Department of Psychiatry, Medical School, Universidade de São Paulo (USP), São Paulo (SP), Brazil

• 1 Phelan M, Slade M, Thornicroft G, Dunn G, Holloway F, Wikes T, Strathdee G, Loftus L, McCrone P, Hayward P. The Camberwell Assessment of Need. The validity and reliability of an instrument to assess the needs of people with severe mental illness. Br J Psychiatry 1995;167(5):589-95.
• 2 Slade M, Thornicroft G, Loftus L, Phelan M, Wykes T. Camberwell Assessment of Need. London, UK: Gaskell; 1999.
• 3 Joska J, Flisher AJ. The assessment of need for mental health services. Soc Psychiatry Psychiatr Epidemiol. 2005;40(7):529-39.
• 4 Ruggeri M, Lasalvia A, Nicolaou S, Tansella M. The Italian version of the Camberwell Assessment of Needs (CAN), an interview for the identification of needs of care. Epidemiol Pschiatr Soc 1999;8(2):135-67.
• 5 Hansson L, Bjorkman T, Svensson B. The assessment of needs in psychiatric patients. Interrater reliability of the Swedish version of the Camberwell Assessment of Needs instrument and results from a cross-sectional study. Acta Psychiatr Scand1995;92:285-93.
• 6 Kilian R, Bernert S, Matschinger H, Mory C, Roick C, Angermeyer MC. The standardized assessment of the need for treatment and support in severe mental illness: development and testing of the German version of the Camberwell Assessment of Need EU. Psychiatr Prax 2001;28(Suppl 2):S79-83.
• ⁷
  7 American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM IV). 4th ed. Washington (DC): American Psychiatric Press; 1994.
• 8 Bartko JJ, Carpenter WT Jr. On the methods and theory of reliability. J Nerv Ment Dis 1976;163(5):307-17.
• 9 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-74.
• 10 McCrone P, Leese M, Thornicroft G, Schene A, Knundsen H, Vazquez-Barquero J, Lasalvia A, Padfield S, White IR, Griffiths G. Reliability of the Camberwell Assessment of Needs European version. EPSILON Study 6. European psychiatry services: input linked to outcome domains and needs. Br J Psychiatry 2000;Suppl 39:S34-40.
• 11 Arvidsson H. Test-retest reliability of the Camberwell Assessment of Need (CAN). Nord J Psychiatry 2003;57(4):279-83.
• 12 Saito Y, Sozu T, Hamada C, Yoshimura I. Effective number of subjects and number of raters for inter-rater reliability studies. Stat Med 2006;25(9):1547-60.
• 13 Yeh HS, Luh RL, Liu HJ, Lee YC, Slade M. Reliability of the Camberwell assessment of need (Chinese version) for patients with schizophrenia at a day-care center of Taiwan. Soc Psychiatry Psychiatr Epidemiol. 2006;41(1):75-80.

Appendix 1

Publication Dates

• Publication in this collection
  27 Nov 2006
• Date of issue
  June 2007

History

• Received
  18 May 2006
• Accepted
  10 Oct 2006