User satisfaction with psychosocial healthcare services, Southern Brazil

Abstracts

OBJETIVO: Avaliar a satisfação dos usuários com o atendimento nos centros de atenção psicossocial. MÉTODOS: O estudo foi realizado em centros de atenção psicossocial dos estados: Paraná, Santa Catarina e Rio Grande do Sul, em 2006. A estratégia de investigação foi dividida em métodos quantitativo e qualitativo. Para avaliação quantitativa foram utilizados dados epidemiológicos do tipo transversal de amostra 1.162 usuários de 30 centros de atenção psicossocial. O instrumento utilizado para avaliar a satisfação dos usuários foi a Escala Brasileira de Avaliação da Satisfação (SATIS-BR), de 1 a 5 pontos. Para avaliação qualitativa foram realizados cinco estudos de caso, utilizando-se metodologia de avaliação de quarta geração. Os dados foram obtidos em observação de campo e entrevistas (entre dez e 13 usuários em cada campo, totalizando 57 usuários) e apresentados aos usuários em oficinas de validação e negociação. RESULTADOS: A escala SATIS-BR indicou avaliação positiva de todos os itens, com média de 4,4 (desvio-padrão, DP=0,4). A comunicação e o relacionamento com a equipe apresentaram média 4,5 (DP=0,5); o acesso a informações a partir da equipe apresentou média 4,8. A satisfação com o serviço apresentou a menor média, com 4,1; as condições gerais de instalação do serviço apresentaram média 3,9. No estudo qualitativo, o tratamento no serviço foi bom e o resultado satisfatório. O reposicionamento do usuário contribuiu para satisfação, assim como o acesso ao atendimento, a ruptura com o isolamento físico e social, o acolhimento de suas demandas e ajuda para organizar a vida. CONCLUSÕES: Os resultados complementares de ambas as estratégias do estudo mostram satisfação dos usuários com o atendimento nos centros estudados.

Serviços de Saúde Mental; Satisfação do Paciente; Avaliação de Serviços de Saúde; Estudos de Casos; Estudos Transversais

ORIGINAL ARTICLES

Luciane Prado Kantorski^I; Vanda da Rosa Jardim^I; Christine Wetzel^II; Agnes Olschowsky^II; Jacó Fernando Schneider^II; Rita Maria Heck^I; Valquíria de Lourdes Bielemann^I; Eda Schwartz^I; Valéria Cristina Christello Coimbra^I; André Luis Alves de Quevedo^I; Suélen dos Santos Saraiva^I

^IDepartamento de Enfermagem. Faculdade de Enfermagem e Obstetrícia. Universidade Federal de Pelotas. Pelotas, RS, Brasil

^IIDepartamento de Assistência e Orientação Profissional. Universidade Federal do Rio Grande do Sul. Pelotas, RS, Brasil

Correspondence

ABSTRACT

OBJECTIVE: To evaluate user satisfaction with psychosocial healthcare services.

METHODS: Qualitative and quantitative study conducted in psychosocial healthcare services in the states of Paraná, Santa Catarina and Rio Grande do Sul, Southern Brazil, in 2006. The study combined quantitative and qualitative methods. Quantitative epidemiological data from a cross-sectional study including 1,162 users of 30 psychosocial healthcare services was used. The instrument used to evaluate users' satisfaction was the Brazilian version of the WHO Users' Satisfaction Scale (SATIS-BR), a 1 to 5-point scale. Qualitative data was collected from five case studies using a fourth generation approach. Information was obtained from field observations and interviews (between ten and 13 users in each field, totaling 57 users) and presented to users in validation and negotiation workshops.

RESULTS: The SATIS-BR scale showed that users positively evaluated all items, overall mean 4.4 (SD=0.4). Communication and relationship with psychosocial healthcare services staff had mean 4.5 (SD=0.5), and access to information through staff had mean 4.8. Satisfaction with care service was the lowest, mean 4.1, and general service infrastructure had mean 3.9. The qualitative study revealed that, according to users, the quality of treatment provided was good and the outcome was satisfactory. Their new status, better access to services, ending of their physical and social isolation, response to their demands and helping them reorganize their lives - all contributed to users' satisfaction.

CONCLUSIONS: The complementary results of both study approaches showed that users are satisfied with care provided at the psychosocial healthcare services studied.

Descriptors: Mental Health Services. Patient Satisfaction. Health Services Evaluation. Case Studies. Cross-Sectional Studies.

RESUMEN

OBJETIVO: Evaluar la satisfacción de los usuarios con la atención en los centros de atención psicosocial.

MÉTODOS: El estudio fue realizado en centros de atención psicosocial de los estados del Sur de Brasil: Paraná, Santa Catarina y Rio Grande do Sul, en 2006. La estrategia de investigación fue dividida en métodos cuantitativo y cualitativo. Para evaluación cuantitativa fueron utilizados datos epidemiológicos del tipo transversal de muestra 1.162 usuarios de 30 centros de atención psicosocial. El instrumento utilizado para evaluar la satisfacción de los usuarios fue la Escala Brasilera de Evaluación de la Satisfacción (SATIS-BR), de 1 a 5 puntos. Para evaluación cualitativa fueron realizados cinco estudios de caso, utilizándose metodología de evaluación de cuarta generación. Los datos fueron obtenidos en observación de campo y entrevistas (entre diez y 13 usuarios en cada campo, totalizando 57 usuarios) y presentados a los usuarios en oficinas de validación y negociación.

RESULTADOS: La escala SATIS-BR indicó evaluación positiva de todos los ítems, con promedio de 4,4 (desviación-estándar, DP= 0,4). La comunicación y la relación con el equipo presentaron promedio 4,5 (DP= 0,5): el acceso a informaciones a partir del equipo presentó promedio 4,8. La satisfacción con el servicio presentó el menor promedio, con 4,1; las condiciones generales de instalación de servicio presentaron promedio 3,9. En el estudio cualitativo, el tratamiento en el servicio fue bueno y el resultado satisfactorio. La reubicación del usuario contribuyó para satisfacción, así como el acceso a la atención, la ruptura con el aislamiento físico y social, el acogimiento de sus demandas y ayuda para organizar la vida.

CONCLUSIONES: Los resultados complementarios de ambas estrategias de estudio muestran satisfacción de los usuarios con la atención en los centros estudiados.

Descriptores: Servicios de Salud Mental. Satisfacción del Paciente. Evaluación de Servicios de Salud. Estudios de Casos. Estudios Transversales.

INTRODUCTION

Mental and behavioral disorders cause a great deal of personal and social distress. They affect 450 million people worldwide and the World Health Organization (WHO) has estimated that nearly 25% of people develop at least one mental disorder during their lifetime. In addition to these conditions' economic and social cost, affected individuals are vulnerable to violations of their human rights, and social isolation, and have low quality of life and increased risk of death. The disability-adjusted life years (DALY) for neuropsychiatric disorders is reported as 13.8%. When all mental and behavioral disorders and related factors are combined they are as high as 33%, comprising four out of six main causes of disability.

The guiding principles of deinstitutionalization are reflected in a set of regulations that has been introduced by the Ministério da Saúde (Brazilian Ministry of Health) since the early 1990s and has had a great impact on the implementation of public policies in mental health.^b

A new challenge has been set. The Ministério da Saúde has clearly defined mental health policies focused on initiatives and public authorities have been working to assure the delivery of mental health care in accordance with the proposed guidelines. A new scenario for mental health research has been created prioritizing studies based on a systematic scientific approach to evaluate services established during the psychiatric reform process.

The objective of the present study was to evaluate users' satisfaction with attention provided at psychosocial care services.

METHODS

The present study was based on data from a larger study evaluating psychosocial care services (CAPS) in southern Brazil. There were evaluated CAPS facilities in the states of Paraná, Santa Catarina, and Rio Grande do Sul in 2006. The study comprised both quantitative and qualitative approaches.

The quantitative evaluation of quality of care can help guiding the formulation of more effective new strategies for service reestructuring.^7-9 This study consisted of three complementary sub studies: description of infrastructure of care facilities; description of the working process and organization of care in CAPS in the southern region; and evaluation of outcomes in mental health care.

The sample studied included coordinators, health workers, family members, and users from 30 facilities (CAPS I and CAPS II) from three different states based on the proportion of services per state.

Three cities were selected to participate in the study in the state of Paraná: Cianorte (CAPS I), and Curitiba and Francisco Beltrão (both CAPS II). Nine cities were selected in Santa Catharina: Xaxim, Timbó, Rio do Sul, Orleans, and Içara (CAPS I), and Caçador, Joinville, Florianópolis, and Criciúma (CAPS II). Nine CAPS I facilities were selected in the state of Rio Grande do Sul: Santo Ângelo, Panambi, Santiago, Bento Gonçalves, Triunfo, Parobé, São Sepé, Santana do Livramento, Capão do Leão; and nine CAPS II facilities: Carazinho, Passo Fundo, Alegrete, Bagé, Santa Maria, Santa Cruz do Sul, Rio Grande, Esteio, and Porto Alegre.

Users and family members were drawn and those included in the sample were interviewed at home and at CAPS facilities.

From the perspective of a potential effect of care model change, the estimate of the sample size was based on the following outcomes: user satisfaction; health status and autonomy, social inclusion and empowerment. The sample size was estimated for an alpha error of 5% and 95% power using Epi Info 6.04. Given the scarcity of studies from this mental health perspective and the fact that the literature has shown different measures and indicators of variability, there were used, values with the smallest likelihood of error. To the highest size value estimated was added 30% for controlling for confounders and losses. Differences in the level of complexity among the three models of care found in CAPS were taken into consideration in the sample. A total of 1,200 users and family members were included in the study.

Given the different proportion of CAPS by state, and due to logistic issues, there was established that 40 users and 40 family members would be interviewed at each facility for a total of 30 CAPS.

The group of 40 users selected was proportionally categorized by complexity of care into intensive (12 subjects), non-intensive (12), and semi-intensive (16).

The study questionnaires were developed and validated according to specific mental care regulations. The instruments were pre-tested in services that did not participate in the study. User satisfaction was evaluated using the Brazilian version of the Users' Satisfaction Scale (SATIS-BR). This scale was developed by the WHO Division of Mental Health and validated in Brazil. It consists of a 5-point Likert scale, where 5 means the greatest satisfaction and 1 means the lowest satisfation.^3-5

Data were collected by 14 previously trained interviewers and their work was supervised by two supervisors.

After data collection, the questionnaires were then coded and doubled entered independently into a database using Epi Info. Both entries were compared and corrected for major errors and inconsistencies. All analyses were carried out using Stata program.

Quality was controlled during the application of each data collection instrument. Data was checked by all interviewers at the end of the interview, and questionnaires were reviewed by the supervisors. In addition, 5% of interviews were replicated.

The qualitative study comprised a fourth generation evaluation (FGE), which was constructivist, and responsive technique with a hermeneutic dialectic approach. The FGE, developed by Guba & Lincoln^10-12 and adapted by Wetzel (2005),

Five CAPS were selected by intention based on data obtained during the quantitative evaluation and their adequacy to the regulations of Decree No. 336/2002.^b There were also taken into consideration length of time in operation of a service, length of service of health workers, and availability of stakeholders to be involved.

The qualitative step of the study was conducted as a case study¹⁸ including field observations (between 282 and 650 hours in each field) and interviews.

There were interviewed between ten and 13 users in each field (total of 57 users), between ten and 14 family members in each field (total of 60 family members), and between ten and 26 health workers in each field (total of 88 health workers).

After data was collected in the interviews, an interim analysis was conducted to develop workshops for information validation. The analyzed data was presented to the applicable groups so they could have access to all information and to offer them an opportunity for making changes or confirming it as reliable. Negotiation was carried out in a group and all interviewees of a given circle were invited to participate.

Based on units of information extracted from discourses, markers for evaluation were defined to guide analyses. A marker is a given category abstracted from empirical data, which has an explanatory feature indicating an evaluation factor. For example, the marker for outcome indicating qualitative aspects to be evaluated was the mental health outcome, which enabled to understand subjects' satisfaction with service and their new status to respond to daily demands. Markers followed the conceptual principles used in the quantitative step of evaluation focusing on facility infrastructure, process and results.^7-9 From this wider perspective, the following markers were subdivided: facility infrastructure - environment; process - activities such as treatment support, care team, characteristics and organization of work process, service flexibility, family involvement; result - psychosocial care outcome.

The study was approved by the Research Ethics Committee of Faculdade de Medicina da Universidade Federal de Pelotas (Protocol 074/05 of 11 November 2005). All subjects signed a free informed consent agreeing to participate in the study.

RESULTS

Quantitative study

There were 38 losses of a total of 1.200 users selected to the sample. Of those users included in the sample, 61,4% attended CAPS in Rio Grande do Sul, which is consistent with greater proportion of services in that state.

Most users were females (63.9%) with mean age of 42 years (standard deviation [SD]=12.3) and 70.9% self-referred as white skin color. Mean income in the previous month was R$ 338.02 while per capita family income was R$ 264.16.

The medical diagnosis was reported by 97% of users; 34.3% reported major depressive disorder, 28.3% psychosis, and 8.9% schizophrenia.

User satisfaction was positively evaluated for all items, with mean score of 4.4 (SD= 0.4). The results for each item of the scale are shown in Table 1.

Satisfaction with service had the lowest mean score (4.1); general facility infrastructure had mean score of 3.9, and "communication and relationship with the health team" had mean score of 4.5.

The highest means were seen for access to information and respect and dignity (mean scores of 4.8). Items with the lowest means were general facility infrastructure (mean score of 3.9); appearance and comfort, and users' involvement in treatment decision making (mean scores of 4.1).

The positive evaluation supported the decision to return to the service to seek help for 95.9% of the users studied and the recommendation of the service to a friend or family for 96.9% of them. Reasons for recommending the service were good quality of attention, support, and help provided for 36.7% of the users; good relationship with providers and best treatment option for 23.6%; and treatment and medication for 21.2%.

Although users have positively evaluated services, 58.5% believed that CAPS required improvement.

The most common factors that required improvement were: facility infrastructure (21.8%) users mentioned the need for more physical space (care rooms, workshop rooms, yard, sport court), renovations, better hygiene conditions (especially in the restrooms), aesthetically customized spaces (distinctive pleasant and colorful rooms); providers (19%) - better staffing and qualification; activities (15.4%) - greater variety of workshops, tea groups, physical activities, family groups, lounges for meeting people, opportunities for working and selling workshop products; attention (14.6%) - users referred to the need for reducing provider turnover, increasing the number of visits, having medication available at the facilities, treating substance abuse, providing care to more people and having more discussion groups; resources (10.5%) - lack of funds, supplies, and support from local administrations; meeting (8.1%) - increasing interaction between staff and users; medication (5%) - drug dispensing at CAPS and regular provision of drugs.

Qualitative study

According to users, treatment provided at CAPS was good and the outcome was satisfactory (Table 2). As a result, they gained more autonomy, had less crises (reduced frequency and intensity), had more opportunities of social interaction, socialization, and of enriching their daily life which is not restricted to the experience of the mental disorder.

Users reported that attention provided at CAPS facilities was qualitatively better than that provided by the Sistema Único de Saúde (SUS - Brazilian National Health Service). Another factor contributing to their satisfaction was their new status, as they had an opportunity to acquire new skills during workshops, which made them gain more autonomy and be pro-active, teaching the new skills learned to other users. Several factors that qualitatively contributed to their satisfaction were: ending of their social and physical isolation, attending a service where staff are familiar with them (they know their name, family, area), having their demands met (there is a professional who can be contacted even during weekends and off-hours) and having support to reorganize their life (dealing with internal conflicts, family members, community and simple tasks such as managing their benefits over one-month period).

Although users were qualitatively satisfied with the service provided, they referred to the existing poor conditions such lack of materials for workshops, irregular provision of drugs (except for Case 2), limited physical space and activities, understaffing, low qualification and lack of support to CAPS health workers, few activity options, little involvement in management decisions and difficult referral to other local mental health services (especially for admission and basic care).

DISCUSSION

The higher concentration of users and services in the state of Rio Grande do Sul is consistent with data from the Ministério da Saúde that showed that this state has 0.7 CAPS facilities per 100,000 inhabitants, while the state of Santa Catarina and Paraná have 0.6 and 0.4, respectively. According to the Ministério da Saúde criteria, good to fairly good coverage is seen with higher than 0.5 CAPS facilities per 100,000 inhabitants and very low coverage is seen when it is below 0.2.

Users were on average older than the general population by about ten years⁶ and younger also by ten years to those subjects of a study including users of home care usually provided to patients discharged after prolonged psychiatric admissions.¹³ Mean income found in the sample shows the social vulnerability of the population studied and the exclusion of those suffering from mental conditions.^14,17 On the other hand, the medical diagnoses reported indicate that CAPS generally provide care to those suffering from more severe conditions, which is consistent with the policy guidelines of the Brazilian psychiatric reform.^b

The SATIS-BR scale revealed users' high satisfaction with CAPS in southern Brazil. Another study using the same scale to evaluate home care found mean score of 4.52 (SD= 0.48).¹³

The findings regarding users' relationship with health workers are similar to those reported in another study.¹⁶ The major weakness of services is regarding the infrastructure of facilities, as pointed out in other studies.^3,15,16

In the present study, there may be evaluation bias resulting from the users' difficulty of expressing their dissatisfaction or not expressing their gratitude.² This is true especially for public services in developing countries. In spite of that, given the approach used, the results indicate that CAPS have been a positive reference for mental health care. The services evaluated represent a valuable substitute to the reasoning of exclusion of psychiatric hospitals.

The psychosocial care approach aims at helping patients to acquire a new status. Rather than suffering the effects of conflicts and psychic contradictions, it allows patients to identify themselves as an involved agent and to act as a change agent.⁶ The evaluation of the present study required a qualitative approach and each one of the 57 users studied was empowered over the course of the evaluation process, showing aspects related to psychosocial care outcome and indicating what could be improved in CAPS. To bring users to the evaluation scenario is challenging and essential to emancipative constructivist processes as proposed in our study. Users' validation and negotiation, as part of the FGE, reinforced the importance of evaluations that create an opportunity for users to express themselves freely, acquire new status, and be empowered.

Freedom, before being an element of thought or will, was the element that enabled human beings to act and go out and meet people, exercising their power of speech and action. In addition, freedom required a public setting where people could meet.¹ The present study prioritized in its construction process the creation of public settings where each individual involved in mental health care at CAPS could be included through words and actions in the evaluation. It is expected that the present study can be a reference to be applied in different scenarios such as academic settings, public policies and daily operation of CAPS.

By approaching quantitative and qualitative techniques, there were found convergent and divergent aspects, particularly in regard to users' satisfaction with CAPS. There was an agreement between factors with the highest scores in the SATIS-BR scale and themes that described reasons for satisfaction in the qualitative study. Despite users' significant positive evaluation, they pointed out to aspects that required improvement and adjustments.

As pointed out by users, we emphasize the need for improving facility infrastructure, and providing more supplies and medications as well as better training and qualification to health workers and CAPS staffing. Users also pointed to the need of offering a wider variety of activities and care at CAPS. Both studies also revealed issues related to management, especially concerning financing, user involvement in management decisions as well as service organization and referrals.

REFERENCES

• 1. Arendt H. Entre o passado e o futuro. São Paulo: Perspectiva; 1979.
• 2. Avis M, Bond M, Arthur A. Questioning patient satisfaction: an empirical investigation in two outpatient clinics. J Adv Nurs. 1997;44(1):85-92.
• 3. Bandeira M, Pitta AMF, Mercier C. Escalas da OMS de avaliação da satisfação e da sobrecarga em serviços de saúde mental: qualidades psicométricas da versão brasileira. J Bras Psiquitr.1999;48(6):233-44.
• 4. Bandeira M, Pitta AMF, Mercier C. Escala de avaliação da satisfação de familiares com os serviços de saúde mental: SATIS-BR. J Bras Psiquiatr. 2000;49(8):293-300.
• 5. Bandeira M, Mercier C, Perreault M, Libério MMA, Pitta AMF. Escala de avaliação da satisfação de familiares com os serviços de saúde mental: SATIS-BR. J Bras Psiquiatr. 2002;51(3):153-66.
• 6. Costa-Rosa A. O modo psicossocial: um paradigma das práticas substitutivas ao modo asilar. In: Amarante P, organizador. Ensaios: subjetividade, saúde mental, sociedade. Rio e Janeiro: Editora Fiocruz; 2000.
• 7. Donabedian A. The definition of quality and approaches to its assessment. Explorations in quality assessment and monitoring. Ann Arbor: Health Administration Press; 1980. v.1.
• 8. Donabedian A. La calidad de la atención medica: definición y métodos de evaluación. Ediciones científicas. México: Prensa Medica Mexicana; 1984.
• 9. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743-8. DOI: 10.1001/jama.260.12.1743
• 10. Guba E, Lincoln Y. Effective Evaluation. Improving the usefulness of evaluation results throug responsive naturalistic approaches. San Francisco: Jossey Bass Publishers;1985.
• 11. Guba E, Lincoln Y. Effective evaluation. San Francisco: Jossey Bass Publishers;1998.
• 12. Guba E, Lincoln Y. Fourth generation evaluation. Newbury Park: Sage Publications; 1989.
• 13. Jaegger RC, Guitton AP, Lyrio JM, Santos MM, Freitas RC, Gonçalves SR, et al. A experiência de morar fora: avaliação da satisfação de usuários em um serviço de saúde mental. Cad Saude Coletiva. 2004;12(1):27-39.
• 14. Ludermir AB, Melo Filho DA. Condições de vida e estrutura ocupacional associadas a transtornos mentais comuns. Rev Saude Publica. 2002;36(2):213-21. DOI: 10.1590/S0034-89102002000200014
• 15. Pitta AMF, Silva Filho JF, Souza GW, Lancman S, Kinoshita RT, Cavalcanti MT, et al. Determinantes da qualidade de serviços de saúde mental em municípios brasileiros: estudo da satisfação com os resultados das atividades desenvolvidas por pacientes, familiares e trabalhadores dos serviços. J. Bras Psiquiatr. 1995;44(9):441-52.
• 16. Silva Filho JF, Cavalcanti MT, Cadilhe GM, Vilaça AT, Avancini E, Machado HO, et al. Avaliação da qualidade de serviços de saúde mental no município de Niterói - RJ. A satisfação dos usuários como critério avaliador. J Bras Psiquiatr.1996;45(7):393-402.
• 17. Wohlfarth T. Socioeconomic inequality and psychopathology: are socioeconomic status and social class interchangeable. Soc Sci Med. 1997;45(3):399-410. DOI: 10.1016/S0277-9536(96)00355-3
• 18. Yin RK. Estudo de caso: planejamento e métodos. Trad. de D Grassi. 3. ed. Porto Alegre: Bookman; 2005.

User satisfaction with psychosocial healthcare services, Southern Brazil

Satisfacción de los usuarios de los centros de atención psicosocial de la región Sur de Brasil

ª

In Brazil, despite its tradition of institutionalizing patients that has contributed to stigmatization and social isolation of those suffering from mental conditions, starting from 1970s the movement for psychiatric reform gained momentum advocating comprehensive mental health care with an empowering approach. The psychiatric reform has been brought about by several different movements and conceptual lines.

^c

guided the theoretical-methodological approach of the study. The instruments of data collection were interviews with health workers, users, and family members (defined as the stakeholding groups to form the hermeneutic dialectic circle) and field observations (prior ethnography).

^d

Most CAPS users are women, which corroborates that seen in the sample used for SATIS-BR scale validation where females accounted for 70%.⁴ Similarly, other studies investigating health services reported higher proportion of women among users.¹⁶

Publication Dates

• Publication in this collection
  04 Aug 2009
• Date of issue
  Aug 2009

History

• Accepted
  04 June 2009
• Reviewed
  10 May 2009
• Received
  19 Nov 2008