Psychosocial Care Network: development and validation of a multidimensional instrument to assess implementation (IMAI-RAPS)

Abstract

This is a methodological study for the development and validation of the Multidimensional Instrument for Evaluating of the Implementation of Psychosocial Care Network (IMAI-RAPS) in Minas Gerais (MG)/Brazil. The study was carried out in three stages: evaluability study, development of the IMAIRAPS, application of the Delphi Technique for content and appearance validation of the questions. The analysis of official documents, literature review and a structured engagement with program members were carried out to clarify its operationalization and focus on the central aspects to be evaluated. A theoreticallogical model of RAPS was built according to the Donabedian triad: structure, process and result and organized into: Minimum Units (Mental Health Care and Psychosocial Rehabilitation), Connectivity (Network Articulation), Integration (Governance and Management of the Care), Normativity (Mental Health Policy and Participation and Social Control), Subjectivity and Structure (Services, Logistics System and Health Education). The IMAI-RAPS was derived from this model, which was validated by 44 experts in the field, indicating the approach of relevant, useful and viable questions for evaluating the structure and process of implementing the program in MG. The use of the Delphi Technique made it possible for the developed products to be marked out by Psychosocial Care Network scholars or professionals from different regions of the country, increasing the analytical power of the tool.

Keywords:
Mental Health; Health Evaluation; National Health Systems; Health Policy

Resumo

Trata-se de um estudo metodológico para desenvolvimento e validação do Instrumento Multidimensional para Avaliação da Implantação da RAPS (IMAI-RAPS) em Minas Gerais (MG)/Brasil. O estudo foi executado em três etapas: estudo de avaliabilidade, desenvolvimento do IMAI-RAPS, aplicação da Técnica Delphi para validação de conteúdo e aparência das questões. Foram realizados a análise de documentos oficiais, revisão da literatura e um engajamento estruturado com membros do programa para esclarecer sua operacionalização e focalizar os aspectos centrais a serem avaliados. Um modelo teórico-lógico da RAPS foi construído de acordo com a tríade donabediana: estrutura, processo e resultado e organizado em: Unidades Mínimas (Assistência à Saúde Mental e Reabilitação Psicossocial), Conectividade (Articulação da Rede), Integração (Governança e Gestão do Cuidado), Normatividade (Política de Saúde Mental e Participação e Controle Social), Subjetividade e Estrutura (Serviços, Sistema Logístico e Educação em Saúde). Desse modelo derivou-se o IMAI-RAPS que foi validado por 44 experts da área indicando a abordagem de questões relevantes, úteis e viáveis para avaliação da estrutura e processo de implantação do programa em MG. A utilização da Técnica Delphi possibilitou que os produtos desenvolvidos fossem balizados por estudiosos ou profissionais da RAPS de diversas regiões do país aumentando o poder analítico da ferramenta.

Palavras-Chave:
Saúde Mental; Avaliação em Saúde; Sistemas Nacionais de Saúde; Política de Saúde

Introduction

Mental health represents one of the highest costs for global public health, with an estimated 13% of the global burden of diseases due to mental and neurological disorders and/or disorders resulting from abuse of alcohol or other drugs (VIGO; THORNICROFT; ATUN, 2016). This high – and increasing – global burden is associated with several factors, such as the widespread prevalence, early onset, and chronic course of these disorders, as well as the considerable current gap in treatment (KOHN et al., 2018). Thus, there is an urgent need to strengthen the capacity of health systems to respond to these disorders effectively (ZHOU et al., 2018).

Latin America has made significant progress in improving mental health services in recent decades. These improvements, following changes made earlier in wealthier countries, centered on replacing a focus on hospital care with healthcare structured on community services integrated into primary healthcare (PHC) and emphasizing the protection of human rights and the agency of the users (CALDAS DE ALMEIDA, 2013; PATEL et al., 2018). In Brazil, a national mental health policy was legislated in 2001, driven by the Psychiatric Reform movement, which advocated for a community model of care. Since then, there have been several changes in the implementation of territorial services and clinical practices based on psychosocial care, leading the Brazilian experience to occupy a prominent place in the field of global mental health (JACOB et al., 2007; ALMEIDA, 2019).

In 2010, in an effort to further develop decentralization and regionalization strategies to reduce the fragmentation of care in the Brazilian unified health system (Sistema Único de Saúde – SUS), the healthcare networks were restructured, integrating actions and services across several levels of complexity (BRASIL, 2010). In this context, a psychosocial care network (Rede de Atenção Psicossocial - RAPS) was established to improve and coordinate between various healthcare entities, including primary healthcare (Atenção Primária à Saúde – APS), specialized psychosocial care, urgent and emergency care, transient residential care, and hospital care, with an emphasis on deinstitutionalization strategies and psychosocial rehabilitation (BRAZIL. MINISTÉRIO DA SAÚDE, 2011; MOREIRA; ONOCKO-CAMPOS, 2017). There was also a focus on expanding existing services, providing benefits for vulnerable populations such as users of alcohol or other drugs, children and adolescents, homeless individuals, and indigenous populations, as well as increasing access and linking users to healthcare providers (BRAZIL. MINISTÉRIO DA SAÚDE, 2011).

However, it has proven to be a complex task to provide integrated mental healthcare that is universally accessible to the Brazilian population, as well as being anchored in multidisciplinary work and expanded clinical practice. The existence of gaps between the political-assistance guidelines and daily practice has been noted in previous studies (FIOCRUZ. FUNDAÇÃO CALOUSTE GULBENKIAN, 2015), raising questions about the scope of the changes made and highlighting the need for an evaluation that can find the answers (MOREIRA; BOSI, 2019).

Evaluations of the Brazilian PCN fall short of the mark in spite of an expansion of studies, as demonstrated in a recent systematic review by Costa et al. (2015). Most research still focuses on isolated services or specific populations. Likewise, few studies use standardized and validated instruments to evaluate the PCN (COSTA; COLUGNATI; RONZANI, 2015). Moreover, most of these studies deal with specific themes and services, such as the assessment scale of patient satisfaction with mental health services (SATIS-BR), the assessment scale of the workplace impact on mental health services (IMPACTO-BR) (BANDEIRA; PITTA; MERCIER, 2000; BANDEIRA; SILVA, 2012), and an instrument to evaluate the structure and process of mental healthcare in psychosocial care centers (Centros de Atenção Psicossocial – CAPS) (AZEVEDO; SALVETTI; TORRES, 2017). Thus, instruments that can holistically evaluate the Brazilian PCN continue to be a significant gap in the literature (AMARAL; BOSI, 2017; ONOCKO-CAMPOS et al., 2018).

Therefore, this study aimed to develop and validate a structured and reproducible multidimensional instrument for evaluating the structure and process (DONABEDIAN, 1978) of the PCN and its implementation in the Brazilian state of Minas Gerais (MG). This instrument was designed to identify the weaknesses and strengths of the PCN implementation in order to support management and decision-making, as well as fostering a dialogue on the findings from different regions and enabling comparisons over time.

Methodology

This methodological study was conducted using a quantitative approach and a cross-sectional design to validate a multidimensional questionnaire with respect to its content and layout. The questionnaire is an instrument designed to analyze the implementation of the PCN according to the dimensions of structure and process (CHAMPAGNE et al., 2011). Methodological studies use a variety of methods to obtain, organize, and analyze data on the creation and validation of research instruments (POLIT; YANG, 2016). Thus, this study was carried out in three stages: (1) evaluability assessment, (2) development of the measurement instrument, and (3) application of the Delphi technique to validate the content and appearance of the questions.

For the evaluability assessment, a stage consisting of the evaluation plan, a methodology based on the seven-element system of Thurston and Ramaliu (2005), was used as follows: (1) identification and review of available documents; (2) description of the program identifying its goals, objectives, and activities; (3) construction of the theoretical-logical model with a graphical representation of available resources, intended activities, expected impacts, and assumed causal connections; (4) preliminary understanding of how the program operates; (5) development of a theoretical evaluation model; (6) identification of evaluation users and other key stakeholders; and (7) obtaining an agreement on the evaluation procedure.

The first step consisted of identifying people who were involved or affected by the program and who would be able to support and contribute to the evaluation. These stakeholders were chosen to participate in the evaluation plan using the criteria proposed for evaluating public health programs (CENTERS FOR DISEASE CONTROL AND PREVENTION, 1999). The stakeholders selected included agents involved in the planning and coordination of the assessed program, active health professionals, and members of the academic community. The convenience sample consisted of members of the federal, state (Minas Gerais), and local (Belo Horizonte, the state capital) mental health agencies, as well as regional technical offices of the MG Health Department. The bulk of participants was involved in the Minas Gerais PCN since the aim was to advance research into the implementation of the network in the state (CHAMPAGNE et al., 2011).

From January to July 2019, a review was carried out of the scientific literature as well as documents available on government websites relating to the technical and legal frameworks, covering the period from 1991 to 2019. Information on how the PCN operates and the stakeholders involved was of vital importance for describing the program, preparing the theoretical-logical model, and coming to an agreement on the evaluation method.

The PCN theoretical-logical model was based on Amaral and Bosi's (2017) conceptual-analytical model of health networks, which contains five dimensions: minimum units, connectivity, integration, normativity, and subjectivity. This study considered the five dimensions as components, which were further divided into subcomponents. For each subcomponent, the necessary activities (process) to achieve the expected short and medium-term results by applying resources (structure) were listed (CHAMPAGNE et al., 2016). As described later, researchers and stakeholders participated in the model review, re-elaboration, and consensus.

The multidimensional questionnaire was thus formed using 82 evaluative questions on the basis of the theoretical-logical model, 11 of which related to structural dimensions and 71 to process dimensions. The 71 questions on process dimensions were subdivided into five components: 34 on minimum units, 14 on connectivity, six on integration, nine on normativity, and eight on subjectivity. The instrument followed the plan proposed in the design of the PCN and was structured to be answered by individuals involved in mental health programs in the municipalities of MG.

The content and appearance of the questions were validated using the Delphi technique, a method of decision-making used by groups of judges aiming to reach a consensus (JÜNGER et al., 2017). The criteria used to select judges in this study followed the recommendations of Scarparo et al. (2012), prioritizing working experience, master’s or doctoral studies, or academic positions involving scientific activities in the field over the last five years. The judges chosen included members of the national mental health agency, the MG state mental health agency, and regional technical mental health offices, who also indicated other professionals with suitable experience and knowledge of the Brazilian PCN. In addition, the database of the National Council for Scientific Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico – CNPq) was accessed through the Lattes Platform to search for the most active and influential Brazilian researchers in the field of the study. This search aimed to compose a panel of judges with a variety of experience and professional activities.

The selected judges were all sent the study’s objectives and methodology, the justification of the validation process, and a request for their participation in the study. Subsequently, the form containing the questions of the multidimensional questionnaire was sent using Google Forms®, with each judge asked to consider every question in light of the criteria proposed by PASQUALI (2010) relating to relevance, objectivity, and clarity. The responses were organized on a five-level Likert scale: 1, strongly agree (SA); 2, agree (A); 3, neither agree nor disagree (NN); 4, disagree (D); and 5, strongly disagree (SD). Each question also had a space for judges to suggest changes or comments.

Four attempts were made to send the form between 11/19/19 and 12/20/19. Judges who did not submit their responses to the form during this period were excluded from the sample. Before participating in this study, the experts signed an informed consent form, and their anonymity was guaranteed throughout the entire process.

The content validity index (CVI) was used to identify the degree of agreement among the experts. Although the cutoff points used in the literature to denote consensus vary between 50 and 80% (JÜNGER et al., 2017), this study uses a cutoff point of 75% agreement. The methodology specifies that up to three questionnaire rounds must be conducted among the participants if the 75% consensus agreement is not reached in the first round. The CVI was calculated using the sum of responses classified as SA and A divided by the total number of responses. The content validity ratio (CVR) was also calculated to compare the sum of responses classified as SA and A with the value expected due to random chance. If there is a likelihood of over 5% that the amount of agreement observed is due to random chance, the item must be eliminated or revised. Finally, a Kappa coefficient of agreement was calculated as the ratio between the proportion of times that the judges did agree and the maximum proportion of times that the judges could agree. Kappa values range from -1 (total absence of an agreement) to 1 (total agreement) (ALEXANDRE; COLUCI, 2011).

This study was approved by the human research ethics committee under registry number CAAE: 77798217.1.3001.5091 and is part of the first author’s doctoral research. The study received funding from the Research Support Foundation of MG (Fundação de Amparo à Pesquisa de MG – FAPEMIG).

Results

Program description

The provision of psychosocial care by the territorial services of the Brazilian SUS began in the 1970s, when institutionalization was the primary way of approaching people in mental distress (FIOCRUZ. Fundação Calouste Gulbenkian, 2015). After stalling in the country’s legislature for 12 years, Law No. 10,216 was passed in 2001 and has provided a legislative framework and new direction for the psychosocial care model.

The new care policy was initially based on creating territorial psychosocial care centers (Centro de Atenção Psicossocial – CAPS), as illustrated in Figure 1. These services were established as an intermediate step between outpatient clinics and hospitalization and have progressively increased their level of complexity, adding crisis management, 24-hour operation with overnight admission, and monitoring of critically ill patients after stabilization and their reinsertion into society. Thus, the CAPS subverted the logic of the existing hierarchy, bringing together several levels of healthcare in a single unit (QUINDERÉ; JORGE; FRANCO, 2014; ONOCKO-CAMPOS et al., 2018). CAPS were further diversified through the creation of specific services for people with mental distress from alcohol or other drug use (CAPS AD) and children and adolescents (CAPSi). Meanwhile, several strategies were instituted to expand the new care model, such as the restructuring of hospital psychiatric care to favor small hospitals and short hospitalizations; the development of criteria to protect users’ rights; the creation of therapeutic residential services (TRS) and rehabilitation assistance for long-term hospitalizations (Return Home Program); incentives for the creation of social cooperatives; the integration of mental health care into the primary healthcare system (APS); plans to care for children, adolescents, and homeless individuals in mental distress; integration with emergency services; and the inclusion of backup mental health beds in general hospitals (BRAZIL, 2015).

Figure 1
Timeline of the Brazilian psychosocial care network (PCN)

According to the report Mental Health in Data by the Brazilian Health Department (HD), periodically released until 2015, the implementation of the PCN accelerated considerably since 2001, with 86% of the population covered by 2014. also compares the number of services implemented in Brazil between 2002 and 2014, reflecting the implementation of a vast network of psychosocial care services with the redistribution of financial resources from the hospital network to community services. In this period, the budget allocated to hospitals decreased from 95% of total spending on mental health to less than 30%, while overall spending on public mental health care increased (BRAZIL, 2015).

The so-called “new” national mental health policy (Política Nacional de Saúde Mental – PNSM) was launched in 2017 after changes in the political direction of the federal government. Professionals and researchers involved in the public provision of mental health services in Brazil saw this policy as a disruption of a policy making tradition that sought consensus with the social organizations historically involved in the process. While the established basis of care provision was not changed, the policy was seen as strengthening a logic of specialization and hospital-centricity through the inclusion of psychiatric hospitals and mental health outpatient clinics (multi-professional specialized mental healthcare team) in the list of components of the PCN, as well as the replacement of beds in general hospitals with specialized wards with up to 30 beds, public funding for the acquisition of electroconvulsive therapy equipment, and the maintenance and financing of therapeutic communities (ALMEIDA, 2019; BASTOS, 2019; ONOCKO-CAMPOS, 2019).

Since then, the data on the PCN provided by the Brazilian Health Department has been limited. Although the regulations surrounding the new PNSM have not been revoked, the technical note that was published describing the intended changes was removed from its official website after protests, leaving doubts regarding the magnitude of the changes that will be implemented in the future (ONOCKO-CAMPOS, 2019).

Understanding the operationalization of the program, identifying those interested in the evaluation, and obtaining an agreement on the evaluation

Throughout the study, several meetings were held between the research group (teachers, technical support scholarships, master’s members, and senior students under scientific initiation) and the stakeholders. Two meetings were conducted with members of the national mental health agency, eight with members of the MG state mental health agency, three with the 28 regional technical mental health offices in MG, and one meeting with the mental health agency of Belo Horizonte. The discussions considered the opinions of all participants involved in the operationalization of the PCN and the main points to be addressed in the research. Participants attended the working meetings and provided feedback as well as information and administrative documents. This involvement made it possible for researchers to approach practical aspects of the operationalization of PCN, helping to address the usefulness, feasibility, property, and accuracy quality standards of evaluative research (CENTERS FOR DISEASE CONTROL AND PREVENTION, 1999; YARBROUGH et al., 2010; CRAIG; CAMPBELL, 2015). In addition, stakeholders who showed an interest in the PCN evaluation – even if they did not directly participate in the study – were identified. These interested individuals included 1) national and state technical offices interested in the evaluation as a management tool to monitor the progress of the PCN implementation, diagnose gaps in care, foster regional cooperation in the use of mental health services, and allocate financial investments; 2) regional technical offices seeking to use the results to support municipalities, diagnose gaps in care, and foster regional cooperation in the use of services; 3) municipal technical offices seeking better information on the region’s PCN, as this would help improve and develop new strategies for inter-municipal cooperation in the sharing of services; 4) professionals within the PCN who are interested in better understanding the regional PCN, since this would help improve their local activities; 5) researchers aiming to use the information generated in the evaluation of the PCN to advance research into the public mental healthcare provided by the SUS and expand efforts to evaluate the Brazilian PCN; 6) users and their relatives advocating for improvement in mental healthcare.

Elaboration of the theoretical-logical model for the development of the multidimensional questionnaire

The theoretical-logical model is a visual representation of the functioning of the program evaluated and an objective analysis of the presumed causal relationships between its elements, demonstrating how they work together to solve the problem that gave rise to the intervention (CHAMPAGNE et al., 2016). For the modeling of the PCN, the “Minimum Units” component, including the subcomponents of mental healthcare and psychosocial rehabilitation, was structured as the interventions that were essential for implementing the PCN as originally proposed (). Although Amaral and Bosi (2017) privileged the network’s services in this component, we chose to follow the Donabedian triad of structure/process/result to allocate the services, human resources, logistics system, and health education of professionals in the “Structure” component (DONABEDIAN, 1978). The “Connectivity” component (subcomponent: network articulation) detailed the intra-sectorial and inter-sectorial actions, including the collaborative care actions. The “Integration” component included the subcomponents: governance and care management. This arrangement encompasses the relationships between the services and the interventions carried out over time for the continuous monitoring of each user. For the “Normativity” component (sub-components: mental health policy and social participation and control), the PNSM was considered the guideline for PCN orientation. It should be noted that the PCN, like other healthcare networks, presupposes an integrated operation with its components intertwining and generating indirect causal relationships across various components. However, for better visualization, only the presumed direct relationships were displayed, as shown in Figure 2.

Figure 2
Theoretical and logical model of the Brazilian psychosocial care network (PCN)

Using the Delphi technique to validate the content and appearance of the multidimensional instrument for evaluating the implementation of the PCN (IMAI-RAPS)

Of the 90 invited judges, 44 (48.9%) evaluated the questions extracted from the theoretical-logical model. The age of the judges ranged from 30 to 67 years, with an average of 44.5 years, and 59.1% were female. Postdoctoral researchers made up 9.1% of the judges, with 22.7% reporting themselves as having a doctoral degree, 25% having a master’s degree, 34.1% a postgraduate certification, and 9.1% as having completed the undergraduate level. A majority (63.6%) of judges were psychologists, with nurses making up 13.6%, medical doctors 11.4%, and the remaining 11.4% split between social workers, physiotherapists, occupational therapists, and pharmacists. Concerning the relationship between the judges and the PCN, 47.7% of the specialists are professionals within the network, 34.1% are researchers, and the remaining 18.2% are professionals who work or have experience in management, consulting or coordination related to the PCN. It should be noted that prominent researchers from several regions of the country were among the participants in the group.

All of the questions presented to the judges were validated because all of them received an agreement degree (CVI) above the establiShed threshold of 0.75, with satisfactory values in the other tests. The lowest values for objectivity were 0.77 CVI, 0.55 CVR, and Kappa 0.78. The analysis of the clarity criterion, in turn, revealed that one item was not sufficiently straightforward (CVI: 0.75, CVR: 0.50, and Kappa: 0.76), leading to the exclusion of the question. Comments from several evaluators that certain questions were unnecessary or redundant led to the exclusion of 26 more questions. In addition, most judges suggested excluding the questions related to the subjectivity component, despite being validated, noting that qualitative techniques are needed due to the nature of the subject. Only one question from the subjectivity component was maintained and relocated in the normativity component, as it relates to the sub-component “participation and social control.”

The instrument was submitted to other changes suggested by the judges, such as minor alterations in the text and splitting two questions into four to avoid misinterpretations. The alterations were possible without applying another round of the Delphi technique since there was a consensus above 0.75, and different judges provided similar suggestions on simple changes relating to certain of the terms used. Meanwhile, the instrument was considered intelligible in terms of its appearance. After considering all these changes, the questionnaire was reduced from 82 to 55 questions, of which 21 were on minimum units, 12 on connectivity, eight on integration, eight on normativity, and six on structure (ANNEX 1).

It should be highlighted that the instrument does not include questions related to the quantity of PCN services implemented. Although this information provides important context for the “Structure” component, it represents information that can be accessed through governmental systems such as DATASUS. In addition, including questions on the subject would contradict stakeholders’ emphasis on the need to reduce the questionnaire in order to ensure greater uptake by respondents. Finally, it is noteworthy that the judges considered the instrument to be comprehensive and well-prepared overall. It should also be noted that the comments and suggestions of the judges were detailed instead of general, showing extensive knowledge of the PCN and deep involvement in the validation phase of the study.

Discussion

Although implementing national policies for psychosocial care is an essential tool for improving the population’s mental health (PATEL et al., 2018), and despite the active promotion of mental healthcare by international organizations, only 60% of the member countries of the World Health Organization (WHO) had mental health policies in 2011, while 71% had mental health plans and 59% had legislation in the area. Hence, the evaluability assessment of this study could help to document the progress in implementing the public policy for psychosocial care in Brazil.

However, as demonstrated, these advances were most evident prior to 2016, when a shift in government policies resulted in considerable alterations to the national mental health policy in 2017, jeopardizing the achievements of earlier decades (ALMEIDA, 2019). In addition, the chronic underfunding of this and other public health policies and the scarcity of large-scale evaluation studies have also been associated with the growing lack of official data since 2015. Thus, Onocko-Campos (2019) warns of the need to evaluate the implementation of services to avoid wasting efforts and financial resources.

Given the subjective aspects that pervade psychosocial care and the recent reformulation of the Brazilian care model, systematizing strategies to evaluate current practices is a challenging but necessary activity to monitor their implementation and functionality (OLIVEIRA et al., 2014; COSTA; COLUGNATI; RONZANI, 2015; AMARAL; BOSI, 2017; CORREIA; GOULART; FURTADO, 2017). Moreover, evaluation studies on mental healthcare networks are also scarce in the international literature (ASHWOOD et al., 2018; MONTGOMERY et al., 2019). Regarding the implementation analysis, a 2015 study highlights the lack of tools for mapping adult mental healthcare services and proposes a protocol to evaluate and monitor the development of integrated systems in eight European countries (SALVADOR-CARULLA et al., 2015). Other experiences that use instruments similar to those studied in this work are the European service mapping schedule (ESMS), developed to standardize the classification of health services and the care provided to adults in situations of mental distress; the international classification of mental healthcare (ICMHC), which aims to evaluate the different care specialties offered by the health system; and the European sociodemographic schedule (ESDS), aiming at a standardized description of the sociodemographic characteristics of the local of implementation. The ESMS also underwent adaptations for application in other countries and populations, such as children and adolescents (ROMERO-LÓPEZ-ALBERCA et al., 2019). Another instrument designed to provide data on mental health systems is the World Health Organization's assessment instrument for mental health systems (WHO-AIMS), which aims to collect essential information for the global mental health system (NEISI et al., 2005). However, the literature reports that the descriptions generated by the application of that tool still provide very general findings (ROMERO-LÓPEZ-ALBERCA et al., 2019). In addition, due to the distinctive characteristics of each national health system, most of these instruments are difficult to replicate and cross-culturally adapt, thus calling for the development of tools designed for the Brazilian model.

Before developing a program evaluation study, it is recommended to conduct a detailed analysis of its policy or concrete situation to determine the feasibility and need for the evaluation (FIGUEIREDO; ANGULO-TUESTA; HARTZ, 2019). Thus, evaluability assessments have gained prominence as they provide a detailed analysis of the intervention, contributing to future evaluative studies by exploring the expectations and needs of the interest groups involved and the degree of organization and implementation of the evaluated initiative (CRAIG; CAMPBELL, 2015; FIGUEIREDO; ANGULO-TUESTA; HARTZ, 2019). Accordingly, the theoretical-logical model developed in this study was essential to understanding the basic theoretical premises of the PCN, condensing the peculiarities of this complex program in a single image, representing a product that can be used in subsequent evaluative research. Significantly, the disagreements brought by the “new” policy were not neglected by these efforts to track the national mental health policy. The potential induction of changes by the federal government cannot be ignored due to its ability to modify the funding of services and/or procedures (MOREIRA; FERRÉ; ANDRADE, 2017; ALMEIDA, 2019; BASTOS, 2019; ONOCKO-CAMPOS, 2019).

Identifying the potential use of the results and assessing the sustainability of the proposed intervention are two additional aspects of this assessment. Therefore, this activity cannot be carried out solely by the management nor applied only when urgent or convenient (TANAKA; TAMAKI, 2012). Hence, this evaluability assessment engaged members of the national and state mental health agencies, regional technical offices, and workers within the PCN of MG to develop the theoretical-logical model that identifies the central aspects to be evaluated.

Amaral and Bosi (2017) pointed out that the evaluation of the organization of health policies in care networks is not sufficiently studied in the field of public health research. However, understanding the effects of the articulation between services is a fundamental issue since the lack of healthcare coordination is one of the most frequent causes of low problem-solving capacity in healthcare networks (VÁZQUEZ et al., 2015; DE ALMEIDA; DE OLIVEIRA; GIOVANELLA, 2018). With respect to mental health, these factors are even more essential since the possibility of treatment goes beyond a single type of service and a single therapeutic modality (PATEL et al., 2018). Governance, addressed in the theoretical-logical model of this study in the "connectivity" and "integration" components, therefore requires political and technical action to meet the demand for already limited resources in constantly changing circumstances; as a result, it is frequently identified as one of the most critical and complex aspects of any mental health system (NICKELS et al., 2018). In addition, to achieve the integration of care services in the PCN, the adoption of the concept of health regions as administrative units, instead of the municipal model initially adopted by the SUS, is essential for ensuring access to shared facilities that are otherwise financially unfeasible for small municipalities (MACEDO et al., 2017; MOREIRA; FERRÉ; ANDRADE, 2017; TRAPÉ; CAMPOS, 2017).

Interestingly, although this study was designed to evaluate PCN implementation at the state level, it could be reproducible, with minor adaptations, in other regions of the country since the national regulations were the starting point for the formulation of the study model. Moreover, despite having been designed for response at the municipal level, the instrument was generated based on the structure of the PCN as a hierarchical network of services in each health region. A good indication of this reproducibility was the validation of the evaluative questions in the theoretical-logical model by experts who study mental health in various parts of the country. Content validation was used because it is essential for developing new measurement instruments since it represents the initial stage for associating abstract concepts with observable and measurable indicators (COLUCI; ALEXANDRE; MILANI, 2015). However, the application of the Delphi technique by electronic means is a recent procedure with the advantages of reaching diverse and geographically distant actors and the rapid distribution, collection, and data processing; though it also has limitations, such as the increased rates of non-response that leads to considerable sample losses (JÜNGER et al., 2017).

The limitations of this study are related to the complexity of the PCN, encompassing aspects beyond the analytical capacity of a structured questionnaire. Thus, although identifying evaluative questions is fundamental for further evaluation (ARAUJO et al., 2018), more in-depth questions pointed out in the literature and by some judges should be qualitatively analyzed in subsequent studies.

Conclusion

This study included an investigation of the historical processes that culminated in the current Brazilian PCN system, as well as modeling, planning, and verifying the possibility of evaluating its implementation in MG. It therefore aims to enrich the debate on evaluative methodologies that can analyze the relationship between the components of the PCN as well as the relationships between these components and their context. It also served to validate the theoretical-logical model and the multidimensional instrument for evaluating the implementation of a PCN (IMAI-RAPS), which can be disseminated for use in future research.

Considering the current national situation, with marked crises in public health and the recent COVID-19 pandemic that has increased the personal and social costs of mental illness to an unprecedented magnitude (LAKE, 2020), sharing the information produced in this study is expected to advance studies documenting the progress already achieved by the PNSM in the country and identifying the remaining obstacles, enhancing the strategic management of the SUS for improving psychosocial care services.

Referências

• ALEXANDRE, N. M. C.; COLUCI, M. Z. O. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciência & Saúde Coletiva, v. 16, n. 7, p. 3061-3068, jul. 2011.
• ALMEIDA, J. M. C. de. Política de saúde mental no Brasil: o que está em jogo nas mudanças em curso. Cadernos de Saúde Pública, v. 35, n. 11, p. e00129519, abr. 2019.
• ALMEIDA, J. M. Mental health services development in Latin America and the Caribbean: achievements, barriers and facilitating factors. International Health, v. 5, n. 1, p. 15-18, mar. 2013.
• ALMEIDA, P. F.; OLIVEIRA, S. C.; GIOVANELLA, L. Integração de rede e coordenação do cuidado: o caso do sistema de saúde do Chile. Ciência & Saúde Coletiva, v. 23, n. 7, p. 2213-2228, jul. 2018.
• AMARAL, C. E. M.; BOSI, M. L. M. O desafio da análise de redes de saúde no campo da saúde coletiva. Saúde e Sociedade, v. 26, n. 2, p. 424-434, jun. 2017.
• ARAUJO, R. F. de; et al. Avaliabilidade da compatibilização de indicadores e metas dos instrumentos de planejamento do Ministério da Saúde. Saúde em Debate, v. 42, n. 118, p. 566-578, set. 2018.
• ASHWOOD, J. S. et al. Evaluation of the Mental Health Services Act in Los Angeles County. Rand health quarterly, v. 8, n. 1, p. 4-7, ago. 2018.
• AZEVEDO, D. M. de; SALVETTI, M. D. G.; TORRES, G. V. de. Qualidade da assistência em saúde mental: desenvolvimento de um instrumento de avaliação. Revista de Enfermagem do Centro-Oeste Mineiro, v. 7, p. e1685, dez. 2017.
• BANDEIRA, M.; PITTA, A. M. F.; MERCIER, C. Escalas brasileiras de avaliação da satisfação (SATIS-BR) e da sobrecarga (IMPACTO-BR) da equipe técnica em serviços de saúde mental. Jornal Brasileiro de Psiquiatria, v. 49, n. 4, p. 105-115, abr. 2000.
• BANDEIRA, M.; SILVA, M. A. da. Escala de Satisfação dos Pacientes com os Serviços de Saúde Mental (SATIS-BR): estudo de validação. Jornal Brasileiro de Psiquiatria, v. 61, n. 3, p. 124-132, jul. 2012.
• BASTOS, F. I. Políticas de drogas no Brasil contemporâneo: aportes da ciência, da clínica e do liberalismo moderno. Cadernos de Saúde Pública, v. 35, n. 11, p. e00125519, set. 2019.
• BRASIL. Portaria no 4.279, de 30 de dezembro de 2010. Estabelece diretrizes para a organização da Rede de Atenção à Saúde no âmbito do Sistema Único de Saúde (SUS). Diário Oficial da União, Brasília, DF, 31 dez. 2010.
• BRASIL. Portaria nº 3.088, de 23 de dezembro de 2011. Institui a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União, Brasília, DF, 26 dez. 2011.
• BRASIL. Portaria nº 3.588, de 21 de dezembro de 2017. Altera as Portarias de Consolidação nº 3 e nº 6, de 28 de setembro de 2017, para dispor sobre a Rede de Atenção Psicossocial, e dá outras providências. Diário Oficial da União: seção 1, Brasília, DF, p. 236, 22 dez. 2017.
• BRASIL. MINISTÉRIO DA SAÚDE. Saúde Mental em Dados - 12, ano 10, no 12. Informativo Eletrônico. Brasília, 2015 (acesso em: 21/12/2020)
• CENTERS FOR DISEASE CONTROL AND PREVENTION. Framework for Program Evaluation in Public Health. Morbidity and mortality weekly report, v. 48, n. 11, p. 1-58, set. 1999.
• CHAMPAGNE, F. et al. Análise da implantação. In: BROUSSELLE, A. et al. (Org). Avaliação: Conceitos e Métodos. 1. ed. Rio de Janeiro: Ed. Fiocruz, 2011. p. 292.
• CHAMPAGNE, F. et al. Modelizar as intervenções. In: BROUSSELLE, A. et al. (Org). Avaliação: conceitos e métodos. 2. ed. Rio de Janeiro: Editora Fiocruz, 2016. p. 61-74b.
• COLUCI, M. Z. O.; ALEXANDRE, N. M. C.; MILANI, D. Construção de instrumentos de medida na área da saúde. Ciência & Saúde Coletiva, v. 20, n. 3, p. 925-936, abr. 2015.
• CORREIA, P. C. I.; GOULART, P. M.; FURTADO, J. P. A avaliabilidade dos Núcleos de Apoio à Saúde da Família (Nasf). Saúde em Debate, v. 41, n. spe, p. 345-359, mar. 2017.
• COSTA, P. H. A. da; COLUGNATI, F. A. B.; RONZANI, T. M. Avaliação de serviços em saúde mental no Brasil: revisão sistemática da literatura. Ciência & Saúde Coletiva, v. 20, n. 10, p. 3243-3253, out. 2015.
• CRAIG, P.; CAMPBELL, M. Evaluability Assessment: a systematic approach to deciding whether and how to evaluate programmes and policies: a What Works Scotland Working Paper. Glasgow, 2015.
• DONABEDIAN, A. The quality of medical care. Science, v. 200, n. 4344, p. 856-864, maio. 1978.
• FIGUEIREDO, T. A.; ANGULO-TUESTA, A.; HARTZ, Z. Avaliabilidade da Política Nacional de Regulação no SUS: uma proposta preliminar. Physis: Revista de Saúde Coletiva, v. 29, n. 2, set. 2019.
• FIOCRUZ; FUNDAÇÃO CALOUSTE GULBENKIAN. Inovações e Desafios em Desinstitucionalização e Atenção Comunitária no Brasil. Seminário Internacional de Saúde Mental: Documento Técnico Final. Rio de Janeiro: Fiocruz. Fundação Calouste Gulbenkian. Organização Mundial da Saúde. Ministério da Saúde, 2015.
• JACOB, K. S. et al. Mental health systems in countries: where are we now? The Lancet, v. 370, n. 9592, p. 1061-1077, set. 2007.
• JÜNGER, S. et al. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine, v. 31, n. 8, p. 684-706, set. 2017.
• KOHN, R. et al. Mental health in the Americas: an overview of the treatment gap. Revista Panamericana de Salud Pública, v. 42, p. e165, out. 2018.
• LAKE, J. A Mental Health Pandemic: The Second Wave of COVID-19. Psychiatric Times, 15 jun 2020. Disponível em: https://www.psychiatrictimes.com/view/a-mental-health-pandemic-the-second-wave-of-covid-19. Acesso em: 21 dez. 2020.
• MACEDO, J. P. et al. A regionalização da saúde mental e os novos desafios da Reforma Psiquiátrica Brasileira. Saúde e Sociedade, v. 26, n. 1, p. 155-170, mar. 2017.
• MONTGOMERY, L. et al. An evaluation of mental health service provision in Northern Ireland. Health and Social Care in the Community, v. 27, n. 1, p. 105-114, jun. 2019.
• MOREIRA, D. D. J.; BOSI, M. L. M. Qualidade do cuidado na Rede de Atenção Psicossocial: experiências de usuários no Nordeste do Brasil. Physis: Revista de Saúde Coletiva, v. 29, n. 2, p. 1-22, fev. 2019.
• MOREIRA, L. M. de C.; FERRÉ, F.; ANDRADE, E. I. G. Financiamento, descentralização e regionalização: transferências federais e as redes de atenção em Minas Gerais, Brasil. Ciência & Saúde Coletiva, v. 22, n. 4, p. 1245-1256, abr. 2017.
• MOREIRA, M. I. B.; ONOCKO-CAMPOS, R. T. Ações de saúde mental na rede de atenção psicossocial pela perspectiva dos usuários. Saúde e Sociedade, v. 26, n. 2, p. 462-474, maio. 2017.
• NEISI, L. et al. A review of the world health organization assessment instrument for mental health system: WHO-AIMS. Quarterly Journal of Nursing Management, v. 6, n. 3, p. 66-77, 2005.
• NICKELS, S. V. et al. Access to mental health care in El Salvador: a case study of progress toward decentralization. Revista Panamericana de Salud Pública, v. 42, p. e172, out. 2018.
• OLIVEIRA, M. A. F. de et al. Processos de avaliação de serviços de saúde mental: uma revisão integrativa da literatura. Saúde em Debate, v. 38, n. 101, p. 368-378, jun. 2014.
• ONOCKO-CAMPOS, R. T. et al. Atuação dos Centros de Atenção Psicossocial em quatro centros urbanos no Brasil. Revista Panamericana de Salud Pública, v. 42, p. e113, out. 2018.
• ONOCKO-CAMPOS, R. T. Saúde mental no Brasil: avanços, retrocessos e desafios. Cadernos de Saúde Pública, v. 35, n. 11, p. e00156119, jan. 2019.
• PASQUALI L. Instrumentação psicológica: Fundamentos e práticas. Porto Alegre: Artmed, 2010.
• PATEL, V. et al. The Lancet Commission on global mental health and sustainable development. The Lancet, v. 392, n. 10157, p. 1553-1598, out. 2018.
• POLIT, D.; YANG, F. Measurement and the measurement of change. China: Wolters Kluwer, 2016.
• QUINDERÉ, P. H. D.; JORGE, M. S. B.; FRANCO, T. B. Rede de Atenção Psicossocial: qual o lugar da saúde mental? Physis: Revista de Saúde Coletiva, v. 24, n. 1, p. 253-271, mar. 2014.
• ROMERO-LÓPEZ-ALBERCA, C. et al. Standardised description of health and social care: A systematic review of use of the ESMS/DESDE (European Service Mapping Schedule/Description and Evaluation of Services and DirectoriEs). European Psychiatry, v. 61, p. 97-110, ago. 2019.
• SALVADOR-CARULLA, L. et al. Developing a tool for mapping adult mental health care provision in Europe: The REMAST research protocol and its contribution to better integrated care. International Journal of Integrated Care, v. 15, out./dez. 2015.
• SCARPARO, A. F. et al. Reflexões sobre o uso da técnica Delphi em pesquisas na enfermagem. Rev Rene, v. 13, n. 1, p. 242-251, jan. 2012.
• TANAKA, O. Y.; TAMAKI, E. M. O papel da avaliação para a tomada de decisão na gestão de serviços de saúde. Ciência & Saúde Coletiva, v. 17, n. 4, p. 821-828, abr. 2012.
• THURSTON, W. E.; RAMALIU, A. Evaluability assessment of a survivors of torture program: Lessons learned. Canadian Journal of Program Evaluation, v. 20, n. 2, p. 1-25, set. 2005.
• TRAPÉ, T. L.; CAMPOS, R. O. The mental health care model in Brazil: analyses of the funding, governance processes, and mechanisms of assessment. Revista de Saúde Pública, v. 51, p. 1-8, mar. 2017.
• VÁZQUEZ, M.-L. et al. Evaluating the effectiveness of care integration strategies in different healthcare systems in Latin America: the EQUITY-LA II quasi-experimental study protocol. BMJ Open, v. 5, n. 7, p. e007037, jul. 2015.
• VIEIRA, S. Como elaborar questionários. 1. ed. São Paulo: Atlas, 2009.
• VIGO, D.; THORNICROFT, G.; ATUN, R. Estimating the true global burden of mental illness. Lancet Psychiatry, v. 3, n. 2, p. 171–178, fev. 2016.
• YARBROUGH, D. B.; SHULA, L. M.; HOPSON, R. K.; CARUTHERS, F. The Program Evaluation Standards: A guide for evaluators and evaluation users. 3a ed. Thousand Oaks, CA: Corwin Press, 2010.
• ZHOU, W. et al. Policy development and challenges of global mental health: a systematic review of published studies of national-level mental health policies. BMC Psychiatry, v. 18, n. 1, p. 138, dez. 2018.

ANNEX 1

Multidimensional instrument for evaluating the implementation of the PCN (IMAI-RAPS)

Do you have a graduate degree? ( fill in all that apply)	⚪ No	⚪ Yes, Graduate certification in other areas rather than the health field
	⚪ Yes, Graduate certification in Mental Health	⚪ Yes, Specialization by Distance Education
	⚪ Yes, Graduate certification in Health Management	⚪ Yes, a Master's degree
	⚪ Yes, Specialization in Public Health	⚪ Yes, a Doctorate
	⚪ Yes, Graduate certification in another health area	⚪ Other:
Length of service in the SUS (in years worked):
Length of service in Mental Health (in years worked):
Municipality:
I. Minimum units component
Assistance
How often are the following mental health actions performed in your municipality?
1. Multidisciplinary assistance
Care in which professionals with different backgrounds assist the user and share decisions about the follow-up of the case
⚪ Always   ⚪ Most of the time   ⚪ Sometimes   ⚪ Rarely   ⚪ Never   ⚪ I don’t know   ⚪ N/A
2. Therapeutic workshops and/or body motion or physical activities
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
3. Use of a coordinated therapeutic approach
Set of proposals for joint therapeutic activities directed to an individual, family, or community. These proposals aim to outline an intervention strategy for the user, relying on the resources of the healthcare team, community, family, and the subject
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
4. (CONNECTIVITY) Case discussions among the team members serving a user
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
5. (CONNECTIVITY) Case discussions between teams of different PCN services
through the nuclei for the family healthcare (NASF) or directly between the teams involved
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
6. Does the psychiatrist (local or from another agreed municipality) discuss the case with other non-medical professionals, and are they open to joint approaches to manage the case?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
7. (CARE MANAGEMENT) Is there a queue of outpatient care waiting for a psychiatrist, psychologist, or another mental health professional?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
8. Are conversation circles, groups, workshops, or other health promotion activities offered in your municipality?
(e.g., self-care, proper use of medications, lifestyle habits, physical activity, diet, consequences of harmful use of alcohol or other drugs)
⚪ Yes     ⚪ No   ⚪ I don’t know   ⚪ N/A
In primary healthcare of your municipality:
9. The professionals of the basic health units monitor users with common mental disorders:
e.g., depression, anxiety, somatoform complaints
⚪ YES
⚪ Most of the time — fully follow-up on stabilized users
⚪ Most of the time — performs clinical care when requested by the mental health team
⚪ Sometimes   ⚪ Rarely   ⚪ Never   ⚪ I don’t know   ⚪ N/A
10. Do the basic health unit professionals monitor users with severe mental health disorders?
⚪ YES
⚪ Most of the time — fully follow-up on stabilized users
⚪ Most of the time — performs clinical care when requested by the mental health team
⚪ Sometimes   ⚪ Rarely   ⚪ Never   ⚪ I don’t know   ⚪ N/A
11. Do the professionals of the basic health units monitor users in psychological distress due to alcohol or other drug abuse?
⚪ YES
⚪ Most of the time — fully follow-up on stabilized users
⚪ Most of the time — performs clinical care when requested by the mental health team
⚪ Sometimes   ⚪ Rarely   ⚪ Never   ⚪ I don’t know   ⚪ N/A
Mental health crisis care:
Situations involving moments of destabilization such as severe acute psychiatric symptoms; severe rupture of family and/or social relationships; refusal of treatment; alarming and risky situations in the family and/or social context; risk of suicide
12. Is it carried out in the local center for psychosocial care (Centro de Atenção Psicosossial – CAPS) or the agreed CAPS of other municipalities of the same region?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
13. Are patients referred to a psychiatric hospital?
For example, consider the following options: 1) the patient is treated at the municipal emergency care unit and referred to the psychiatric hospital as soon as possible; 2) the patient is approached by the mobile emergency care service SAMU (Serviço de Atendimento Móvel de Urgência) and taken directly to the nearest psychiatric hospital; 3) the CAPS approaches the patient, but due to the lack of overnight admission or beds in general hospitals, they are referred to the psychiatric hospital.
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
In your municipality, regarding assistance to patients with psychological distress due to alcohol or other drug abuse:
14. Are severe cases treated at a local CAPS or the agreed CAPS of other municipalities of the same region?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
15. Is assistance performed in therapeutic communities?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
16. Do children and adolescents with psychological distress receive care in municipal services?
⚪ Always, in a CAPS for children and adolescents (or another specific service) or a basic health unit, depending on the severity of each case.
⚪ Most of the time, in a CAPS for children and adolescents (or another specific service) or a basic health unit, depending on the severity of each case.
⚪ Sometimes, in a CAPS for children and adolescents (or another specific service) or a basic health unit, depending on the severity of each case.
⚪ Sometimes, in services not specific to the health network
⚪ Rarely     ⚪ Never     ⚪ I don’t know         ⚪ N/A
17. Does the municipality promote sports and/or cultural activities in the community ?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
18. Are strategies carried out in your municipality to prevent the harmful use of alcohol or other drugs?
⚪ Yes   ⚪ No   ⚪ I don’t know   ⚪ N/A
If yes to the previous question (18), which ones?
Psychosocial rehabilitation:
Set of actions intended to increase the social skills of the subject, helping their social integration.
Regarding the psychosocial rehabilitation activities carried out in your municipality:
19. Are workshops with marketed products held?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
20. Are psychosocial interventions carried out to integrate the user into social networks? For example, encouraging the user’s participation in family or community groups, among others
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
21. Are actions carried out to insert the user of the PCN into the job market?
⚪ Yes   ⚪ No   ⚪ I don’t know   ⚪ N/A
22. Do PCN professionals encourage users to create and/or participate in cooperatives or solidarity ventures that facilitate their productive insertion?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
23. Do PCN professionals perform actions to increase the users' autonomy?
For example, assistance in requesting identity cards and employment record books, preparing curriculum vitae and regularizing banking issues, among others
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
24. Are there actions to deinstitutionalize long-term users of psychiatric hospitals or other institutions?
For example, user insertion in therapeutic residential services, restoring family and/or social ties, and interactions between hospitalized people and their families and citizens in spaces to promote meetings within the hospital and other parts of the city.
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
II. Connectivity component
Regarding the relationships between the services of the PCN of your municipality:
25. Do the various services act independently, without contacting each other?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
26. Does your municipality have collaborative care actions in mental health?
Collaborative work between a team specialized in mental health and teams responsible for basic healthcare, hospital care, or urgent/emergency actions to expand and complement the actions. These actions can occur through or independently of the nuclei for the family healthcare (NASF)
⚪ Yes   ⚪ No   ⚪ I don’t know   ⚪ N/A
27. Do the collaborative care actions include case discussions or shared care?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
28. How often do professionals of the collaborative care actions go to the units included in the main facility?
⚪ Weekly   O Biweekly   ⚪ Monthly   ⚪ Bimonthly or less   ⚪ I don’t know ⚪ N/A
29. Is the collaborative care actions involved in mental health from primary healthcare?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
30. Is there involvement of the collaborative care actions in mental health from the urgency and emergency network?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
31. Is there involvement of the collaborative care actions in mental health from the reference hospital network?
Local or regional hospitals admitting mental health users
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
32. Mental health professionals in your municipality develop actions together with other areas of the municipal administration, such as:
Leisure, justice, education, sports, social work, and guardianship council, among others
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
Regarding the articulation of the PCN in your municipality, how often are the activities below carried out?
33. Case discussions between the team assisting the user and the hospital team when users are hospitalized for mental health reasons
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
34. Network meetings
With the participation of representatives of various PCN services
⚪ Monthly or more frequent   ⚪ Bimonthly
⚪ Variable frequency, according to demand (periodicity higher than bimonthly)
⚪ Not performed         ⚪ I don’t know         ⚪ N/A
III. Integration component
Governance
35. Does the municipality have a specific coordinator for mental health?
⚪ Yes   ⚪ No   ⚪ I don’t know   ⚪ N/A
36. Does your municipality have agreements with other municipalities of the same region to share the PCN services?
Whether receiving or sending users
⚪ Yes   ⚪ No   ⚪ I don’t know   ⚪ N/A
37. If you responded “yes” to the previous question (36), do you think that the agreements in which you participate are suitable for the care of the PCN users?
For example, some municipalities have an agreement to use the PCN of a neighboring municipality, though, in practice, there is no available transportation, or the distances d ⚪ Not justify the patient travel to a one-day stay
⚪ Very Effective   O Effective   O Reasonably Effective   O Little Effective
⚪ Not effective     ⚪ I don’t know   ⚪ N/A
Care management
Regarding the monitoring of mental health users in your municipality:
38. Is one (or more) reference professional responsible for the user's follow-up?
The professional responsible for monitoring the user, the individual therapeutic project, and the primary contact with the family, community, and job networks
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
39. Are there agreed-on flow channels for users to circulate through the various health services of the local PCN?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
40. In the transition between services, is there a discussion among professionals to come up with a shared care approach?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
41. During the user’s journey through the PCN services, are professionals from different services jointly responsible for monitoring the case?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
IV - Normativity component
Mental health policy
42. Are the mental health actions of your municipality based on the guidelines of the national mental health policy?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
43. Are your municipality’s mental health professionals aware of the PCN operation in your region?
Regarding flows, demands, and service protocols
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
44. Are there actions to evaluate and monitor mental healthcare in your municipality?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
If you selected “always,” “most of the time,” or “sometimes” in the previous question (44):
How are these actions carried out?
Participation and social control
45. Do the users of the PCN in your municipality participate in assemblies, councils, commissions, forums, or other spaces for collective discussion on mental health?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
46. Do user assemblies take place in the mental health services of your municipality?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
47. If you selected “always,” “most of the time,” or “sometimes” in the previous question (46): Are user assemblies deliberative (i.e., have decision-making power after consultation or reflection)?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
48. Do users and/or their families participate in decisions about their treatment?
For example, having room to give an opinion on which drugs to take, workshops to attend, and family member to contact
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
49. Do the PCN professionals in your municipality participate in assemblies, committees, forums, or other spaces for collective discussions on mental health?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
V — Structural dimension
Health Education
Concerning the actions for health education of the professionals of the PCN of your municipality:
50. Does the municipality offer educational activities in mental health for PCN professionals?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
51. Does the municipality encourage the participation of PCN professionals in educational activities?
For example, giving salary incentives to graduate completion, per diem, partial or full workload release, and a career development plan
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
52. For selecting professionals to work in mental health services in your municipality, is the experience in related works or the appropriate training in mental health considered?
For example, postgraduate certification in mental health or other related courses
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
Logistics System
53. Does the mental healthcare in your municipality have a vehicle available for transporting users, or does it provide mechanisms for their transportation to the service, such as transport vouchers?
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
54. Does the mental healthcare in your municipality have a vehicle for transporting professionals in service?
For example: for home care, medication application, and collaborative care activities
⚪ Always   ⚪ Most of the time   ⚪ Sometimes ⚪ Rarely   ⚪ Never ⚪ I don’t know ⚪ N/A
55. Is there a unified electronic medical record accessible at different healthcare points in your municipality?
⚪ Yes   ⚪ No   ⚪ I don’t know   ⚪ N/A

Editor: Jane Russo

Publication Dates

• Publication in this collection
  05 May 2023
• Date of issue
  2023

History

• Received
  21 Mar 2021
• Accepted
  10 Feb 2022
• Reviewed
  21 Nov 2021