Patients and physician’s self-assessment regarding clinical stability in severe mental disorders: a cross-sectional study

Objective: This study explores the relationship between patients’ self-assessment and physicians’ evaluation regarding clinical stability. Methods: This cross-sectional study was carried out at the general outpatient clinic of the Instituto de Psiquiatria da Universidade Federal do Rio de Janeiro (IPUB-UFRJ) in a large sample (1,447) of outpatients, of which 67.9% were patients with severe mental disorders (SMD). We collected information using a structured questionnaire developed for this purpose, filled in by the patient’s physician. Clinical stability was assessed by means of five psychiatric instability criteria and by the physician’s global clinical impression over the six previous months. The patients’ self-assessment was based on a question about how they evaluated their health status: stable/better, worse, does not know. For the analyses, patients’ self-evaluation was considered as our standard. Results: The sample was composed of 824 (57%) women with an average age of 49 years. The most prevalent diagnoses within the SMD category corresponded to 937 patients, of whom 846 (90.3%) assessed themselves as stable/better. The physicians’ evaluations agreed more with patients with bipolar disorders and less with schizophrenics regarding stability. As for patients with depressive disorder, physicians agreed more with them regarding instability. Conclusion: The data analysis confirms our hypothesis that the selfassessment made by patients with SMD was accurate regarding their health condition, and that the selfassessment made by patients who considered themselves stable agree with the physicians’ evaluation.

In psychiatric patients with severe mental disorder (SMD), particularly those with psychotic conditions, the four elements mentioned above are frequently defective 4,9,[14][15][16][17]23 , a fact that can jeopardize the accuracy and reliability of selfassessments in this population. No surprisingly, patients with SMD provide inaccurate self-assessment reports compared with other information, including informant reports, due to different degrees of impairment of the investigated area 3,4,9,[11][12][13][14]16 .
The concept of insight encompasses psychology, psychiatry and neurology and is better seen as a multidimensional phenomenon 3,7,23 .It comprehends at least three fundamental characteristics: the person's self-awareness that he or she is suffering from an illness; understanding the cause and source of specific symptoms; and recognizing the need for treatment 3,4,7,23,24 .Although most of the studies on insight have been carried out in psychosis, lack of insight is a symptom also of bipolar spectrum 25,26 , anxiety, obsessive-compulsive 25,27 , eating 25,28 , and neurological disorders 25,29 , among others.
A more complete view on the term "insight" encompasses the notion of cognitive insight 7,8,30 -a type of metacognition 5,[7][8][9][10]25,29,31 that refers to a patient's ability to self-reflect and to the level of self-certainty they feel in the interpretation they give to their unusual experiences 8,25,30 . A comonly used and well-validated measure of cognitive insight is the Beck Cognitive Insight Scale (BCIS) 30 , a 15-item self-assessment questionnaire with a self-reflectiveness subscale and a self-certainty subscale.There is evidence of both overlap and distinctiveness between cognitive insight and clinical insight, and this might have clinical implications 3,9,30 .
In a self-evaluative process, cognitive insight has been hypothesized to be associated with neurocognitive function 30 .Exploring this association, several studies have focused on psychotic disorders and have found different correlations 7 : 1. a positive correlation between self-reflectiveness and better neurocognitive functioning, and 2. a link between stronger self-certainty and worse neurocognitive functioning.According to these studies, the relationship between clinical insight, cognitive insight and neurocognition is very complex and it would be an interesting theme for future research 7 .These three domains of impaired awareness are referred to as Introspective Accuracy (IA) and fail to correlate with each other 12,16 .Previous studies about IA in patients with schizophrenia have shown that poor performers in cognitive and functional assessments are more likely to be inaccurate in their self-assessments of cognition and functioning 12 .
Due to discrepancies in terminology and meaning related to the knowledge individuals have about themselves, which may be referred to as consciousness, awareness, insight, self-knowledge, etc., for empirical research it is important to make a distinction between the terms insight and awareness, to clarify what the phenomenon under assessment is about 32 .This distinction is based on differences between the "objects" of such self-knowledge.In neuroscience, the object refers to impairment of function and the correspondent phenomenon of self-knowledge is awareness.It is based on a direct appraisal of impairment and it is evaluated quantitatively.In clinical psychology/psychiatry, the object refers to mental symptoms/disorders, subjective experiences, and the corresponding phenomenon is insight.Based on both direct and indirect appraisals of change, they are evaluated quantitatively and qualitatively 32 .
Despite the importance of assessment for empirical research and clinical psychology/psychiatry, we have not identified studies comparing SMD patients' subjective selfassessment of their own health status with their physician assistant's clinical evaluation.Therefore, the aim of this study is to compare the evaluations, performed through a structured questionnaire filled in by the patient's physician, in a sample of outpatients treated in the Instituto de Psiquiatria da Universidade Federal do Rio de Janeiro (IPUB-UFRJ), Brazil.
The current article expands on our previous work about clinical stability 33 by reporting on the following evaluations: 1. Clinical stability assessment based on five psychiatric instability criteria over the six previous months; 2. the physician's global clinical impression, and 3. the patient's subjective self-assessment.Our hypothesis is that the selfassessments of patients with SMD are accurate regarding their health status and the self-assessment made by patients who considered themselves stable agree with the evaluation of the psychiatrist.

Study's location and design
The IPUB's outpatient clinic is a university-based mental health facility totally included in Brazil's National Healthcare System.It provides individual consultations in Psychiatry, Psychology, and Nursing, in addition to teaching and research activities.Particularly in Psychiatry, the outpatient clinic assists around 1,300 patients per month, most of them referred from Primary Care.Its clientele comes from all regions of the city of Rio de Janeiro and macro-regions of the State of Rio de Janeiro 33 .As an academic institution, IPUB's outpatient clinic is the setting for the practice and supervision of psychiatry interns.The clinic provides care for both long-term and recently diagnosed patients; thus, interns have the opportunity to provide clinical follow-up for the same patient during 2 or 3 years.After that period, the assistant intern changes, but not the supervising staff, and the patients are not discharged -therefore, many of them have been under the indirect care of the staff for more than two decades.
This investigation has a descriptive cross-sectional design and presents data collected from July to October 2015.It is a retrospective study deriving from the research "The general psychiatry outpatient clinic of IPUB and the proposal for the Exit Door: A sociodemographic assessment of the patients".The study was fully approved by the local Institutional Review Board (CAAE 45260315.2.0000.5263).Informed consent was waived.

Participants
We aimed to investigate all individuals scheduled at the outpatient clinic for three consecutive months.The patients who were not being followed up by the same physician in the six months prior to the study were excluded.The sample corresponded to 1,447 participants, and all of them were included regardless of diagnosis and clinical status.

Material
The evaluations were performed using a specific structured questionnaire developed by the authors, composed of three aspects: 1. sociodemographic variables and clinical information; 2. clinical stability assessment based on an overall subjective clinical impression and on five objective clinical criteria; 3. patients' subjective self-evaluation.
The sociodemographic variables and the clinical information were collected from the patient's medical record.The clinical diagnosis according to ICD-10 was based on the physician assistant's clinical assessment.The five criteria concerning clinical stability were selected from relapse factors found in the literature: 1. Occurrence of psychiatric hospitalization; 2. Exacerbation or emergence of acute manifestations of the disease; 3. Change in medication or a significant increase in the doses used to treat the underlying disease; 4. Significant suicidal ideation and/or suicide attempt; 5. Worsening of the primary mental disorder due to psychoactive substance use/abuse 33 .The sixth item was an open one, to be filled in by the physician assistant with other criteria considered relevant, such as: psychosocial criteria, clinical diseases, non-adherence to treatment, etc.The negative answer to all the five criteria was our gold standard for Clinical Stability.Therefore, we considered that patients with one single affirmative answer were unstable.Finally, a third assessment level was proposed: the patients' own impression about his/her health status.The instrument is presented in Table 1.

Procedure
The instrument was filled in by trained psychiatrists.They were oriented about the objective of the research, the psychosocial care network, the patients' medical evaluations, the patients' self-assessment, and the indicated moment for data collection.

Comparing the assessments
The concordance between the physicians' evaluations (overall clinical impression versus five clinical criteria) was considered meaningful agreement (K = 0.68; SE = 0.026; p < 0.0001) 33 .For this study, we chose the physicians' subjective clinical impression for the analysis between physicians' assessment and patients' self-assessment.
In order to compare the physicians' evaluations and the patients' self-assessment, the patients' answers about their health status were converted to a dichotomous response: 1. Better/Stable = Yes; 2. Worse and Does not know = No.

Statistical analysis
For the investigated aspects, we obtained measures of central tendency (means) and relative frequencies (proportions) with 95% confidence intervals.To check the statistical significance of the differences, we used Student's t-test for means and J Bras Psiquiatr.2022;71(2):74-82 chi-square test for proportions.Agreement between stability assessed by the criterion of the physician's clinical impression and stability assessed through the patient's self-evaluation was assessed through kappa and contingency tables, measuring sensibility, specificity, positive predictive value and negative predictive value.To accomplish the main aim of this study, the patients' self-evaluation was considered our standard during the calculations.
All the analyses were run by R basic statistics package, except for kappa, which used the FMSB library, and contingency tables, calculated by means of the epiR library.

Sociodemographic variables
The number of individuals involved in the study was 1,447.Regarding sex, 824 patients were women (57%).Age varied from 16 to 89 years, with average age of 49.13 (SD = 12.83).All Planning Areas of the city of Rio de Janeiro were considered in the distributions of the dwelling areas.A total of 343 (23.7%) patients out of 1,447 lived outside the city.These results were presented in our previous work 33 .
Among the five clinical criteria, the criterion "exacerbation or emergence of acute manifestations of the disease" presented the highest percentage of positive answers.The lower percentage corresponded to "hospitalization" 33 .The sixth criterion (an open criterion) was not included in the analysis due to the low number of answers it received.

Clinical stability by the five clinical criteria
The clinical stability of the sample was assessed in the six previous months.In the 1,447 participants, 946 (65.38%) [95% CI: 62.88-67.78]were evaluated as stable by means of the five criteria.This means that all the criteria received negative answers.Regarding stability by sex, the proportion was practically the same (p-value = 0.974).With respect to the SMD diagnostic categories, 646 patients out of 983 were considered stable by physicians.In this group, the categories F20-F29 and F30-F39 did not differ significantly (p-value = 0.22) regarding stability (Table 2).5][36] Y -N b.Was there exacerbation or emergence of acute manifestations of the disease? 34,36Y -N c.Was there a change in medication or a significant increase in the doses used to treat the underlying disease? 34,38Y -N d.Was there a significant suicidal ideation and/or suicide attempt? 34,35,385][36][37] Y -N f.Others:

Patient's subjective self-assessment versus physician's global clinical impression
A total of 1,051 (72.63%) out of the 1,447 patients were considered stable by the physician assistants.The relationship between the patients' self-assessment and the physicians' impression presented the following results: 1. Concerning stability: 1,004 (95.5%) patients out of 1,051 agreed with physicians and 47 (4.5%) disagreed; 2. Concerning instability: 114 (28.8%) patients out of 396 agreed with physicians and 282 (71.2%) disagreed.

Stability by SMD patients' subjective selfassessment versus stability by physician's global clinical impression
Our main analysis involved comparing the physicians' evaluation and the patients' self-assessment regarding stability.Table 4 shows that, in this comparison, the physicians' evaluation about patient stability predicted 82% [298/362] of the 362 patients with schizophrenia who considered themselves stable, but was less able to foresee instability achieving 63% [24/38] of the 38 patients who considered themselves unstable.As for the 319 patients with bipolar disorder who considered themselves stable, the physicians predicted 77% [247/319] and almost 100% of patients who considered themselves unstable (97% [31/32]).On the other hand, concerning the 57 patients with depressive episode who considered themselves stable, the physicians were less able to predict their self-evaluation (65% [37/57]), and predicted instability more accurately (86% [6/7]).Regarding the patients with recurrent depressive disorder, the physicians' assessment was similar to that of

DISCUSSION
To the best of our knowledge, this is the first study that assesses the clinical stability of patients with SMD assisted at a psychiatry outpatient clinic of a university.We expand on our previous work 33 by analyzing the relationship between the patient's self-assessment of their own health status and the physician's global impression.
We investigated a large sample consisting predominantly of female patients (57%) with average age of 49 years.The majority (72.63%) was considered clinically stable in the last six months, and 67.9% were diagnosed with SMD.According to the physicians' evaluations, there were no significant differences in the stability of patients belonging to SMD categories, nor regarding patients' sex.The prevalence of diagnoses considered SMD is in accordance with the institution's level of complexity, as, in addition to medical care, it offers hospitalization.The majority of the stable patients was being followed up with a quarterly periodicity and was being assisted at the institution for more than 10 years (47.1%) 33.Usually, according to the norms of IPUB's Medical Residency, patients are followed up for a period of 2-3 years by the same resident physician.
Concerning the 1,447 patients, when comparing the physicians' evaluation and the patients' self-evaluation, we found that there was a high level of agreement between patients who considered themselves stable and the medical assessment; on the other hand, we observed a significant level of disagreement between depressive patients who considered themselves stable/better but were assessed as unstable by the physicians.
For the purpose of analyzing the relationship between these evaluations, we considered the most prevalent diagnoses within the SMD categories: F-20, F-31, F-32 and F-33.They corresponded to 937 patients, excluding 464 patients who were not considered to have these diagnoses.Of these, 846 (90.3%) of the patients rated themselves as better/ stable.Having the patient's self-assessment as the standard for this analysis, when comparing the self-assessments of patients who considered themselves stable or unstable with the physician's evaluation, the degree of agreement between them adjusted by kappa can be interpreted as fair agreement, in all the SMD diagnostic categories.
Depressive episode requires additional comments.Its results were counterintuitive, as depressive subjects were expected to have a better insight and judgments closer to those of the physicians, when compared with psychotics.Although the kappa could be considered fair (0.23), it was one of the lowest among our results.This indicates that patients and doctors do not consider the same data when evaluating stability in depressive episodes."Fair" agreement is probably the result of shared and non-shared components evaluated by physicians and patients when judging stability.The shared components form a "latent factor" observed by both groups, but could not discriminate between "stable" and "unstable" cases.Overall, the outpatients' sample was considered particularly stable and the analysis revealed that the major part of the patients with SMD evaluated themselves as stable.Therefore, it is not surprising that the comparison between the stability evaluations of physicians and patients showed a significant agreement.Although the physicians had good knowledge of their patients, we highlight the level of disagreement between the two evaluations, mainly regarding instability in patients with schizophrenia and stability in patients with depressive disorders.The statistical analyses showed that the chance of a physician identifying instability in patients who considered themselves unstable was 63% for schizophrenic patients, and, for the identification of stability in depressive patients who considered themselves stable, the chance was 65%.
In this study, the object of the patients' self-assessment is knowledge about their own health status in a broad sense.This encompasses self-perception of their body, subjective mental health status, and awareness of the presence of a mental disorder or mental symptom.This self-assessment presupposes capacity of insight and its object, self-knowledge about one's own health status, can be understood as a sense of insight with a wide scope, in line with Marková's terminology 32 , although it does not refer only a specific disorder, symptom or impairment of function.Instead, it is a more abstract and complex construct related to aspects of direct self-experience 32 , consistent with metacognitive skills, as it includes awareness of thought processes and reasoning styles 7,8,30 .
The major part of the studies about insight and/or self-assessments select patients based on clinical stability criteria, such as: 1. no change in medication dosage 10,13 ; 2. no dependence on active substances 31 ; 3. no crises or hospitalizations during a period of time before recruitment for the study 5,10,13,31 .In our study, we included all patients scheduled for outpatient consultation at the institution in a period of three consecutive months, and only patients who were being followed up by the same physician for less than six months were excluded.Therefore, all patients, regardless of diagnosis and clinical status, were evaluated and were able to evaluate themselves.
In psychiatry, many studies about insight and selfassessment are conducted with patients with schizophrenia and patients with bipolar disorder, with or without psychosis and depression, and their evaluations are usually performed by clinicians.These studies seek to identify different domains of insight, the course of the disease, the patient's perception of the disease, treatment adherence and everyday functioning, among others.Usually, they employ rating scales 3,5,[10][11][12][13][14]30 , questionnaires and tests 6,12,31 in the evaluations. Inour study, the object of medical assessment was the clinical stability of patients undergoing outpatient follow-up, most of whom were patients with SMD.This medical evaluation was carried out both objectively and subjectively.As for patient's selfassessment, they were asked about their own health status, independently of being considered stable or unstable by the physician.
In our study, the high concordance between the evaluations is related to patients' clinical stability.We believe that the stability of our sample is due, among other factors, to the reduction of the psychopathological symptoms, the connection with the institution, the patients' certainty that they will be followed up at the same outpatient clinic, and to assistances provided on a regular basis.In addition to pharmacological treatment, several patients have been accompanied by psychologists and some have participated in activities at the IPUB's Daily Care Center.The meta-analysis conducted by Mintz et al. 6 corroborates our analysis, as it showed that, when the psychotic episode resolves and the positive symptoms ameliorate, patients are more accurate in their interpretations of perceptions and more aware of the disorder.Furthermore, we identified similarities with the longitudinal study carried out by Lysaker et al. 31 , which included patients with schizophrenia or schizoaffective disorder in a post-acute phase of illness in a six-month baseline.To the authors, cognitive insight is a constant construct in individuals with psychotic disorder who are stabilized.It is not without reason that many studies in this field seek to evaluate patients in conditions of psychological stability.
Globally, our results are consistent with the literature regarding insight and self-assessment in patients with SMD.In relation to schizophrenia, the disagreement between physicians and patients, as the former considered the latter unstable while the latter considered themselves stable, suggest that patients have a poor insight, hence difficulty in being aware of mental disorders and its impairments.The same applies to bipolar disorder.In patients with depression (F32-F33), the disagreement related to stability suggest that these patients have a different perception compared to the physicians' and, possibly, greater self-reflection and an accurate self-assessment.This result could well be in line with the findings of Demyttenaere et al. 39 that physicians and patients seem to differ in their ranking of what is most important for cure in depression.
Additionally, our results offer another possibility of reasoning.Having the patients' self-assessment as our standard, why were the physicians good in identifying stability, especially in mood disorders (F31, F32, F33), and not so good in identifying instability in patients with schizophrenia or bipolar disorder?Obviously, the medical evaluations and the patients' subjective self-assessment have distinct epistemological statuses, but was there a systematic error?Were they based on another attribute?It is probably safe to say that, when the physician identified instability and the patient agreed, this patient was unstable, but the point is: what about when the patient disagreed?In our view, not necessarily patients were so psychically impaired that they could not perceive their own condition.Perhaps they did not consider things that they had already become accustomed to or learned how to deal with as problems, such as: hearing voices, apathy, disorganized thoughts, etc. Regarding patients with depression, the physicians were able to identify patients who considered themselves unstable and they considered several patients who were feeling well as unstable.In the latter case, physicians were probably considering as problems things that the patients were not experiencing as such.As Demyttenaere et al. 39 states, physicians mainly focus on alleviation of depressive symptoms while patients mainly focus on the restoration of quality of life, functioning and positive affect.
Generally, the medical assessment of the patient's health status is considered the most reliable approach, given that physicians are well trained to identify diseases and to apply assessment tools.For them, a set of signs and symptoms defines a particular disease or mental status.At this point, we agree with Hunt et al. 19 and add that subjective measures can provide an important complement to objective measures because they give direct access to the perceived health status.Furthermore, in psychiatry, measures of insight should give attention to the impact of interpersonal, cultural, and socio-economic contexts, as Tranulis et al. 21states.We believe that in the physician-patient relationship, taking into account the patients' narrative can reduce the chance of physicians misconstruing a patients' report of their selfevaluation and can also create a basis for narrowing the gap between lived experience and the scientific account.
Our study presents some limitations, such as: a) the nonutilization of a standardized instrument to obtain psychiatric diagnoses; b) difficulty in assessing stability based on a nonlongitudinal study design; c) the non-utilization of validated clinical assessment scales; d) the non-utilization of validated self-assessment scales; e) the non-utilization of validated discrepancy scales.
However, this is the first study carried out with a sample that, instead of selecting, included all patients with scheduled consultations in the study's period.The non-utilization of a clinical scale was an intentional option that aimed to make the assessment resemble, as much as possible, a real assistance provided for patients.Also, we sought to use a more naturalistic method in which patients could spontaneously evaluate themselves.
Having as our standard the patient's point of view and what has been found, our study provides significant implications for medical practice and favors the empathic understanding of patients and their illnesses.Further research should investigate the discrepancy between the physician's assessment and the patient's self-assessment from the patient's point of view.
Our study, conducted with a sample of psychiatric outpatients, shows that most patients had SMD and were considered stable by physicians and by themselves.The correlation between the physicians' evaluation and the patients' self-assessment suggest that the discrepancy is related to level of insight and to the different ways in which it is possible to measure and understand stability.

1 .
Sociodemographic and clinical information: a. Initials of patient: b.Sex: F -M c.Number of Medical Record: d.Date of Birth: __/__/____ e. ICD-10*: _____-___ (__________________________________) Other: ____ -___ f.Which is the periodicity of the consultations?Weekly, fortnightly, monthly, bimonthly, quarterly, other: g.Being followed up at IPUB since: ________ (year) h.Dwelling area (Planning Area or district and city): 2. Clinical stability assessment 2.1.Subjective clinical impression: a.In relation to the patient's clinical stability, do you consider that your patient was stable in the last six months?Y -N 2.2.Objective clinical assessment:

3 .
Patients' self-assessment: a.How does your patient evaluate his/her health status?Better/stable -worse -does not know * International Statistical Classification of Diseases and Related Health Problems 10 th Revision.

Table 1 .
Sociodemographic Data and Psychiatric Clinical Assessment Questionnaire

Table 2 .
Sample distribution of the frequency of stability by clinical criteria according to sex and ICD-10 diagnostic categories -Rio de Janeiro, RJ 2 Pearson's chi2(1) = 0.0011; p-value = 0.974.b p-value between the clinical stability of the diagnostic categories (p = 0.22).

Table 3 .
Distribution of the frequency of patients' subjective self-assessment according to the ICD-10 diagnostic categories -Rio de Janeiro, RJ (n = 937) * International Statistical Classification of Diseases and Related Health Problems 10 th Revision.