How do experts recognize schizophrenia: the role of the disorganization symptom

Como os especialistas reconhecem a esquizofrenia: o papel do sintoma desorganização

ORIGINAL ARTICLE

How do experts recognize schizophrenia: the role of the disorganization symptom

Como os especialistas reconhecem a esquizofrenia: o papel do sintoma desorganização

Denise Razzouk^I; Jair de Jesus Mari^I; Itiro Shirakawa^I; Jacques Wainer^II,III; Daniel Sigulem^II

Psychiatry Department, Universidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil

^IPsychiatry Department, Universidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil

^IIDepartment of Medical Informatics, Universidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil

^IIIComputing Institute, Universidade Estadual de Campinas (UNICAMP), Campinas (SP), Brazil

^{Correspondence}

ABSTRACT

OBJETIVE: Research on clinical reasoning has been useful in developing expert systems. These tools are based on Artificial Intelligence techniques which assist the physician in the diagnosis of complex diseases. The development of these systems is based on a cognitive model extracted through the identification of the clinical reasoning patterns applied by experts within the clinical decision-making context. This study describes the method of knowledge acquisition for the identification of the triggering symptoms used in the reasoning of three experts for the diagnosis of schizophrenia.

METHOD: Three experts on schizophrenia, from two University centers in Sao Paulo, were interviewed and asked to identify and to represent the triggering symptoms for the diagnosis of schizophrenia according to the graph methodology.

RESULTS: Graph methodology showed a remarkable disagreement on how the three experts established their diagnosis of schizophrenia. They differed in their choice of triggering-symptoms for the diagnosis of schizophrenia: disorganization, blunted affect and thought disturbances.

CONCLUSIONS: The results indicate substantial differences between the experts as to their diagnostic reasoning patterns, probably under the influence of different theoretical tendencies. The disorganization symptom was considered to be the more appropriate to represent the heterogeneity of schizophrenia and also, to further develop an expert system for the diagnosis of schizophrenia.

Keywords: Artificial intelligence; Expert systems; Knowledge acquisition; Schizophrenia; Psychotic disorders

RESUMO

OBJETIVO: As pesquisas sobre o raciocínio clínico foram importantes para o surgimento de sistemas de apoio à decisão diagnóstica. Essas ferramentas são desenvolvidas por meio de técnicas de inteligência artificial e têm com objetivo principal auxiliar o médico no diagnóstico de doenças complexas. A abordagem utilizada para a construção desses sistemas constitui na formulação de um modelo baseado na identificação de padrões no raciocínio dos expertos quando de uma tomada de decisão diagnóstica. Este estudo descreve a metodologia empregada para identificar os elementos-chave utilizados no raciocínio de três expertos no processo de diagnóstico do transtorno da esquizofrenia.

MÉTODO: Para explorar o raciocínio clínico foram selecionados três expertos em esquizofrenia de dois centros universitários de São Paulo. Foi utilizado o método dos grafos, por meio do qual o experto podia esquematizar a combinação de sintomas-chave que ele utilizava para identificar um diagnóstico de esquizofrenia.

RESULTADOS: A partir da análise qualitativa dos grafos foi possível notar uma diferença marcante nos padrões de raciocínio diagnóstico. Essa diferença ocorreu, sobretudo, nos sintomas-chave do processo de decisão diagnóstica: desorganização, afeto embotado e distúrbio do pensamento.

CONCLUSÕES: Os resultados apontam para uma diferença substancial entre os expertos quanto a um padrão de raciocínio diagnóstico provavelmente influenciado por diferentes correntes teóricas. Essas diferenças constituem um impedimento para a construção de um modelo único. O sintoma desorganização foi considerado o elemento-chave mais apropriado para representar a heterogeneidade da esquizofrenia e ser modelado para a construção de sistema de apoio à decisão diagnóstica.

Descritores: Inteligência artificial; Sistemas especialistas; Aquisição de conhecimento; Esquizofrenia; Transtornos psicóticos

Introduction

During the last three decades researchers have been exploring the minds of expert physicians in order to investigate how they established their diagnoses.^1-4 Experts acquire their knowledge through practice, personal skills and technical information. Their reasoning is characterized by efficacious and cognitive shortcuts triggered by a few elements such as signs and symptoms.⁵ Since the 1970's, artificial intelligence techniques have been applied to develop expert systems.² The main focus of the research in this area was to transfer human skills into intelligent tools.

Expert systems or decision support systems are computerized tools that simulate human reasoning. They provide advice on the choice of adequate prescriptions, adverse drug interactions, and suggest the most probable diagnosis.^6-8 These systems comprise a knowledge base and an inference engine (program that reasons with the computerized understanding of the knowledge base and thus translates it into natural language for the end user). The rationale for the construction of a knowledge base relies on the identification of a set of rules or a pattern of distinct features used by the experts to recognize a diagnosis within a clinical problem-solving situation. In other words, the knowledge base is a database that embodies the knowledge extracted from the experts, i.e., the translation of expertise into cognitive models.⁹ The knowledge base can thus be used as a reference source by the end user to whom an explanation may be given as to how the diagnosis was attained. Accordingly, expert systems should provide useful information pathways in the organization of a clinical reasoning model for students or physicians from other specialties.⁹

However, these tools are still too incipient to explore the complexity of clinical decisions and the shortcomings of medical knowledge. Expert systems are most useful for medical specialties in which a well-established etiopathogenesis accounts for a disease model.² Since the etiopathogenesis of most psychiatric disorders is unknown, a definitive disease model and its constructs have not yet been validated.^10-11 Besides, psychiatric symptoms are heterogeneous in origin, form and structure and their identification is affected by the way in which the psychiatrist mentally portrays such symptoms.^12-13 As a result, experts have based their diagnosis on abstract models, organized according to their own clinical experience.

The development of expert systems comprises three stages: knowledge acquisition, modeling knowledge and evaluation of performance of the prototype. The first step refers to the extraction and organization of knowledge to further construct a knowledge base. This is a complex process that requires selecting, compiling and organizing the best available knowledge into a specific domain.^14-15 The main issue, especially with regard to psychiatry, is the difficulty to select the best available knowledge by using standardized criteria for the knowledge acquisition process. In psychiatry, and especially in schizophrenia and psychotic disorders, expert systems are rare due to problems with the accuracy and validity of its constructs.^16-18 The available methods for knowledge acquisition are not standardized and are based on qualitative techniques such as 'brainstorms', open interviews and discourse analysis.

1. Objective

This study describes all the steps of the knowledge acquisition of expert system for the diagnosis of schizophrenia: to identify reasoning patterns for the diagnosis of schizophrenia, and to compare these patterns between three experts and to select a sharable reasoning pattern between them. The main reason for developing this system is to provide physicians with a tool that leads to the accurate and early diagnosis of schizophrenia. The schizophrenia domain was chiefly selected due to the difficulties encountered to reach a diagnosis and to the need of early clinical intervention to prevent a worse prognosis.

Method

Three experts in schizophrenia were selected from two universities of the state of Sao Paulo. The selection criteria were based on their more than fifteen years of expertise in psychiatry, especially with schizophrenia, and on the fact that they had published articles on schizophrenia and were currently developing academic activities in this domain. The experts were interviewed by a psychiatrist (DR) with expertise in the domain of schizophrenia: more than fifty years of clinical practice with psychotic disorders (inpatient and outpatient). Four clinical vignettes on schizophrenia and schizophreniform disorders were elaborated based on charts obtained from the Outpatient Schizophrenia Program of the Universidade Federal de São Paulo. Charts were selected based on the quality of descriptions of the psychopathological symptoms. The main purpose was to elicit from the experts a report on how they reached their diagnosis of schizophrenia. In this first phase each vignette was shown to the expert who was therefore asked to select in the text the symptoms that corresponded to the diagnosis of schizophrenia and to mention the ones that were absent. In the second phase the experts were asked to rank the symptoms they had selected in the first phase according to their relevance for the diagnosis of schizophrenia. The purpose of the third phase was to employ the graph methodology developed by Leão & Rocha¹⁹ to represent the decision-making process in a clinical problem-solving situation. In this situation, the experts were asked to build graphs representing the diagnosis of schizophrenia. The node and the vertices composed a graph. In this case the node represented the diagnosis of schizophrenia and the vertices represented the most important features (called symptoms) associated with the diagnosis of schizophrenia. To build the graphs the experts selected only the symptoms required for the diagnosis of schizophrenia. They were allowed to construct more than one graph for each case. Each symptom on the graph received a score ranging from 0 to 10 according to its specificity for the diagnosis of schizophrenia. A qualitative analysis was used for the comparison of the graphs produced by the three experts. Afterwards, the most frequent symptoms with the highest scores presented in the graphs were considered to be the triggering symptoms. These features, named as triggering symptoms, were considered to be the main elements involved in the diagnosis process.

The project was approved by the UNIFESP's Ethics Committee (process number 715/98).

Results

The experts constructed nineteen diagnostic decision graphs to represent schizophrenia for the three vignettes. No graph was made from vignette 4, which corresponded to the Schizophreniform disorder but there was perfect agreement between them (Table 1).

In a comparison between the diagnoses of the four vignettes made by the three experts, there was disagreement particularly with regards to vignette 2 (Table 1). Considering vignette 2, in which the diagnostic agreement was less satisfactory than the others, of note, they shared only the delusion symptom but reached different conclusions (Figure 1). The experts 1 and 2 have agreed on their diagnoses but have not shared any other symptoms in the graphs. The experts 2 and 3 have disagreed on their diagnoses but have shared three symptoms. However, the differences as to how to attain the diagnosis became evident in case 3, in which the diagnostic agreement was good (Table 1) but there was no common symptom between the three experts (Figure 2).

The clinical reasoning patterns of the three experts were different. Expert 1 constructed eight graphs in which there was predominance towards disorganization, delusion and Schneider's first-rank symptoms. Expert 2 sketched out four graphs with emphasis on thought disorder, cognitive impairment and delusion. On the other hand, the eight graphs made by Expert 3 emphasized on blunted affect symptoms and Schneider's first-rank symptoms. The triggering symptoms, which are the most important symptoms for the diagnosis of schizophrenia, were identified according to their frequency and scores on the graphs.²⁰ Disorganization, blunted affect and Schneider's first-rank symptoms had the highest weight and frequency on the graphs. Among these symptoms, disorganization (values: 8-10) and blunted affect (values: 8-10) showed the smallest variability in the attributed values and also had the highest median. Consequently, they were considered to be the triggering symptoms for the diagnosis of schizophrenia. Thought disorder had lower values (median 7.0) as compared to those of disorganization (median 9.0) and blunted affect (median 9.0). However, thought disorder was present in all the graphs constructed by one of the experts, which was therefore considered as being the triggering symptom for that expert's diagnostic decision. Schneider's first-rank symptoms were not considered as triggering symptoms as different experts had used them, although none of them chose them as a pattern to diagnose schizophrenia. The term disorganization was particularly used by one of the experts to define a group of features comprised of negative symptoms, personality changes, inadequate behavior and socio-occupational dysfunction.

Discussion

The analysis demonstrated that the three experts did not share a common reasoning pattern to identify schizophrenia. Thus, it was not possible to identify a consensus on which symptoms were the most adequate for the decision-making diagnostic process. The triggering symptom for the diagnosis of schizophrenia was not the same for all three experts: blunted affect, thought disturbances and disorganization. However, the three experts agreed mainly regarding symptoms such as delusion and Schneider's first-rank symptoms, although these symptoms are frequently present in other disorders and are not adequate to differentiate schizophrenia from other psychoses.^21-23 Therefore, there was a consensus between the experts only regarding unspecific symptoms. The operational diagnostic criteria were developed based on consensus among experts, but these criteria require a clinical judgment to decide about the presence of symptoms. For instance, the ICD-10 allows the diagnosis of schizophrenia based on generic symptoms such as positive symptoms. However, none of the three experts made their diagnosis of schizophrenia based mainly on positive symptoms. On the other hand, the DSM-IV criteria for schizophrenia disorder require at least six months of symptoms and also socio-occupational dysfunction. However, the three experts made diagnosis of schizophrenia even when these requirements were absent. Probably the core of the diagnostic process was based on a particular clinical reasoning and not on algorithms or consensual rules. Thus, it is reasonable to assume that the different triggering symptoms employed by the experts were due to differences in their understanding of the construct of schizophrenia. This can be explained by the heterogeneity of schizophrenia in which multiple combinations of symptoms lead to the same diagnosis. Vignette 3 was a typical example in which the triggering symptoms did not coincide, although there was agreement between the experts in relation to the diagnostic hypothesis. Vignette 2 was a typical example in which the symptoms partially coincide although there was no perfect agreement between the experts as to their diagnoses. In the first example, there would have been no problem in constructing a knowledge base because the identification of one valid pathway suffices to represent a reasoning schema for the diagnosis of schizophrenia. The second example, however, could have been a considerable hindrance for the knowledge acquisition since there would have been no way to know which combined pathway had been correct.

In other words, they agreed for unspecific symptoms and disagreed for the most specific symptoms. It may be interpreted that they did not share a common reasoning pathway and therefore, it was not possible to organize these reasoning patterns into a unique diagnostic-decision model.

Furthermore, the construction of the knowledge base also involves the choice of a theoretical model. Therefore, we analyzed the consequences of choosing one of the three reasoning patterns, e.g., if the blunted affect had been chosen as the diagnostic-triggering symptom to construct the knowledge base, it would have been difficult to identify the first psychotic episode, and the concept of schizophrenia would have been narrowed. On the other hand, if thought disturbances had been considered as the triggering-symptom, it would have been difficult to differentiate schizophrenia from schizoaffective and affective disorders. Therefore, the third reasoning pattern would have been quite useful because the disorganization symptom was described by the expert as a broader concept that comprises a cluster of symptoms: negative symptoms, behavioral disorders, personality changes and problems with social adaptation. This concept involved different aspects, which were in accordance with the researchers' findings, such as a greater specificity of negative symptoms and multidimensional definitions of symptoms, encompassing all the heterogeneous symptoms of schizophrenia.^24-31 However, the term disorganization is a definition adopted by only one expert and the validation of this 'new construct' is a difficult challenge in terms of generalization, reliability and applicability.

The methods to explore the clinical reasoning are not standardized and mostly adopt qualitative techniques. The graph method was useful and simple to extract and to objectively represent psychiatric clinical decisions. However, this is a qualitative method that requires a clinical expertise from the interviewer, being very time-consuming and only applicable to small samples. Thus, it is important to emphasize the limitations of this technique regarding the generaliseability of these results.

Conclusions

The reasoning used by the experts for solving intricate problems concerning the diagnosis of schizophrenia was the focus of the investigation and also served as the main source for the construction of a knowledge base. In this study it was possible to compare the agreements and disagreements in diagnostic decision-making for schizophrenia among three experts. The results point to pitfalls in the extraction of the knowledge based on the three experts, in order to construct the final model and to develop an expert system for the diagnosis of schizophrenia. It may well be worthwhile to develop more refined and standardized techniques on knowledge acquisition that would facilitate the knowledge representation process. It should be noted that there is more than one triggering-symptom for the diagnosis of schizophrenia. Thus, multiple combinations of symptoms may be recognized by the expert as being specific and this is probably also related to their knowledge and context. It is important to investigate whether such combinations of symptoms are or not part of the same construct.

Acknowledgments

This study has been funded by a grant from the Fundação de Amparo a Pesquisa do Estado de São Paulo (FAPESP).

Dr. Denise Razzouk developed this study as part of her doctorate training at the Department of Psychiatry of the Federal University of Sao Paulo. The study was supported by a grant from the Brazilian Ministry of Education (CAPES). Prof. Jair Mari is an I-A Researcher within the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

Submitted: 3 December 2004

Accepted: 6 August 2005

Financing: Fundação de Amparo à Pesquisa do Estado de >São Paulo - FAPESP (no. 98/11120-5)

Conflict of interests: None

• 1. Kassirer JP. Diagnostic reasoning. Ann Intern Med. 1989;110(11):893-900.
• 2. Szolovits P, Ramesh PS, Schwartz WB. Artificial intelligence in medical diagnosis. Ann Intern Med. 1988;108(1):80-7.
• 3. Patel VL, Kaufman DR. Medical informatics and the science of cognition. J Am Med Inform Assoc. 1998;5(6):493-502.
• 4. Norman GR. The epistemology of clinical reasoning: perspectives from philosophy, psychology, and neuroscience. Acad Med. 2000;75(10 Suppl):S127-35.
• 5. Anderson JR. Development of expertise. In: BG Buchanan, DC Wilkins, editors. Readings in knowledge acquisition and learning: automating the constructions and improvement of expert systems. San Mateo: Morgan Kaufman; 1993. p. 61-77.
• 6. Shortliffe EH. Clinical decision support systems. In: Shortliffe EH, Perrault LE, Wiederhold G, Fagan LM, editors. Medical informatics: computers applications in health care. Boston, Ma: Addison-Wesley Longman; 1990. p. 466-502.
• 7. Johnston ME, Langton KB, Haynes B, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med 1994;120(2):135-42.
• 8. Shea S, DuMouchel W. A meta-analysis of 16 randomised controlled trials to evaluate computer-based clinical reminder systems for preventive care in ambulatory setting. J Am Med Inform Assoc. 1996;3(6):399-409.
• 9. Buchanan BG, Smith RG. Fundamentals of expert systems. In: BG Buchanan, DC Wilkins, editors. Readings in knowledge acquisition and learning: automating the constructions and improvement of expert systems. San Mateo: Morgan Kaufman; 1993. p. 127-48.
• 10. Edlund MJ. Causal models in psychiatric research. Br J Psychiatry. 1986;148:713-7.
• 11. Kendler KS. Toward a scientific psychiatric nosology. Strengths and limitations. Arch Gen Psychiatry 1990;47(10):969-73.
• 12. Berrios GE, Chen EY. Recognizing psychiatric symptoms. Relevance to the diagnostic process. Br J Psychiatry 1993;163:308-14.
• 13. Markova IS, Berrios GE. Mental symptoms: are they similar phenomena? The problem of symptom heterogeneity. Psychopathology 1995;28(3):147-57.
• 14. Shaw MLG, Woodward JB Modeling expert knowledge: In: Buchanan BG, Wilkins DC, editors. Readings in knowledge acquisition and learning: automating the constructions and improvement of expert systems. San Mateo: Morgan Kaufman; 1993. p. 78-91.
• 15. Heathfield HA, Wyatt J. Philosophies for the design and development of clinical decision support systems. Methods Inf Med 1993;32(1):1-8; discussion 9-17.
• 16. Servan-Schreiber D. Artificial Intelligence and psychiatry. J Nerv Ment Dis. 1986;174(4):191-202.
• 17. Ohayon MM. Utilization of expert systems in psychiatry. Can J Psychiatry 1993;38(3):203-11.
• 18. Razzouk D, Shhirakawa I, Mari JJ. Sistemas inteligentes no diagnóstico da esquizofrenia. Rev Bras Psiquiatr. 2000;22(Suppl 1):35-7.
• 19. Leão BF, Rocha AF. Proposed methodology for knowledge acquisition: a study on congenital heart disease diagnosis. Methods Inf Med. 1990;29(1):30-40.
• 20. Razzouk D, Mari JJ, Shirakawa I, Wainer J, Sigulem D. Knowledge acquisition in schizophrenia: clinical reasoning patterns among three experts. Schizophr Res. 2003;63(3):295-6.
• 21. Berner P, Katschnig H, Lenz G. First-rank symptoms and Bleuler's basic symptoms. New results in applying the polydiagnostic approach. Psychopathology. 1986;19(5):244-52.
• 22. Crichton P. First-rank symptoms or rank-and-file symptoms? Br J Psychiatry. 1996;169(5):537-40; discussion 541-50.
• 23. Peralta V, Cuesta MJ. Diagnostic significance of Schneider's first-rank symptoms in schizophrenia. Comparative study between schizophrenic and non-schizophrenic psychotic disorders. Br J Psychiatry 1999;174:243-8.
• 24. Mauri M, Borri C, Giannotti D, Zambotto S, Cassano GB, Akiskal HS. Psychotic symptoms patterns and the diagnosis of schizophrenia. Psychopathology 1992;25(1):5-10
• 25. Buchanan RW, Carpenter WT. Domains of psychopathology: an approach to the reduction of heterogeneity in schizophrenia. J Nerv Ment Dis 1994;182(4):193-204.
• 26. Tsuang MT, Stone WS, Faraone SV. Toward reformulating the diagnosis of schizophrenia. Am J Psychiatry 2000;157(7):1041-50.
• 27. Costello CG. Research on symptoms versus research on syndromes: arguments in favour of allocating more research time to the study of symptoms. Br J Psychiatry 1992;160:304-8.
• 28. Dollfus S, Brazo P. Clinical heterogeneity of schizophrenia. Psychopathology 1997;30(5):275-81.
• 29. Berner P. Conceptualization of schizophrenia: the symptom-oriented approach. Psychopathology 1997;30(5):251-6.
• 30. Peralta V, Cuesta MJ. Clinical models of schizophrenia: a critical approach to competing conceptions. Psychopathology 2000;33(5):252-8.
• 31. Strömgren E. The concept of schizophrenia: the conflict between nosological and symptomatological aspects. J Psych Res. 1992;26(4):237-46.

Publication Dates

• Publication in this collection
  24 Mar 2006
• Date of issue
  Mar 2006

History

• Received
  03 Dec 2004
• Accepted
  06 Aug 2005