Investigation of cognition in schizophrenia: psychometric properties of instruments for assessing working memory updating

Investigação da cognição na esquizofrenia: propriedades psicométricas de instrumentos para avaliação de atualização da memória de trabalho

Abstracts

Objective

This study describes the development of two updating measures of working memory (WM): Letter Updating Test (LUT) and Word Updating Test (WUT).

Methods

In stage 1, items were created and the instruments were assessed by experts and laymen. In stage 2, tests were given to 15 patients with schizophrenia and 15 paired controls. All were able to understand and respond to the instruments. In stage 3, 141 patients with schizophrenia and 119 healthy controls aged 18 to 60 took part; they were assessed on WM, processing speed (PS) and functional outcome.

Results

The results showed adequate rates of internal consistency for both measures developed, for both the total sample and each group separately, as well as evidence of convergent validity, discriminant validity and sensitivity to differentiate performance among the groups. Principal component analysis yielded two components, one for updating tests and other for PS measures, indicating factorial validity. Positive and significant, yet low, correlations were found with functionality measures.

Conclusion

These results provide adequate psychometric parameters for the measures developed, applicable to cognitive research settings in schizophrenia.

Cognition; working memory; updating; schizophrenia; neuropsychology

Objetivo

O estudo descreve o desenvolvimento de duas medidas de atualização da memória de trabalho (MT): Teste de Atualização de Letras (TAL) e Teste de Atualização de Palavras (TAP).

Métodos

Na etapa 1 foram criados itens e os instrumentos foram analisados por experts e leigos. Na etapa 2, os testes foram aplicados em 15 pacientes com esquizofrenia e 15 controles pareados. Todos foram capazes de compreender e responder aos instrumentos. Na etapa 3, participaram 141 pacientes com esquizofrenia e 119 controles saudáveis com idades entre 18 e 60 anos, avaliados em MT, velocidade de processamento (VP) e funcionalidade.

Resultados

Os resultados revelaram bons índices de consistência interna para ambas as medidas desenvolvidas, tanto para a amostra total como para cada grupo separadamente, bem como evidências de validade convergente com medidas de MT, validade discriminante com medidas de VP e sensibilidade para discriminar o desempenho entre os grupos. Análise de componentes principais revelou que os testes de atualização apresentaram altas cargas e um fator separado das medidas de VP. Relações positivas, significativas, porém baixas, foram encontradas com medidas de funcionalidade.

Conclusão

Esses resultados fornecem bons parâmetros psicométricos para as medidas desenvolvidas, aplicáveis em contextos de pesquisa cognitiva da esquizofrenia.

Cognição; memória de trabalho; atualização; esquizofrenia; neuropsicologia

INTRODUCTION

It has been shown through a dense body of evidence that, in schizophrenia, there is an impairment of working memory (WM)¹, as a result of altered neural mechanisms which underlie its function². The extent of this impairment is associated with both the level of adaptive functioning of patients and the age of onset of the symptoms^3-5. Furthermore, individuals sharing genetic components of vulnerability to schizophrenia, yet not affected by it, also show WM impairment⁶.

WM is typically defined as the skill of maintaining and handling information in the mind for a short period of time, playing and essential role in complex cognition. Although the WM term is used in different contexts by different communities of researchers and models of WM may differ radically in their scope and focus⁷, the Baddeley model⁸ exerts major influence over the organization of this skill, contributing to methods of measurement.

According to this model, there are four different major components in WM: 1) the visuospatial sketchpad, which holds visual and spatial representations of objects; 2) the phonological loop, which is a linguistic system enabling processing of linguistic representations through articulatory mental rehearsing; 3) the central executive, which guides manipulation and transformation of information sustained in reservoirs, protection from interference due to competing information, temporal coding, and updating of the contents of WM; and 4) the episodic buffer, which creates representations based on the integration of different information involved, allowing for the register of an operation as a coherent "episode". Each one of these subsystems has subprocesses that interact with each other.

Due to the centrality of WM in schizophrenia, this construct is thought to be crucial in understanding the pathophysiology of this disorder². However, as suggested by Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS), there exists a variety of mechanisms involved in WM, such as goal maintenance, interference control, maintenance over time, updating, strategic encoding, long-term memory reactivation, and capacity of information that can be maintained in WM¹. Some of these mechanism present an abundant degree of evidence for immediate translation (including neural and psychological construct validity, evidence for impairment in schizophrenia and link to functional outcomes) and use in clinical trials testing new agents to improve cognitive functions in schizophrenia, while others, such as updating, remain relatively lacking in evidence¹. This study is focused, then, on the updating process, which is one of the functions of the central executive and is considered to have little consensus for immediate use for translation.

In everyday functioning, when an individual is challenged to cope with novel or complex situations, or when there are several competing possible responses, he/she may need to implement or create a new strategy to achieve success⁹. Some authors have proposed the existence of a dynamic gating mechanism responsible for updating or implementing behavior when already learned schemas are insufficient to cope with the situation. Thus, a deliberate or conscious control may alter the ongoing cognitive process and construct a new schema. Once said implementation or schema is completed, then the gate mechanism is closed and other executive mechanisms responsible for maintaining the updated or implemented representations, begin to play a relevant role as a bias signal that alters the information flow, preventing possible competition processes to take place in other parts of the system¹⁰. Therefore, it is thought that WM process requires rapid updating and robust maintenance as achieved by a selective gating mechanism^11-13.

It is thought that updating consists of actively monitoring and codifying information coming in, reviewing the items that are being sustained, contrasting this information with new information and, finally, replacing the information that has become irrelevant. Ecker et al.¹⁴ suggested that a typical updating task can be decomposed into 3 major component processes: retrieval, transformation, and substitution. Through structural equation models, these authors tested the link between updating task performance and WM capacity measures. Although the WM capacity was a strong predictor of updating skills, some component processes, especially, substitution skills seemed to be independent of WM capacity. Thus, substitution skills were assumed to make an independent mechanism of updating and essential to be incorporated in updating paradigms.

It has long been established that damage to the basal ganglia can produce similar cognitive impairments as damage to the frontal cortex^15-17. Such similarities are thought to be accounted by the close interaction between the basal ganglia and the frontal cortex¹⁸. Thus, different computational models are proposed in order to explain such gate mechanism along with neurotransmitter systems. For example, it is suggested that dopaminergic signals from the basal ganglia serve as "gating" signals that indicate when to update the contents of WM^13,15. Braver and Cohen¹¹ have suggested that updating information in WM is facilitated by dopamine, and that phasic dopamine signals help to gate or regulate what information must be loaded into WM. Considering that WM deficits in schizophrenia may be caused by catecholaminergic deficits, all these models can be applied in the study of updating in schizophrenia.

One study showed that patients with schizophrenia have impaired performance in updating tasks. Galletly et al.¹⁹ used event-related potentials (ERPs) to distinguish between patients' ability to update WM from their ability to detect and respond to target stimuli. Patients responded to a so-called two-in-a-row task (TIAR task). In this task, participants were asked to issue a response every time they identified two identical stimuli shown in sequence. To do this, it is necessary to continually codify and monitor each stimulus and contrast it with the previously shown stimulus. This demands significant and continual executive control. Patients with schizophrenia show less evoked potential when detecting non-target stimulus, suggesting less capacity over time to actively monitor the input of stimuli in the system. They are also less accurate in the responses they should update. However, as suggested by the CNTRICS¹, more research is needed to justify the use of updating in clinical trials to improve cognition in schizophrenia¹.

In this way, in this present study aimed at investigating cognitive deficits in schizophrenia and its relation with other characteristics of the illness, it is fundamental that tests be available with the appropriate psychometric properties. Therefore, the goal of this study was to develop and adapt two measures for updating WM based on two paradigms that are widely used internationally^20-24. They are: Letter Memory Task (LMT)²² and Keep Track Task (KTT)²⁴. More specifically, an effort was made to develop items for the tasks and investigate their internal consistency and evidence of validity based on relationships with other variables.

METHODS

The study was developed based on three stages, described below:

Stage 1 – Development and tasks

In the Letter Updating Test (LUT) (based on the Letter Memory task)^21,22 ten lists containing letters are shown to the participant. The subject is asked to pay attention to a computer screen where the letters are displayed. Each letter is shown separately and remains displayed on the computer screen for two seconds. With each new item, the participant has to say the last three letters in the list shown. In other words, to ensure the updating process, the individual should continually say each letter shown on the screen, adding the subsequent letters to it. Therefore, when the fourth letter on the list appears, the oldest letter (the first letter, in this case) should be eliminated, forming a new sequence. If the letters in the list are "A, E, T, H, G, U, V, X", the participant has to say A.-.. AE-... AET-... ETH-... THG-... HGU-... GUV-... UVX... and then once again say the last three letters at the end, ed. Each letter is shown separately and remains, two 7 letter lists, and two 9 letter lists. In addition to these, four more lists were created (two 5 letter and two 7 letter lists), demanding a greater working memory load, where the participant should update the last four letters of the lists. Three practice tests are done before the test begins. If the participant gets the three letters in the sequence and their order correct, two points are awarded. If the letters are correct, but the order is wrong, the participant receives one point. No points are given if the letters in the sequence are wrong.

For the Word Updating Test (WUT) (based on the Keep Track Task)^20,23, six different target categories are shown to participants: "Animals", "Colors", "Countries", "Distances", "Metals" and "Family Members". Fifteen words, including two or three elements in each target category, are randomly organized and orally presented by the person applying the test, waiting approximately one second between each one, while four or five of these target categories remain displayed the computer screen. The participant is then asked to remember and say the last words in each target category shown. For example, if the categories are "Distances", "Family Members", "Colours" and "Animals", at the end of the test the participant should remember the last word in the "Distance" category, the last word in the "Family Members" category, in "Colour" and in "Animal". The participant, therefore, has to actively monitor each word shown and update a category when there is more than one word from the same category on the list. Before starting the task, six target categories are shown along with all of the words that make up these categories so that participants can familiarize themselves with them. Three trial runs are conducted.

The categories chosen were based on, and adapted to the version presented by Miyake et al.²¹, with most of the words also being chosen according to this study. For Animals, "Lion", "Cat", and "Cow" were chosen; for Colours, "Lilac", "Green" and "Blue"; for Countries "Chile", "Japan" and "Egypt", all on different continents to avoid semantic confusion within this category; for Distances, "Near" and "Far"; for Metals, "Iron" and "Gold"; and for Family Members, "Father", "Son" and "Grandfather". Participants did three test runs using four target categories and three test runs with five categories. The total number of words correctly recalled is considered a dependent variable.

For both instruments, both the instructions and the items were assessed independently by four clinical neuropsychologists with experience in psychometrics, in order to assess the content and form of the items. They were also assessed by three layman in the area, aimed at verifying the clarity of the instructions and comprehension of the task. The advisors were asked to judge whether the item represented the construct of WM updating, the item's level of difficulty and if the instructions were clear. The advisors were also asked to highlight whether the item was necessary or unnecessary and make suggestions that were deemed pertinent. For the LUT, the variation of items differs only in the letter used and the level of difficulty (as where three letters are updated and another where four letters are updated). For the WUT, the variation of the items is only related to the type of category that will be shown and the level of difficulty (as when one part where the words are updated in four categories and another part with five categories). A minimum of 90% agreement between the advisors was achieved [formula used: (number of people in agreement with the item/total number of participants)*100^25,26. Accordingly, no item needed to be reformulated.

Stage 2 – Pilot study

After creating the items and instructions, the two tasks were applied to 15 patients with schizophrenia (60% male), chosen for convenience, from an outpatient unit at the Federal University of Sao Paulo (Unifesp). Subjects were required to have Portuguese as their primary language. To be eligible for participation, participants should have had more than three years of formal education. The lower level of formal education was chosen to test the level of difficulty and comprehension of the items and instructions. Thus, we collected information from the subject on the highest grade of school they had completed. Average age of the patient group was 32 (SD = 10.33) and average education level was 3.89 (SD = 1.09). This group was paired with 15 healthy age and education-matched controls, chosen for convenience, the age base was (M = 31.60; SD = 10.14), sex (60% male) and education level (M = 4.00; SD = 1.15). All of the participants were able to understand the instructions and respond to all of the items. The Mann-Whitney test for comparison between groups showed that the patients with schizophrenia performed worse on the LUT (U = 18.00; Z = 2.42; P < 0.01) and on the WUT (U = 17.00; z = 2.52; p < 0.01)

Stage 3 – Study of psychometric properties

One hundred and forty one outpatients with schizophrenia, ages 18 to 60, were invited to take part in the study. To confirm the schizophrenia diagnosis, all participants were given a Semi-Structured Interview from the Diagnostic and Statistical Manual of Mental Disorders, 4^th Ed. (SCID-IV)²⁷. The patients were also evaluated with the Positive and Negative Syndrome Scale (PANSS)²⁸, Calgary Depression Scale for Schizophrenia (CDSS)²⁹, Clinical Global Impression, (CGI)³⁰, and Global Assessment of Functioning (GAF)³¹. The patients were being treated with atypical antipsychotics, and medication dosages were stable for at least 4 weeks prior to cognitive assessment. A sample, selected based on convenience, of 119 healthy controls, paired with the patient group according to age, gender and academic level, was also assessed. All were given the SCID-IV and did not meet the criteria for psychiatric disorders. Participants in both groups had to have finished at least Primary School and may not have abused alcohol or other drugs for at least two months. Moreover, they had to be at least 18 years old, their mother tongue had to be Portuguese and they had to have an IQ ≥ moreover, they had to be at least 18 years old, their mother tongue had to be Portuguese³².

Instruments

Positive and Negative Syndrome Scale (PANSS)²⁸: the PANSS was applied to the group for assessment of psychopathology. This measure includes 30 items: seven positive symptoms, seven negative symptoms and sixteen general psychopathology items. Symptoms are measured on a Likert scale ranging from 1 to 7, with a 1 being no symptoms and a 7 being a severe presence of symptoms.

Visual Working Memory Test (VWM):^33,34 this is a computerized task where the participants see one to four 3 x 3 matrices, with a stimulus in each matrix. Next, the participant sees spatial manipulations represented by arrows, indicating the direction that the stimulus should be moved. For instance, an arrow pointing left followed by an arrow pointing up indicates that the participant should manipulate the stimulus in the matrix by placing it one column to the left and one line above its initial position. The participant is tasked with selecting the final position of the stimulus after executing the manipulations indicated. The individual must update the location where the stimulus is moved to with each new arrow that appears. Altogether there are 26 items with a growing degree of difficulty. There is no time limit to provide the response, with application being automatically interrupted by the system after five consecutive errors.

Digit Span BacTest from the Wechsler Intelligence Scales^35,36: this study only used the score on the indirect part of the test. For Digit Span Backward, the examinee is read a sequence of numbers and recalls the numbers in reverse order.

Trail Making Test (TMT) (Public Domain): although considered a measure of attention allocation and flexibility, the TMT has important WM loads. This test requires an individual to draw lines sequentially connecting 24 encircled stimuli distributed on a sheet of paper. In Part A, the circles are numbered 1 – 25, and the patient should draw lines to connect the numbers in ascending order. In Part B, the circles include both numbers (1 – 13) and letters (A – L); as in Part A, the patient draws lines to connect the circles in an ascending pattern, but with the added task of alternating between the numbers and letters (i.e., 1-A-2-B-3-C, etc.). The patient should be instructed to connect the circles as quickly as possible, without lifting the pen or pencil from the paper. Examiner must time the participant as he or she connects the "trail". If the subject makes an error, point it out immediately and allow the patient to correct it. Errors affect the subject's score only in that the correction of errors is included in the completion time for the task. The time to complete TMT-A and TMT-B were measured, and the difference in score between TMT-A and TMT-B (B-A) was the depend variable used in this study.

Processing Speed Assessment Tasks: two measures were used to measure processing speed (PS). The first consists of the color naming part (congruent) of the version of the Computerized Victoria Stroop described in Berberian et al.³⁴, wherein the goal is to name the colours (yellow, blue, green and red) of 24 circles that appear on the computer screen for 40 milliseconds). The second PS measure was the Number Identification Task, consisting of showing participants 30 pairs made up of one letter and one number (for example: 8H). Participants were instructed to indicate if the number is even or odd and ignore the letter. Responses for both PS tests were recorded with a microphone and the average reaction time per item correctly answered was the dependent variable.

Two measures were used to assess adaptive functioning: the Global Assessment of Functioning (GAF)³¹ and the Clinical Global Impression Scale (CGI)³⁰. The first scale scores the level of functionality from 0-100. The second is used to assess the overall severity of the patient, considering frequency and intensity of symptoms. It scores from 1 (normal, not ill) to 7 (extremely ill).

Procedure

All study procedures were approved by the Ethics Committee of the institution where it was conducted. After signing the consent form, subjects were assessed individually in an appropriate room. Assessment sessions began after an Informed Consent Agreement was signed. At the first session, patients were evaluated by psychiatrists trained in application of the SCID and PANSS (with Kappa intra-rater coefficient ranging from 0.63 to 1.0) to determine study eligibility. At the same time, the R1 test was applied by psychologists. The project's total cognitive assessment was done in one session lasting approximately 90 minutes. Participants were given a snack and were free to leave the assessment location whenever they wanted. Cognitive assessments were individual, took place in an appropriate room and were done by psychologist trained in assessment.

Data analysis

Data distribution was verified using skewness, kurtosis and the Kolmogorov-Smirnov tests. Logarithmic transformations were performed for measures that did not achieve normality in distribution. A new analysis of distribution was then carried out to certify if the data achieve normally distribution. After this confirmation, all the analyzes could performed. For calculating precision, internal consistency was verified using Cronbach's Alpha and the split-half method with correction using the Spearman-Brown formula. Pearson correlations were used for evidence of validity based on the relation with other variables, between the two tasks and other tasks that engage WM ability (VWM, Digit span backward, and the difference in score between TMT-A and TMT-B). Correlations between updating and PS measures were implemented to verify evidence of discriminant validity, since studies of cognition in schizophrenia suggest seven different cognitive dimensions, with PS being a separate construct of WM. Updating measures were correlated with the Digit Span Test – Backward Order, the TMT-Part B and the Visual Working memory Test for evidence of convergent validity. Analysis of the Pearson correlation was also used between the two updating measures and the GAF. Principal Components Analysis (PCA) of the variable LUT, WUT, VWM, Backward Digit Span, Color naming (stroop), and Number Identification, was also used to find evidence of validity based on the internal structure of the measures, based on the aforementioned studies of cognition in schizophrenia. Factors with eigenvalues of greater than 1 were extracted and factorial loadings of over 0.40 were considered as relevant.

Diagnosis of schizophrenia as an external criteria was used for criteria validity, with the analysis of covariance (ANCOVA) being used, considering non-verbal intelligence as a covariant for comparing the average performance between groups. The intelligence variable as a covariant was used due to the high correlation that exists between WM and fluid intelligence and to the IQ difference between the groups, according to the data shown in table 1.

Table 1
Demographic data of the participants

Domain	Variable	Schizophrenia patients (N = 141)		Healthy control (N = 119)		Test value	P-values
		Mean	SD	Mean	SD
Age (years)		36.14	9.87	34.03	10.43	F = 2.00	0.11
Education (years)		10.65	3.21	11.23	2.74	F = 2.20	0.13
Mother's Educ.		7.10	5.65	6.13	4.18	F = 2.20	0.19
Non-verbal intelligence		27.43	23.26	38.91	23.18	F = 17.82	< 0.001
Duration of illness		6.08	5.05
Age of onset		22.90	7.07
PANSS	Positive symptoms	13.16	4.71
	Negative symptoms	17.58	5.92
	General	30.13	7.30
	Total score	60.12	15.88
GAF		49.86	13.17
CGI		3.85	1.08

RESULTS

The demographic data for both groups is shown in table 1 along with clinical information. Comparison of demographic variables between groups showed a difference among non-verbal intelligence. The exclusion criterion was intelligence under 80, with no participant having this IQ score on the R1 Test.

Considering the total sample, LUT showed a Cronbach's Alpha of 0.83 and a Spearman-Brown coefficient of 0.93. For the WUT, the Alpha was 0.73 and the Spearman-Brown was 0.84. When analyzed separately, by group, the LUT had an alpha of 0.83 for the patient group and 0.77 for the control group. The Spearman-Brown coefficient was 0.82 for patients and 0.78 for the control group. For the WUT, the Alpha was 0.72 for the schizophrenia sample and 0.68 for the control group. The Spearman-Brown coefficient was 0.95 for patients and 0.76 for the control group.

Table 2 shows the rates of correlation amongst all measures used. According to table 2, significant correlations were found between the updating measures and the other instruments that assessed WM. The results point to positive and significant correlations of a moderate magnitude among the updating measures for the different groups with the VWM and Backward Digit Span, indicating that part of the variance in scores could be attributed to a common component, although each task demands specific components. Regarding the Trail Making Test (time difference B-A), significant correlations of a moderate magnitude were found, however they were negative, indicating that better performance on updating measures is associated to a lesser amount of time executing the Trail Making Test. Both updating tests also showed positive and significant correlations of a moderate magnitude among each other. The LUT and WUT updating tests showed negative and significant correlations of low magnitude with the PS measures. In relation to measures of adaptive functioning, positive and significant correlations, but of a low magnitude, were found with the updating measures.

Table 2
Rates of correlation among the cognitive measures used, considering the schizophrenia (N = 141) and control (N = 119) groups and total sample (N = 260)

Measures	Groups	Relation with WM tests				Relation with PS test		Relation with adaptive functioning
		VWM	Backward Digit Span	Trail B	WUT	Congruent Stroop	Number identification	GAF	CGI
LUT	Schizophrenia	0.59**	0.61*	-0.59*	0.52**	-0.19*	-0.37**	0.26**	0.24*
	Controls	0.59**	0.46*	-0.45*	0.61**	-0.27*	-0.26*
	Total	0.62**	0.56**	-0.45*	0.61**	-0.37**	-0.31**
WUT	Schizophrenia	0.63**	0.44*	-0.42*		-0.26*	-0.29**	0.25**	0.29*
	Controls	0.63**	0.40*	-0.47*		-0.22*	-0.19*
	Total	0.72*	0.45**	-0.32*		-0.30*	0.36*

• * p < 0.05.
• ** p < 0.001.

PCA with orthogonal rotation (varimax) was employed for the groups, separately. One single factor emerged for the schizophrenia group, with 50% of variance explained (eigenvalues of 3.00). However, the two processing speed tests showed a lower factor and negative loading when compared to the rest of the factor. Two factors emerged for the control group, with 67% of variance explained. Factor 1 included working memory tests and the two new updating tests (with 43% variance and eigenvalue of 2.81), while the two processing speed measures formed factor 2 (with 24% variance explained and eigenvalue of 1.05). Table 3shows the factor loadings obtained for each factor based on the group.

Table 3
Analysis of the main components by group

Test Score	Factor 1	Factor 2
Schizophrenia
LUT	0.81
WUT	0.80
VWM	0.81
Backward Digit Span	0.70
Color naming (Stroop)	-0.44
Number Identification	-0.55
Controls
LUT	0.80
WUT	0.88
VWM	0.87
Backward Digit Span	0.59
Color naming (Stroop)		0.74
Number Identification		0.78

To verify evidence of validity based on an external criterion, a comparison was done of performance between the groups. Table 4shows the data obtained based on the analysis. Analysis of covariance showed a significant difference between the groups in all measures, with the schizophrenia group having lower scores on updating tasks and being slower on PS tasks.

Table 4
Comparison of average for all cognitive measures between schizophrenia and control groups, with IQ as covariant

	Schizophrenia group	Control group
	Average (SD)	Average (SD)	df	F	p <	d Cohen
LUT	114.50 (26.43)	130.96 (18.17)	1.25	52.10	0.001	0.71
WUT	13.16 (3.82)	16.81 (3.16)	1.25	18.79	0.001	1.31
Color Naming (Stroop)	0.58 (0.16)	0.47 (0.96)	1.25	29.84	0.001	0.15
Number Identification	58.88 (15.84)	52.16 (15.16)	1.25	5.42	0.02	0.43
Digit Span – Backward	0,60 (0,19)	0,67 (0,17)	1.25	9,26	0.003	0.39
TMT – Part A	1.60 (0.20)	1.48 (0.17)	1.25	24.42	0.001	0.64
TMT – Part B	2.09 (0.28)	1.90 (0.20)	1.25	35.24	0.001	0.78
TMT time difference (B-A)	1.88 (0.32)	1.67 (0.23)	1.25	35.55	0.001	0.75
VWM	5.90 (3.96)	8.95 (4.49)	1.25	33.17	0.001	0.72

DISCUSSION

This study reports on the development of two WM updating measures and explores their psychometric properties. Two main aspects are outlined by these results. First, the assessor evaluations of items and instructions as well as application of tests in a pilot sample showed appropriate validity of content and potential for criterion validity using the diagnosis of schizophrenia as an external criterion. Second, the two measures of updating showed excellent psychometric properties, that is, good levels of internal consistency and suitable evidence of validity based on their relationship with other variables.

Insofar as the precision of measures is concerned, the LUT and WUT items showed good internal consistency, based on Cronbach's Alpha in addition to good homogeneity based on the Spearman-Brown split-half method. With this, interpretations regarding results are relatively free of biases determined by particularities of specific items. When considering only the sample of the patients with schizophrenia, on other hand, the WUT items showed a moderate alpha, Overall, the values gained using the Spearman-Brown Coefficient were found to be greater in relation to Cronbach's Alpha Coefficient. In this sense, if a test is more homogenous, greater Spearman-Brown Coefficient values are found in relation to the internal consistency coefficient values. The rates of precision were therefore satisfactory, suggesting good internal consistency among all of the items in the two tests.

Four different measures that engaged WM showed a moderate to high convergence level with the LUT. On the other hand, the WUT only showed convergence with two measures. One possible explanation for this difference could be found by analyzing the constructs that each task engages as well as by interpreting the factor loadings produced. While the TMT and the Backward Digit Span engaged processes such as sustaining and manipulating items^37,38 the VWM, LUT and WUT seem to engage a greater update load²². In the VWM, whenever a new "arrow" indicates where the stimulus should be moved to, an update should be done. In the WUT, the target categories were shown on the computer screen, freeing up the central executive to carry out updates, with a lesser load of the sustaining processes in WM. On the other hand, although the LUT has a greater updating load, this task also engages sustaining of updates, since in order to execute these updates, from three to four items should be simultaneously sustained. This is probably the reason why the LUT had better rates of convergence with all four tasks, while with the WUT this only occurred with tasks involving a greater update load. Corroborating this interpretation, the factor analysis showed that Factor 1 had greater loads of updating tasks, i.e., LUT, WUT and VWM, while the digit span task showed a lower factor loading.

Many studies show that WM is related to functionality in schizophrenia^4,39; however, although significant, the LUT and WUT show only a low correlation with the functional outcome measures.

The study also provided evidence of discriminant validity for both instruments. Although in the general population there could be some level of covariance between executive functions and processing speed³⁸, these constructs are independent of one another at the factorial level. Studies on cognition of schizophrenia also suggest this dissociation^40,41. This study therefore corroborates the weak relationship between the two updating measures and processing speed, corroborating various findings in the area.

Nonetheless, the ACP showed only one factor for the schizophrenia group, when measures of PS and updating were considered, whereas two factors were shown for the healthy control group. In the latter group, the two processing speed measures constituted a separate facet, in line with discriminant validity data. The two processing speed measures for the schizophrenia group showed negative factor loadings, demonstrating an inverse impact on the factor; in other words, the higher the level of updating, the slower the processing speed and vice versa. Despite this, the PS measures showed low factor loadings.

Dickinson et al.⁴²tested a two-level hierarchical cognitive model. At the first level, the model conceives of six latent dimensions (verbal comprehension, perceptual organization, visual memory, verbal memory, processing speed and executive functions/working memory); at the second level is a second-order general factor that determines the variance of the six first-order factors. Through confirmatory factor analysis and application of gradual restrictions on the model's parameters, the researchers found that the schizophrenia group showed more homogenous cognition than the control group. In our study, although cognitive performance in schizophrenia has been suggested to be fractionated^40,41, our results corroborate prior studies⁴² suggesting that people with schizophrenia have a more unitary cognitive structure. This unitary cognitive performance may be a reflection of the variable cortical and subcortical dysfunction present in schizophrenia⁴³ or the existence of a more widespread range of test performance among patients than in healthy controls⁴². That's why focus on group average performance, or use of samples with individuals who are at different phases of the illness, may mask the existence of different cognitive profiles amongst patient subgroups⁴⁴.

Both measures developed proved to be sensitive to differentiation of performance by individuals with schizophrenia from those without any psychiatric diagnosis. The size of the effect of this differentiation ranged from average (LUT) to significant (WUT). Therefore, using the diagnosis as an external criterion, evidence of validity for both measures was established. These results are in line with the substantial literature in the area, reporting losses in WM and updating in schizophrenia^1,19,34.

The study1s main limitation regards the heterogeneity of the sample of patients with schizophrenia, whose ages varied greatly (18 to 60), in addition to the different ages of onset of the illness and its different duration times. The standards of deficits found could therefore be connected to different factors, with comparison of a simple average being a masking effect for different cognitive profiles⁴⁴. More controlled studies could investigate the impact of these variables (age, onset age and illness duration) on cognition, particularly regarding updating of WM in schizophrenia.

Future studies must also engage in looking for evidence backing the utility of the measures presented here for clinical tests aimed at improving cognition in schizophrenia. In order to do this, analysis of the test-retest precision is fundamental, as is investigation of whether these tests are useful as repeated measures, that is, free of learning effects. Likewise, it is important to investigate the relationships between cognitive deficits and the functionality of patients. This study found only a weak relationship, albeit a significant one, between the LUT and WUT and generic functionality measures. New studies, with more specific functionality measures, such as personal hygiene, use of money, transportation use, and leisure, should be done in an effort to clarify whether these measures do in fact have any usefulness in studies involving functionality⁴⁵.

CONCLUSIONS

The study presented two WM updating measures. The results showed good rates of internal consistency and evidence of validity based on the internal structure and relations to other variables, including standards of convergence and divergence, in addition to relations with external criterion, for both instruments. Considering the reality in Brazil, a country with a scarcity of neuropsychological instruments based on constructs of cognitive psychology, this study partly fills this gap, especially in investigation of aspects of WM in schizophrenia, one of the core loss characteristics of the illness. Future studies investigating the relationship of these measures with different aspects of functionality and their efficacy in clinical testing should be performed.

REFERENCES

• ¹
  Barch DM, Smith E. The cognitive neuroscience of working memory: relevance to CNTRICS and schizophrenia. Biol Psychiatry. 2008;64(1):11-7.
• ²
  Goldman-Rakic PS. Architecture of the prefrontal cortex and the central executive. In: Grafman, J, Holyoak, KJ, Boller, F, editors. Structure and functions of the human prefrontal cortex. New York: The New York Academy of Sciences; 1995. p. 71-83.
• ³
  Cornblatt B, Obuchowski M, Roberts S, Pollack S, Erlenmeyer-Kimling L. Cognitive and behavioral precursors of schizophrenia. Dev Psychopathol. 1999;11(3):487-508.
• ⁴
  Cervellione KL, Burdick KE, Cottone JG, Rhinewine JP, Kumra S. Neurocognitive deficits in adolescents with schizophrenia: longitudinal stability and predictive utility for short-term functional outcome. J Am Acad Child Adolesc Psychiatry. 2007;46(7):867-78.
• ⁵
  Seidman LJ, Giuliano AJ, Smith CW, Stone WS, Glatt SJ, Meyer E, et al. Neuropsychological functioning in adolescents and young adults at genetic risk for schizophrenia and affective psychoses: results from the Harvard and Hillside Adolescent High Risk Studies. Schizophr Bull. 2006;32(3):507-24.
• ⁶
  Thermenos HW, Seidman LJ, Breiter H, Goldstein JM, Goodman JM, Poldrack R, et al. Functional magnetic resonance imaging during auditory verbal working memory in nonpsychotic relatives of persons with schizophrenia: a pilot study. Biol Psychiatry. 2004;55(5):490-500.
• ⁷
  Shah P, Miyake A. Models of working memory: an introduction. In: Miyake A, Shah P, editors. Models of working memory: mechanisms of active maintenance and executive control. Cambridge: Cambridge University Press; 1999.
• ⁸
  Baddeley AD. The episodic buffer: a new component of working memory? Trends Cogn Sci. 2000;11:417-23.
• ⁹
  Burgess PW. Theory and methodology in executive function research. In: Rabbitt P, editor. Methodology of frontal and executive function. Hove, UK: Psychology Press; 1997.
• ¹⁰
  O'Reilly RC. Biologically based computational models of high level cognition. Science. 2006;314:91-4.
• ¹¹
  Braver TS, Cohen JD. On the control of control: the role of dopamine in regulating prefrontal function and working memory. In: Monsell S, Driver J, editors. Control of cognitive processes: attention and performance XVIII. Cambridge, MA: MIT Press; 2000.
• ¹²
  Cohen JD, Braver TS, O'Reilly RC. A computational approach to prefrontal cortex, cognitive control and schizophrenia: recent developments and current challenges. Philos Trans R Soc Lond B Biol Sci. 1996;351(1346):1515-27.
• ¹³
  O'Reilly RC, Braver TS, Cohen JD. A biologically based computational model of working memory. In: Miyake A, Shah P, editors. Models of working memory: mechanisms of active maintenance and executive control. Cambridge: Cambridge University Press; 1999.
• ¹⁴
  Ecker UK, Lewandowsky S, Oberauer K, Chee AE. The components of working memory updating: an experimental decomposition and individual differences. J Exp Psychol Learn Mem Cogn. 2010;36(1):170-89.
• ¹⁵
  Frank MJ, Loughry B, O'Reilly RC. Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cogn Affect Behav Neurosci. 2001;1:137-60.
• ¹⁶
  Brown LL, Schneider JS, Lidsky TI. Sensory and cognitive functions of the basal ganglia. Curr Opin Neurobiol. 1997;7(2):157-63.
• ¹⁷
  Brown RG, Marsden CD. Internal versus external cues and the control of attention in Parkinson's disease. Brain. 1988;111(Pt 2):323-45.
• ¹⁸
  Alexander GE, DeLong MR, Strick PL. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci. 1986;9:357-81.
• ¹⁹
  Galletly CA, McFarlane AC, Clark CR. Impaired updating of working memory in schizophrenia. Int J Psychophysiol. 2007;63(3):265-74.
• ²⁰
  Miyake A, Friedman NP. The nature and organization of individual differences in executive functions: four general conclusions. Curr Dir Psychol Sci. 2012;21(1):8-14.
• ²¹
  Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, Wager TD. The unity and diversity of executive functions and their contributions to complex "Frontal Lobe" tasks: a latent variable analysis. Cogn Psychol. 2000;41(1):49-100.
• ²²
  Morris N, Jones DM. Memory updating in working memory: the role of the central executive. Br J Psychol. 1990;81(2):111-21.
• ²³
  Smith EE, Jonides J. Storage and executive processes in the frontal lobes. Science. 1999;283(5408):1657-61.
• ²⁴
  Yntema DB. Keeping track of several things at once. Hum Factors. 1963;5:7-17.
• ²⁵
  Tilden VP, Nelson CA, May BA. Use of qualitative methods to enhance content validity. Nurs Res. 1990;39(3):172-5.
• ²⁶
  Topf M. Three estimates of interrater reliability for nominal data. Nurs Res. 1986;35(4): 253-5.
• ²⁷
  Del-Ben CM, Vilela JAA, Crippa JAS, Hallak JEC, Labate CM, Zuardi A. Confiabilidade da "Entrevista Clínica Estruturada para o DSM-IV – Versão Clínica" traduzida para o português. Rev Bras Psiquiatr. 2001;23:156-9.
• ²⁸
  Higuchi CH, Ortiz B, Berberian AA, Noto C, Cordeiro Q, Belangero SI, et al. Factor structure of the Positive and Negative Syndrome Scale (PANSS) in Brazil: convergent validation of the Brazilian version. Rev Bras Psiquiatr. 2014;36(4):336-9.
• ²⁹
  Bressan RA, Chaves AC, Shirakawa I, de Mari J. Validity study of the Brazilian version of the Calgary Depression Scale for Schizophrenia. Schizophr Res. 1998;32(1):41-9.
• ³⁰
  Lima MS, Soares GBO, Paoliello G, Vieira RM, Martins CM, Mota JI, et al. Versão em português da Impressão Clínica Global – Escala de Esquizofrenia: estudo de validação. Rev Bras Psiquiatr. 2007;29(3):246-9.
• ³¹
  American Psychiatric Associaton. Avaliação multiaxial. In: Manual diagnóstico e estatístico de transtornos mentais (DSM-IV-TR). 4ª ed. Porto Alegre: Artmed; 2002. p. 59-69.
• ³²
  Oliveira R. Teste não verbal de inteligência. São Paulo: Vetor Editora Psico-Pedagógica; 2002.
• ³³
  Primi R. Bateria informatizada de capacidades cognitivas. Itatiba: LabAPE; 2002.
• ³⁴
  Berberian AA, Trevisan BT, Moriyama TS, Montiel JM, Oliveira JAC, Seabra AG. Avaliação da memória de trabalho na esquizofrenia e sua correlação com habilidades de funções executivas. Rev Bras Psiquiatr. 2009;31(3):219-26.
• ³⁵
  Nascimento E, Flores-Mendoza CE. WISC-III and WAIS-III in intellectual assessment of blind people. Psicol Estud. 2007;12(3):627-33.
• ³⁶
  Wechsler D. WAIS-III: Administration and scoring manual. San Antonio: The Psychological Corporation; 1997.
• ³⁷
  Salthouse TA. Relations between cognitive abilities and measures of executive functioning. Neuropsychology. 2005;19(4):532-45.
• ³⁸
  Salthouse T. What cognitive abilities are involved in trail-making performance? Intelligence. 2011;39(4):222-32.
• ³⁹
  Fett AK, Viechtbauer W, Dominguez MD, Penn DL, van Os J, Krabbendam L. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci Biobehav Rev. 2011;35(3):573-88.
• ⁴⁰
  Gladsjo JA, McAdams LA, Palmer BW, Moore DJ, Jeste DV, Heaton RK. A six-factor model of cognition in schizophrenia and related psychotic disorders: relationships with clinical symptoms and functional capacity. Schizophr Bull. 2004;30(4):739-54.
• ⁴¹
  Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, Green MF, Heaton RK. Identification of separable cognitive factors in schizophrenia. Schizophr Res. 2004;72(1):29-39.
• ⁴²
  Dickinson D, Ragland JD, Calkins ME, Gold JM, Gur RC. A comparison of cognitive structure in schizophrenia patients and healthy controls using confirmatory factor analysis. Schizophr Res. 2006;85(1-3):20-9.
• ⁴³
  Antonova E, Sharma T, Morris R, Kumari V. The relationship between brain structure and neurocognition in schizophrenia: a selective review. Schizoph Res. 2004;70(2-3):117-45.
• ⁴⁴
  Kremen WS, Seidman LJ, Faraone SV, Toomey R, Tsuang MT. Heterogeneity of schizophrenia: a study of individual neuropsychological profiles. Schizophr Res. 2004;71(2-3):307-21.
• ⁴⁵
  Wallace CJ. Functional assessment in rehabilitation. Schizophr Bull. 1986;12(4):604-30.

• DECLARATION OF SOURCES
  The present study had financial support from Fundação de Amparo à Pesquisa do Estado de São Paulo (Fapesp, Grants number 2011/50740-5 and 2007/58630-9, Brazil).

Publication Dates

• Publication in this collection
  Jul-Sep 2015

History

• Received
  9 Mar 2015
• Accepted
  28 Aug 2015