Adaptation of the Montreal Communication Evaluation Battery to European Portuguese

Abstracts

Purpose

To present the adaptation of the Bateria Montreal de Avaliação de Comunicação (Bateria MAC-BR) - to the Portuguese social cultural reality - Bateria MAC portuguese version (MAC-PT Battery).

Methods

The adaptation process involved six steps: 1) comparative analyses ofBateria MAC-BR to its original Canadian version; 2) adaptation and development of new stimuli by specialists; 3) analysis by non-specialist judges, 4) analysis by expert judges; 5) pilot Study 1 (n=10); e 6) pilot Study 2 (n=30) as well as inter-rater agreement.

Results

The completion of steps permited significant changes in the MAC Battery, which allowed a proper adaptation to the socio-cultural and linguistic Portuguese reality.

Conclusion

The MAC-PT battery is a useful clinical tool for the communication evaluation of patients with neurological lesion.

Adult; Communication; Language tests; Stroke; Neuropsychological tests

Objetivo

Realizar a adaptação neuropsicolinguística da Bateria Montreal de Avaliação da Comunicação - versão brasileira (Bateria MAC-BR) - à realidade sociocultural portuguesa – Bateria MAC versão Portugal (Bateria MAC-PT).

Métodos

O processo de adaptação envolveu seis etapas: 1) análise comparativa da Bateria MAC-BR com sua versão original canadense; 2) adaptação e desenvolvimento de novos estímulos por especialistas; 3) análise de juízes não especialistas; 4) análise de juízes especialistas; 5) estudo piloto 1 (n=10); 6) estudo piloto 2 (n=30) e concordância entre avaliadores.

Resultados

O cumprimento das etapas levou a mudanças importantes na Bateria MAC, que permitiram a adaptação adequada para realidade sociocultural e linguística portuguesa.

Conclusão

A bateria MAC-PT é uma ferramenta clínica útil para a avaliação da comunicação de pacientes com lesão neurológica.

Adulto; Comunicação; Testes de linguagem; Acidente vascular cerebral; Testes neuropsicológicos

INTRODUCTION

Cerebrovascular injuries may cause severe to mild sequelae, depending on many factors, such as the brain area affected, the extent of injury, the time that took to the patient be attended and the age and education level of the patient⁽¹⁾. These damages can be grouped into motor, cognitive, communicative and behavioral/emotional deficits, besides swallowing disorders and urinary incontinence⁽²⁾ .

The right hemisphere (RH) is involved in several cognitive functions such as memory⁽³⁾, temporal and spatial orientation⁽⁴⁾, attention⁽⁵⁾ and praxis⁽⁶⁾. RH has important role in language, specifically in functional language aspects ⁽¹⁾.

Approximately 50% of patients with right hemisphere damage (RHD) have some kind of acquired communication disorder⁽¹⁾, which can affect social interactions and thus pose psychosocial and functional difficulties for these people⁽⁴⁾. The alterations in communication include: difficulties in maintaining the topic of the conversation⁽⁷⁾, prosodic processing⁽⁸⁾ and understanding metaphors⁽¹⁾, for example.

Therefore, it is important to evaluate communication competences in addition to language skills, not infrequently ignored when it comes to RHD patients, as they exhibit appropriate language skills (phonology, semantics, syntax, morphology), but alterations, including pragmatic communication (figurative language, metaphors, jokes) and prosody, which are very important in the social context⁽⁸⁾.

As far as is known, in Portugal there is not a complete battery that takes into account the evaluation of the communication processes that may be deficient in RHD patients. Hence, it was decided to adapt the instrument Montreal Communication Evaluation Battery, Brazilian expanded version (MAC-BR)⁽⁹⁾, as it is an instrument already adapted from the original Canadian version (Protocole Montréal d'Évaluation de la Communication – Protocole MEC⁽¹⁰⁾) into the Brazilian Portuguese, and it displays appropriate, valid and reliable results, besides being used in other countries, such as Italy and Argentina.

The purpose of MAC-BR is to evaluate four components of the communication process: discourse, pragmatics, lexical-semantic and prosody. In light of this, the present study aimed at carrying out the neuropsycholinguistic adaptation of the Montreal Communication Evaluation Battery – MAC-BR – into the socio-cultural reality of Portugal.

METHODS

Participants

Six different samples took part in this research (Experts, Specialist Judges, Non-specialist Judges, Pilot 1, Pilot 2 and Pilot 3). The characterization of the sample is shown in Table 1.

Table 1
Description of the groups that took part in the MAC-PT Battery adaptation process

Participants	Step	n	Selection criterion
Specialists	New stimuli adapta-tion and develop-ment	10 (9 speech thera-pists and 1 psycholo-gist)	Mastery of theoretical assumptions of language neuropsychology related to language and communication constructs
Non-specialist Judges	Analysis by non-specialist judges	80 (46 university stu-dents, 7 pre-university students and 27 grad-uated professionals)	Effective representation of the general population, untrained in relation to neu-ropsychology knowledge, with over 8 years of formal study, aged between 18 and 80 years old
Expert Judges	Analysis by expert judges	5 (3 speech thera-pists, 1 neuropsy-chologist and 1 lin-guist)	Mastery of theoretical assumptions of language neuropsychology
Pilot 1	Pilot study 1	10 neurologically healthy subjects	Do not present neurological, visual or auditory impairments
Pilot 2	Pilot study 2	30 neurologically healthy subjects	Idem previous
Pilot 3	Pilot study 3	90 neurologically healthy subjects	Idem previous

• Note: MAC-PT = Montreal Communication Evaluation Battery – Portuguese version

Adaptation of the instrument

The adaptation process of MAC-BR Battery into the MAC Portuguese Battery (MAC-PT) version occurred in six steps.

All steps were mediated and analyzed by the authors of the MAC-BR Battery⁽⁹⁾ and the original instrumentProtocole MEC⁽¹⁰⁾. Each step will be explained below.

- Step 1: Comparative analysis of the batteries MAC-BR andProtocolé MEC

Through cooperation between the authors of the original version of theProtocole MEC⁽¹⁰⁾, the authors of the Brazilian version⁽⁹⁾ and the authors of the present study, the aim was to ensure that the MAC-PT Battery did not diverge from the objectives of each task of the instrument, determined by the authors of the original version. Therefore, by means of a videoconference, the first two groups of authors (Canadian and Brazilian) compared their instruments and exposed to the third group their tasks and goals, explaining and adjusting criteria of adaptation, such as: maintenance of the number of tasks and stimuli and equivalence psycholinguistic criteria, which led to the first version of MAC-PT Battery.

- Step 2: Adaptation and development of new stimuli by experts

This step was performed in two separate procedures. In the first, two Portuguese language experts changed the Brazilian version instrument and adapted it to the European Portuguese (EP), through lexical-syntactic adjustments, confirmed by a third specialist. The second procedure was performed from a brainstorming⁽¹¹⁾ consisting of seven experts in language who were responsible for carrying out the appropriate semantic-pragmatic adaptations to the Portuguese socio-cultural characteristics to then generate the second version of the MAC-PT Battery. The criteria and objectives of each task were maintained during this step.

- Step 3: Analysis by non-specialist judges

Non-specialist judges individually analyzed psycholinguistic stimuli criteria, from the linguistic knowledge by the general population. Tasks were applied regarding the items that make up the metaphor interpretation, narrative discourse, conversational speech, speech acts and semantic judgment tests, which were then analyzed by the authors of this study. Features like familiarity of metaphorical phrases, plausibility of phrases, word frequency and degree of directivity (how direct or indirect were the speech acts) were assessed by the judges.

The analysis was carried out from an analog scale ranging of zero to ten. In the conversational speech task, judges should point out how easy it would be to talk about certain topics for five minutes, and the closer to zero was the score, the harder it would be to talk about the subject in question. For the metaphor interpretation task, the degree of familiarity of the words that made up the new metaphors was judged, and the closer to zero was the score, the less familiar were those words. The same procedure was applied in the semantic judgment task and, in addition to that, judges were asked to explain the semantic relationship between pairs of words. Regarding the tasks geared towards metaphors, individuals (judges) should explain the meaning of ten new metaphors and ten idioms and then analyze the degree of familiarity of idioms and new metaphors. In relation to the speech acts task, the analysis of the degree of directivity of the stimuli of speech acts, and the closer to zero was the score, the more indirect was the phrase, and the closer to ten, the more direct was the phrase. Examples were given for each analysis task, with non-battery stimuli. This step resulted in the third version of the MAC-PT.

- Step 4: Analysis by expert judges

Some instruction-stimulus sets applied in the previous step were selected to be analyzed by expert judges in two separate procedures. First, the judges evaluated the cognitive/neuropsychological function or component that was predominantly examined. The goal was to see if these stimuli assessed what it was intended to assess. In the second procedure, the expert judges examined whether each stimulus was appropriate or not to their respective instruction, and suggested modifications, when necessary. This step led to the changes that resulted in the fourth version of the MAC-PT Battery under adaptation.

- Step 5: Pilot Study 1

In order to verify the applicability of some adapted stimuli, ten neurologically healthy subjects were evaluated in the tasks of metaphor interpretation, emotional prosody (comprehension), linguistic prosody (comprehension) and comprehension of speech acts. These tasks were applied and analyzed in accordance with the third version of MAC-PT Battery and the application and corresponding scoring manual, still undergoing adaptation. The aim of this pilot study 1, focused only on some tasks, was to ascertain the comprehension of instructions and stimuli.

- Step 6: Pilot study 2 and interrater agreement with evidence of content validity

In order to complete the adaptation process, the fourth version of the MAC-PT Battery was applied to a second pilot sample of neurologically healthy subjects.

The study included individuals who complied with the following inclusion criteria: European Portuguese as first language; Portuguese nationality; no current or previous history of neurological or psychiatric diseases; no previous history of alcoholism or drug addiction and no sensory impairments (uncorrected auditory and/or visual impairments). These criteria were confirmed through the application of a structured questionnaire with socio-cultural data and health aspects⁽¹²⁾.

In addition, to confirm the absence of suggestive signs of dementia in the participants, subjects had to present scores that should compatible with the normal Portuguese population in the following tests:

1. Mini Mental State Examination – MMSE⁽¹³⁾. Participants who scored higher than 22 for schooling levels from four to eleven years, and scored higher than 27 for schooling levels over 11 years were included.

2. The Clock Drawing Test⁽¹⁴⁾ with a minimum score of 7/8, according to the Sunderland Scale⁽¹⁵⁾, to detect suggestive signs of cognitive impairment, supplementing the analysis performed by the MMSE.

Given these criteria, the sample included 30 subjects (n=30), members of the general population divided into 9 sub-groups, 10 individuals in each age group (18-40, 41-64 and 65-80 years old), where n=11 with 4-9 years of formal study, n=10 with 10-13 years of formal study and n=9 with 14 or more years of formal study.

All participants in this study signed an Informed Consent Form, authorizing their participation in this research. The participation of individuals was voluntary, therefore not paid.

The application of the MAC-PT was followed by the concordance analysis of three evaluators regarding standardized scores and interpretation. The registration protocols of these assessments were subject to an analysis stage of the standardization agreement by two expert judges with a double-blind trial, and consensus of a third expert in neuropsychology of language. The evaluators received guidance on how to rate and judge the responses of the sample on an item by item basis, and they had at hand the application and scoring guide under development of the MAC-PT Battery.

- Step 7: Final adjustments, Pilot study 3 and development of the final version of the MAC-PT Battery

Throughout the adaptation process, the authors of the present study proceeded to adjust the stimuli book and the application and scoring guide of the MAC-PT Battery. The responses generated by the subjects that constituted the pilot study sample for the adaptation were used to finalize the two sets of materials.

The sample investigated at this stage was composed of 30 subjects participating in the Pilot study 2, in addition to 71 individuals who met the same inclusion and exclusion criteria previously mentioned. Given these criteria, the initial sample comprised 101 individuals, resulting in 90 subjects after the following exclusions: 3 subjects (n=3) did not want to continue the assessment alleging tiredness; 4 subjects (n=4) had depressive symptoms; 2 subjects (n=2) displayed scores below the minimum score on the Mini Mental State Examination and 2 subjects (n=2) presented a previous history of neurological diseases.

The application of the final version of the MAC-PT Battery was performed in 90 subjects (n=90), aged between 18 and 80 years old (A=50.4, SD=19.8) and education from four to 19 years (A=11.27, SD=4.313). The sample of the pilot studies was selected by the non-random convenience sampling technique, and the selection was carried out in universities, schools, companies, hospitals, day care centers, among others.

After applying the MAC-PT Battery, participants were divided into nine subgroups, according to age (18-40 years old; 41-65 years old; 65-80 years old) and level of schooling (4-9 years of formal study; 9-13 years and more than 14 years of formal education). In relation to gender distribution, the sample in question was composed of 26.7% of male subjects and 76.3% female subjects (Table 2).

Table 2
Demographic characteristics of the sample by age and education level

Groups	18 - 40 years old			41 - 64 years old			65 - 80 years old
Education (years)	4-9	10-13	+ 13	4-9	10-13	+ 13	4-9	10-13	+ 13
n	10	10	10	10	10	10	10	10	10
Gender (M/F)	8/2	3/7	5/5	3/7	1/9	0/10	0/10	8/2	8/2
Average age	36.1	18.3	25,4	50.5	54.2	52.1	72.7	74.1	70.8
SD	3.93	0.48	4,14	4.81	3.61	6.01	5.46	3.57	4.21
Average years of education	7.20	12.1	16,5	6.4	12.1	15.9	4.5	11.4	15.3
SD	1.32	0.57	0,71	1.96	0.57	1.1	1.58	1.17	1.64

• Note: M = male; F = female; SD = standard deviation

Procedures and data analysis

Regarding the MAC-PT Battery application, expert and non-specialist judges were orally instructed on how to fill out the surveys graphically presented and asked to complete the survey form without the interference of the authors.

Two examiners, speech therapists and specialists were trained to proceed with the application of the tasks of the MAC-PT Battery to the 90 subjects evaluated in the Pilot study 3, in reliance on the application and scoring guide of the MAC-PT, 27% (30) of these evaluations were carried out by a blind expert judge. In this way, each of the 30 selected evaluations were scored twice and these scores were compared. A third expert judge analyzed the assessments that reached a coefficient of concordance lower than 75%⁽³⁾.

All pilot studies participants were individually evaluated in a private room, a quiet, lit and ventilated environment. The tasks were presented to each participant in the sequence provided on the original tests. The evaluation was performed in a single session, lasting approximately one hour and 30 minutes.

For data transcription and analysis, responses were registered by the evaluator and recorded on a digital audio recording equipment.

For the ranking of psycholinguistic criteria by non-specialist judges, averages and standard deviations of judgment values per item and per task were determined. The analysis of responses between expert judges was verified through descriptive analysis of the percentages of correct answers and simple index of concordance between judges, according Andres and Marzo⁽¹⁶⁾. For the pilot studies, the answers were scored in accord with the Scoring and Interpretation Guide of the MAC-PT Battery.

RESULTS

Comparing the Canadian and Brazilian versions, it was found that the Brazilian version maintained the criteria of the Canadian version. This was followed by adaptation procedures from the Brazilian instrument, resulting in the first Portuguese versions of the application guide and the scoring guide of the MAC-PT Battery.

As a consequence of the regular contact with the authors, some suggestions were given for the improvement and small adjustments in the instructions and details for new stimuli construction. One can highlight the following: substitution of the personal pronouns "my", "mine" present in the MAC-BR metaphor interpretation task for nouns that do not directly refer to the personal context of the subject evaluated ("the baker", "the neighbor", "this man", "the woman"); substitution of the idiom "o meu pai me deu uma mãozinha" ("my father gave me a hand") in the MAC-BR for "O vizinho bateu a bota" ("The neighbor kicked the bucket"); change the task name "lexical evocation" for "fluency"; modify the task name "indirect speech acts interpretation" for "speech acts interpretation"; do not use the expression "dear" in the speech acts task; substitute pairs of words "chuva/neve" ("rain/snow") and "cigarro/cachaça" ("cigarette/cachaça") of the semantic judgment task for the pairs "comboio/avião" ("train/plane") and "lápis/borracha" ("pencil/eraser").

The first version of stimuli book and the second version of the application and scoring guide of the MAC-PT Battery resulted from the step of adaptation and development of new stimuli by experts (Table 3).

Table 3
Initial and final stimuli of the MAC-PT Battery in comparison to the Brazilian version during the Step 2

Task	MAC-BR Stimulus	1º Procedure	2º Procedure /Results
Metaphor interpretation	O professor é um sonífero (The teacher is a sleeping pill)	O professor é um soporífero (The teacher is a soporific)	O professor é uma seca (The teacher is a hassle)
	Meu pai é um pavão (My father is a peacock)	O meu pai é um pavão (My father is a peacock)	Este rapaz é um troca-tintas This guy is a fumbler (substitution)
	Esta criança é uma pipoca (This child is a popcorn)	Esta criança é uma pipoca (This child is a popcorn)	Esta criança é um veludo (This child is a velvet) (substitution)
	Este homem joga dinheiro no lixo (This man throws money in the garbage)	Este homem atira dinheiro ao lixo (This man pitches money in the garbage)	*
	A mãe pisa em ovos com seus filhos (The mother walks on eggs with her children)	Idem	A mãe anda em pezinhos de lã com os filhos (The mother tiptoes with her children)
Narrative discourse	Marcos é um agricultor gaúcho (Marcos is a farmer from Rio Grande do Sul)	O Marcos é um agricultor alentejano (Marcos is a farmer from Alentejo)	O António é um agricultor alentejano (António is a farmer from Alentejo)
	na sua fazenda… (in his farm…)	na sua quinta… (in his homestead…)	*
Speech acts interpretation	Você tem algum programa para o fim de semana? (Do you have any plan for the weekend?)	Tens algum programa para o fim de semana? (You have a plan for the weekend?)	Tens alguma coisa para fazer no fim-de-semana? (Do you have anything to do this weekend?)
	Esta sacola está muito pesada (This bag is too heavy)	Este saco está muito pesado (This sack is too heavy)	*
	O apartamento é bem claro (The apartment is very bright)	O apartamento é bem claro (The apartment is very bright)	O apartamento é muito luminoso (The apartment is well illuminated)
	Vou cozinhar massa hoje a noite (I will cook the pasta tonight)	Hoje a noite vou fazer massa (Tonight I will prepare the pasta)	Hoje a noite vou fazer bacalhau no forno (Tonight I will prepare baked cod)
	Eu adoro a cor que a gente escolheu para o carro (I love the color chosen for the car)	Eu adoro a cor que escolhemos para o carro (I love the color we chose for the car)	*
	Cristian, que demora (Cristian, what takes you so long?)	Cristian, que demora (Cristian, what takes you so long?)	Estás a lavar os dentes há 20 minutos (You have been washing your teeth for 20 minutes)
Semantic Judgement	Bomba – Fuzil (Bomb – Rifle)	Bomba - Espingarda (Bomb – Shotgun)	Granada – Espingarda (Grenade – Shotgun)
	Pia – Tanque (Basin – Sink)	Lavatório – Tanque (Washbasin – Sink)	Lavatório – Sanita (Lavatory – Loo)

• * = the stimulus used in the procedure was the same used in the previous one Note: MAC-PT = Montreal Communication Evaluation Battery – Portuguese version; MAC-BR = Montreal Communication Evaluation Battery - Brazilian version

Descriptive data analysis of stimuli by non-specialist judges can be verified inTable 4. Averages and standard deviations of the judgments performed are presented for all psycholinguistic criteria ranking tasks.

Table 4
Results of the familiarity, plausibility and directivity analysis by non-specialist judges

Task	Analysis performed	Themes	Average	Standard deviation	Decision of the authors
Conversational speech	Judgement of the degree of simplicity to talk to another person for 2 minutes about the subjects presented	Family	9.01	6.78	Priority subjects for a dialogue with an average of ≥8.00 were selected (Family, work, leisure, healthy nutrition).
		Work	8.87	7
		Leisure	8.97	7.7
		Global warming	6.27	2
		Fires	6.45	2.8
		Present government	6.97	2.3
		Trânsito	7.34	2.4
		Healthy nutrition	8.16	5.08
Metaphor interpretation	Judgement of the level of familiarity of new metaphors and idioms	New metaphor 1	7.26	1.16	Idioms with an average of ≥8.00 were maintained 3 expert judges were contacted in order to confirm the maintenance of the metaphors. The judges agreed to consider them unfamiliar expressions, determining to maintain the stimuli.
		New metaphor 2	7.89	2.50
		New metaphor 3	6.69	1.79
		New metaphor 4	7.90	2.87
		New metaphor 5	7.40	2.02
		New metaphor 6	7.50	2.09
		New metaphor 7	6.56	1.79
		New metaphor 8	7.80	1.88
		New metaphor 9	5.98	3.36
		New metaphor 10	7.88	2.65
		Idiom 11	8.77	1.65
Metaphor interpretation	Judgement of the level of familiarity of new metaphors and idioms	Idiom 12	9.14	1.00	Idioms with an average of ≥8.00 were maintained 3 expert judges were contacted in order to confirm the maintenance of the metaphors. The judges agreed to consider them unfamiliar expressions, determining to maintain the stimuli.
		Idiom 13	8.8	1.72
		Idiom 14	9.25	0.89
		Idiom 15	9.24	1.17
		Idiom 16	9.31	0.90
		Idiom 17	8.06	2.63
		Idiom 18	9.38	0.89
		Idiom 19	8.54	2.17
		Idiom 20	9.24	1.06
Semantic judgement	Judgement of the familiarity of the words that compose the pairs of the semantic judgement task	Plum	9.63	0.43	From the authors' criteria, all the words were selected, as they displayed an average of ≥7.00.
		Lavatory	9.62	0.43
		Bean	9.66	0.39
		Train	9.66	0.39
		Horse	9.66	0.39
		Plane	9.66	0.39
		Calf	9.59	0.52
		Gold	9.64	0.39
		Apple	9.66	0.39
		Eagle	9.66	0.49
		Pencil	9.66	0.39
		Dish	9.67	0.39
		Rubber	9.66	0.39
		Pearl	9.62	0.49
		Silk	9.62	0.57
		Cotton	9.64	0.43
		Diamond	9.57	0.80
		Grenade	9.5	0.96
		Copper	9.51	0.96
		Rifle	9.55	0.84
Semantic judgement	Judgement of the familiarity of the words that compose the pairs of the semantic judgement task	Sppon	9.67	0.39	From the authors' criteria, all the words were selected, as they displayed an average of ≥7.00.
		Loo	9.6	0.59
		Radish	9.56	0.84
		Sparrow	9.66	0.39
Speech acts interpretation	People's judgement on the situations presented, considering them as not direct at all or extremely direct	Situation 1 direct	7.41	2.79	Direct situations with an average of ≥7,00 were selected as well as indirect situations with an average of ≤5,00 The authors modified situations 9 and 15, since both did not meet the selection criteria established.
		Situation 2 indirect	1.96	1.46
		Situation 3 indirect	1.96	1.33
		Situation 4 indirect	2.52	1.71
		Situation 5 direct	9.12	0.87
		Situation 6 direct	8.51	1.49
		Situation 7 indirect	3.34	2.19
		Situation 8 direct	8.36	1.59
		Situation 9 direct	4.51	2.90
		Situation 10 indirect	2.86	2.17
		Situation 11 indirect	2.17	1.31
		Situation 12 direct	7.25	2.81
		Situation 13 direct	8.51	1.69
		Situation 14 direct	8.66	1.33
		Situation 15 indirect	6.19	2.93
		Situation 16 direct	7.41	2.37
		Situation 17 indirect	4.92	1.8
		Situation 18 direct	8.61	1.3
		Situation 19 indirect	2.18	1.61
		Situation 20 indirect	2.28	1.67

The situation number 9 of the speech acts task, "A Catarina olha para o filho que está a lavar os dentes antes de ir para a escola. Ela diz-lhe: "Estás a parecer um homenzinho"» ("Catarina looks at her son that brushes his teeth before going to school. An she says to him: ‘You look like a grown-up man'.") was evaluated by non-specialist judges as indirect. The authors decided to substitute it for "«…Ela diz-lhe: "Já lavas tão bem os dentes" » ("... She tells him: "You brush your teeth so well"), making it more direct. In the same task, the stimulus number 15, «O Sr. Octávio chega ao trabalho num dia de muito calor. Vira-se para o chefe e diz-lhe: "Isto aqui parece um forno"», ("Mr. Octávio arrives at work in a very hot day. He turns to his boss and tells him: this feels like an oven"), was judged by non-specialist judges as direct, and it was changed for «O Sr. Octávio chega ao trabalho num dia de calor. Ele diz ao chefe: "Está muito frio aqui"» ("Mr Octavio arrives at work on a hot day. He tells the boss: ‘It's too cold here'") in order to make it more indirect.

The second preliminary version of the stimuli book and the third version of the application and scoring guide of the MAC-PT Battery was the result of adaptations and developments of new stimuli conducted after the analysis step by non-specialist judges.

The level of concordance between expert judges in all tasks was 1. It can be concluded that all judges confirmed that the MAC-PT Battery stimuli assessed the cognitive/neuropsychological component they intended to evaluate.

In the second procedure, Step 4, judges considered the stimuli as appropriate regarding the instruction, suggesting modifications when necessary (Table 5).

Table 5
Analysis of instruction/items by expert judges

Process examined	Task	Concordance
Discursive	Narrative discourse - complete	1
Pragmatics	Metaphor interpretation - explanation	0.97
	Interpretation of speech acts - explanation	0.94
Lexical-semantic	Semantic judgement	0.95

From the expert judges' analysis and the suggestions they presented, the authors of the instrument checked all tasks and performed modifications. In the metaphor interpretation task, the term "sentence" present in the instructions was substituted for "phrases". The instruction of the speech acts interpretation test was modified from the suggestion provided by the judges, as they considered it difficult to understand.

Regarding the change in the stimuli, one of the expert judges suggested that an article should be added on the affirmative sentences of the comprehension and linguistic prosody repetition tasks, as in the example: "A Maria vai trabalhar" ("Mary will work"), instead of "Maria vai trabalhar". The authors decided not to modify these stimuli, as this would result in a syntactic clue that would facilitate the identification of affirmative sentences and would make the imperative sentences syntactically incorrect, not meeting the criteria of syntactic-semantic neutrality. In the narrative discourse task, it was suggested that the article "the" should be added before the sentences "António é um agricultor alentejano" ("Antonio is a farmer from Alentejo)" and "António devia estar a trabalhar no fundo do poço" ("António should be working at the bottom of the pits").

In relation to the application procedure of the MAC-PT Battery in a pilot study, it was found that the sample did not present difficulties in understanding the instructions and execution of tasks, managing to answer them properly, except for the three auditory stimuli representing joy in the emotional prosody test, which had only 40% of accurate answers. These stimuli were recorded and judged again by three of the seven expert judges, who rated them as appropriate. It was confirmed that the substitution of items 9 and 15 of the speech acts task were appropriate, and they were properly explained by 100% of the sample.

Just as in the previous step, it was also observed here that the participants did not have difficulties in understanding the tasks instructions and execution.

The level of concordance between judges, resulting from adaptation step, ranged between 50% and 100%, considering all items of the MAC-PT Battery. Concordance analyses with an index of over 80% were considered suitable. Only three items displayed concordance below this value and thus were assessed by a third judge: the idiom "O dono do café passou-se" ("The owner of the cafe was spent") displayed a concordance index of 76.66% between two judges, the indirect speech act "Está muito frio aqui" ("it is very cold here") displayed a concordance of only 50% and the pair of words "Feijão-Rabanete" ("Bean-Radish") of the semantic judgment task reached a value of 73.33%. A third judge evaluated the three items as critical, agreeing with one of the judges who assigned the highest value, maintaining the items and their scoring criteria.

After performing all the adjustments in the instruction-stimulus sets (s) during the adaptation process described above, the application and score guide underwent a final phase of adjustments. The evaluations of the second pilot study were qualitatively analyzed, so that the individuals' responses could contribute to the completion of the latest version of the MAC-PT Battery.

DISCUSSION

The procedures adopted in the adaptation process should be rigorous and evidence-based. In addition, several factors must be considered, including psycholinguistic and cultural criteria. The search for maximum equivalence between the original instrument and its translated version should guide the entire process, avoiding goals and methods distortion⁽¹⁷⁾.

In relation to the procedures used in this study, the step of comparative analysis between the MAC-BR⁽⁹⁾ and Protocole MEC⁽¹⁰⁾ Batteries (Step 1) allowed the authors to follow the adaptation from the Brazilian version of the instrument, as it was found that this version strictly followed the assumptions of the original version, avoiding thus a distortion of the content and a distancing from the original instrument. It is worth noting that it is simpler to adapt an instrument in the same language, nevertheless, one must be as careful as in the adaptation process of an instrument to a distinct language, since the European Portuguese and the Brazilian Portuguese have important linguistic and cultural differences.

After the comparison between instruments and after verifying the stimuli that are not sufficiently appropriate for the culture to which the new instrument will be assigned, it is necessary to promote the analysis by professional experts that will contribute with the adjustment process or the creation of new instructions and/or stimuli⁽¹⁸⁾. The adaptation procedure and development of new stimuli by expert judges (Step 2) was of utmost importance. This step consisted of two separate procedures: 1) performing lexical-syntactic adjustments on the battery and 2) adaptation of stimuli from semantic-pragmatic criteria by a panel of language experts.

The first procedure of step 2 may be compared to pure translations, which are frequently used as a single procedure in studies on adaptation of verbal stimuli tests⁽¹⁹⁾. However, some authors point out that the translation of the stimuli itself is not enough to ensure the quality of the original instrument, since language and culture interfere with neuropsychological assessment processes⁽¹⁸⁾. This type of study should include a specific process to minimize possible misinterpretations caused by language and cultural differences. The present research corroborates these arguments, since the results obtained from the first procedure proved insufficient to adapt the stimulus to the Portuguese socio-cultural and linguistic characteristics, and thus a semantic-pragmatic suitability analysis is essential.

A panel of seven language experts that participated in a brainstorming performed the second procedure⁽¹¹⁾. The adaptations promoted brought quick and quality results, considering that it is more advantageous to provide, within the framework of this study, a discussion among professionals to the detriment of developing an individual material to meet the objectives of this procedure. However, it must be emphasized that this procedure does not allow a quantitative scanning of results as individual applications do.

As a complement to the analysis and contribution of language experts, it is necessary to consult members of the general population to guide the authors in relation to the psycholinguistic criteria of each stimulus⁽¹⁸⁾. The analysis step carried out by non-specialist judges (Step 3) allowed the verification of the understanding of some test instructions, while confirming the quality of psycholinguistic aspects of the adapted and developed stimuli.

The analysis by expert judges (Step 4) was performed through two different procedures. The first proved to be very relevant, since the judges considered the cognitive/neuropsychological function or component as properly evaluated. The second procedure, wherein expert judges observed if each stimulus was adequate or not to its respective instruction, proved to be of little importance in relation to the suggested modifications. This is due to the fact that a great part of the cultural and linguistic changes required for the adaptation were performed in the two previous stages (adaptation and development of new stimuli by experts; and analysis by non-specialist judges). From these results, it is suggested that the adaptation and development of new stimuli by specialists (Step 2) was more important regarding the changes in the instruction-stimulus set.

Unlike some studies that suggest and/or carried out a one-step pilot study⁽¹⁰⁾, two different pilot studies were conducted in this research. From the Pilot Study 1 (Step 5), one could verify that it is necessary to confirm the understandability of instructions and stimuli whose adjustments were the most laborious of the entire battery and that underwent changes after the first pilot study. As well as in the Brazilian adaptation of the MAC-BR from the Canadian instrument, the metaphor interpretation, semantic judgment and speech acts tasks required a rigorous work of psycholinguistic adaptation, due to linguistic and cultural specificities, which do not allow a literal translation.

The Pilot Study 2 (Step 6) and concordance between judges were also considered essential for the socio-linguistic-cultural adaptation of the MAC-BR Battery to the European Portuguese. Through it, it is possible to check the application time of the instrument, a possible tiredness of the subject evaluated, in addition to the understandability of the instruction-stimulus set. From this procedure, as well as the MAC-BR battery⁽⁹⁾, the difficulties encountered by the study participants were resolved and the suggestions they provided were used to carry out improvements in the adapted instrument. The responses were essential for the construction of the scoring guide.

Although the adapted instrument is still being adjusted to the psychometric properties of validity, reliability and standardization, it is considered that the MAC-PT Battery presents content validity⁽²⁰⁾. This is due to the fact that, in addition to the actual adaptation, the concordance analysis between experts could also assist in the search for evidence of content validity⁽²⁰⁾. Besides, their tasks and stimuli were carefully selected to assess the main communicative deficits that may be present in RHD patients. It is noteworthy that the application and scoring guide was adequately adapted and standardized with clear instructions in relation to uniformity of procedures.

It is believed that, based on the discussion proposed here, the flow chart shown in Figure 1 of this paper can serve as a guide for adaptations processes geared towards future verbal neuropsychological instruments. However, it is important to highlight that each adaptation process is unique, because each instrument and each language has its peculiarities. Therefore, even though this flow chart may guide other authors of clinical tools, new steps or sub-steps can be added or rearranged.

Figure 1
Steps of the MAC-PT Battery adaptation process

It worth noting that the applicability of the MAC-PT needs to be investigated in different clinical populations characterized by sequelae in one or more of the communicative processes examined.

CONCLUSION

The MAC-PT Battery was created to meet a clinical and scientific gap in the Portuguese reality, for a better understanding of the RHD. This fact transforms the MAC-PT in an instrument capable of providing clear directions for the evaluation process, also because it is easy to apply and analyze.

REFERÊNCIAS

• ¹
  Tompkins CA. Rehabilitation for cognitive-communication disorders in right hemisphere brain damage. Arch Phys Med Rehabil. 2012;93(1 suppl):61-9. doi:10.1016/j.apmr.2011.10.015
• ²
  Baumann M, Le Bihan E, Chau K, Chau N. Associations between quality of life and socioeconomic factors, functional impairments and dissatisfaction with received information and home-care services among survivors living at home two years after stroke onset. BMC Neurol. 2014;14(1):2-12. doi:10.1186/1471-2377-14-92
• ³
  McDonnell MN, Bryan J, Smith AE, Esterman AJ. Assessing cognitive impairment following stroke. J Clin Exp Neuropsychol. 2011;33(9):945-53. doi:10.1080/13803395.2011.575769
• ⁴
  Fonseca RP, Ferreira GD, Liedtke FV, Müller JL, Sarmento TF, Parente MA. Alterações cognitivas, comunicativas e emocionais após lesão hemisférica direita: em busca de uma caracterização da síndrome do hemisfério direito. Psicol USP. 2006;17(4):241-62. doi:10.1590/S0103-65642006000400013
• ⁵
  Marazziti D, Consoli G, Picchetti M, Carlini M, Faravelli L. Cognitive impairment in major depression. Eur J Pharmacol. 2010;626(1):83-6. doi:10.1016/j.ejphar.2009.08.046
• ⁶
  Russell C, Deidda C, Malhotra P, Crinion JT, Merola S, Husain M. A deficit of spatial remapping in constructional apraxia after right hemisphere stroke. Brain. 2010;133(4):1239-51. doi:10.1093/brain/awq052
• ⁷
  Barnes S, Armstrong E. Conversation after right hemisphere brain damage: motivations for applying conversation analysis. Clin Linguist Phon. 2010;24(1): 55-69. doi:10.3109/02699200903349734
• ⁸
  Paradiso S, Anderson BM, Boles Ponto LL, Tranel D, Robinson RG. Altered neural activity and emotions following right middle cerebral artery stroke. J Stroke Cerebrovasc Dis. 2011;20(2):94-104. doi:10.1016/j.jstrokecerebrovasdis.2009.11.005
• ⁹
  Fonseca RP, Parente MAMP, Côté H, Ska B, Joanette Y, Ferreira GD. Bateria Montreal de Avaliação da Comunicação - Bateria MAC. Barueri: Pró-Fono; 2008.
• ¹⁰
  Joanette Y, Ska B, Côté H. Protocole Montréal d'Évaluation de la Communication. Isbergues: Ortho Edition; 2004.
• ¹¹
  Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. Petrópolis: Vozes; 2003.
• ¹²
  Fonseca RP, Zimmermann N, Pawlowski J, Oliveira CR, Gindri G, Scherer LH et al. Métodos em avaliação neuropsicológica: pressupostos gerais, neurocognitivos, neuropsicolinguísticos e psicométricos no uso e desenvolvimento de instrumentos. In: Fukushima SS Métodos em psicobiologia, neurociêcias e comportamento. São Paulo: ANPEPP; 2012.
• ¹³
  Madureira S, Guerreiro M, Ferro JM. Adaptação à população portuguesa da tradução do "Mini Mental State Examination" (MMSE). Rev Portug Neurol. 1994;32:9-10.
• ¹⁴
  Juby A, Tench S, Baker V. The value of clock drawing in identifying executive cognitive dysfunction in people with a normal Mini-Mental State Examination score. CMAJ. 2002;167(8):859-64. doi:10.1111/j.1532-5415.1989.tb02233.x
• ¹⁵
  Sunderland T, Hill JL, Mellow AM, Lawlor BA, Gundersheimer J, Newhouse PA et al. Clock drawing in Alzheimer's disease. A novel measure of dementia severity. J Am Geriatr Soc. 1989;37(8):725-9.
• ¹⁶
  Andrés AM, Marzo PF. Delta: a new measure of agreement between two raters. Br J Math Stat Psychol. 2004;57(1):1-19. doi:10.1348/000711004849268
• ¹⁷
  Casarin FS, Scherer LC, Parente MAMP, Ferré P, Côté H, Ska B et al. Adaptação do Protocole MEC de Poche e da Bateria MAC Expandida: Bateria MAC Breve. Psico. 2013;44(2):288-99.
• ¹⁸
  Fonseca RP, Casarin FS, Oliveira CR, Gindri G, Ishigaki ECSS, Ortiz KZ et al. Adaptação de instrumentos neuropsicológicos verbais: um fluxograma de procedimentos para além da tradução. Est Inter Psicol. 2011;15:59-69. doi:10.5380/psi.v15i0.25374
• ¹⁹
  Casillas A, Robbins SB. Test adaptation and crosscultural assessment from a business perspective: Issues and recommendations. Int J Test. 2005;5:5-21. doi:10.1207/s15327574ijt0501_2
• ²⁰
  Pasquali L. Instrumentação psicológica. São Paulo: Artmed; 2010.

Publication Dates

• Publication in this collection
  Jul-Sep 2015

History

• Received
  13 Apr 2015
• Accepted
  15 June 2015