Portuguese adaptation of Shulman’s test of pragmatic skills

ABSTRACT

Purpose

The purpose of this study was to apply the Test of Pragmatic Skills in Brazilian children with normal development.

Methods

Participants of this study were 60 children 3:00 to 8:11 years old, equally divided between genders. The examiners had prior contact with the group of children and applied the TOPS – a traditional test that proposes four sets of simple activities, with the common material, applicable in a short period of time – according to the procedure proposed by the author.

Results

The results showed a large variability in performance, illustrating that the linguistic and pragmatic skills develop according to the age. It was also observed that the subjects performed all the categories of communicative intentions required by Shulman, indicating that it is not the presence or absence of a specific intent that influences the score in this test, but the quality of the answers provided, which evolves with age.

Conclusion

Thus, we can conclude that the TOPS proved to be an important instrument that allows answers in informal conversation contexts, verifying that the test is effective in assessing the pragmatic skills of typically developing children and can be an important tool to be used in the assessment of communicative intentions in Brazilian children.

Keywords
Child Language; Speech, Language, and Hearing Sciences; Communication; Evaluation; Protocols

RESUMO

Objetivo

A proposta do presente estudo foi aplicar o Test of Pragmatic Skills em crianças brasileiras com desenvolvimento típico.

Método

Foram participantes desta pesquisa 60 crianças de 3:00 a 8:11 anos, divididas igualmente entre os gêneros. As examinadoras tiveram contato prévio com o grupo de crianças e aplicaram o TOPS – um teste tradicional composto por quatro conjuntos de atividades simples, com material comum, aplicáveis em um curto período de tempo – segundo o proposto pelo autor.

Resultados

Nos resultados, observamos bastante variabilidade no desempenho, exemplificando que as habilidades linguísticas e pragmáticas vão se desenvolvendo de acordo com a idade. Foi possível observar, ainda, que os participantes apresentaram as categorias de intenções comunicativas pretendidas por Shulman, indicando que não é a presença ou ausência de alguma intenção específica que influencia a pontuação nesse teste, mas a qualidade das respostas fornecidas, que evolui com a idade.

Conclusão

Desta forma, podemos concluir que o TOPS mostrou-se importante por permitir respostas em contextos informais de conversação, verificando-se que o teste é eficaz para avaliar as habilidades pragmáticas de crianças de desenvolvimento típico e pode ser um instrumento relevante a ser empregado na avaliação das intenções comunicativas de crianças brasileiras.

Descritores
Linguagem Infantil; Fonoaudiologia; Comunicação; Avaliação; Protocolos

INTRODUCTION

In recent years, the models of understanding the pragmatic aspects of language^(1-3) have had a major impact on the study of language disorders. Within a broad perspective, these models attempt to characterize the individual's communicative competence by reflecting a complex dynamic between concepts: knowledge of the structural language (knowledge of the linguistic code), presuppositional knowledge (ability to make correct judgments about how a statement should be, to be appropriate to the speaker’s intention) and conversational knowledge (understanding conversational rules within an interactive society). This functional perspective provides the basis for understanding how language is determinant for communicative success⁽⁴⁾.

For an individual to succeed in communication, he should not only be able to use language in a given situation appropriately, but he must also be able to understand the social cues used by the interlocutors⁽⁵⁾. This kind of pragmatic competence may include verbal, paralinguistic and non-verbal communication skills, as well as social understanding and communicative context⁽⁶⁾.

Using the context as a premise and thinking about its variety related to the use of language, the listener’s interpretation of the speaker’s intentions is of extreme importance⁽⁷⁾, so it is possible to handle like other aspects of language (phonology, semantics, morphology, and syntax) are used in conversational contexts^(8,9), highlighting the important intersection between language development and social interactions⁽¹⁰⁾.

Pragmatics analyses the function of language in communication⁽¹¹⁾ and refers to a series of rules that explain or regulate the intentional use of language⁽¹²⁾. Communication skills are related to an individual’s ability to participate in an interactive sequence of speech acts with the goal of communicative exchange. Thus, communicative competence is the ability to use language as an effective tool for interacting in other social contexts. This competence involves the speaker’s communicative intent, regardless of the means used for communication.

The communicative act is the minimum unit of analysis of the pragmatic aspects, focusing on each communicative shift, regardless of the general organization of the dialogue⁽¹³⁾. They can be studied in their language structure, the intentions they convey (this is the general focus of the analysis) and their effects on the listener. However, the acts analyzed are necessarily those that fit into a certain idea of assertiveness. The communicative intention is the overlapping point of the intention to communicate, to want to be understood and to understand the social mechanisms necessary to accomplish the task. Therefore, these initiatives are the privileged meeting point between one individual and the other.

However, the communicative exchanges are double. On one side, there is an initiative and, on the other hand, there is the responsiveness⁽¹⁴⁾. The exchange will only be effective if participants share a common core of interaction⁽¹⁵⁾. During a dialogue, there must be an effective adjustment between speech and silent segments as well as non-verbal elements⁽¹⁶⁾. An unanswered statement is considered to be a communicative failure by the initiator, and therefore most responses have a very important value in maintaining the communicative flow. Some authors^(13,16) have studied the question of responsiveness in children. The authors proposed four kinds of adjustment between the preceding discourse and the response. In this way, they aimed to understand the degree of harmony determined between the interlocutors. Answering means understanding the previous discourse, being interested in continuing to speak, providing new information and ensuring a turnaround. In this way, responding is - as well as initiatives - another privileged meeting place between one individual and the other.

Other authors⁽¹⁷⁾ considered responsiveness as a new proposal for pragmatic analysis and concluded that considering responses from the perspective of the functional profile of communication, relevant information on the overall structure of communicative intentions and on broader communicative skills are provided, is very important for language evaluation.

The Test of Pragmatic Skills⁽¹⁸⁾ (ToPS) is a traditional test that, for its ease of application, it has been used in several studies. It is composed of four sets of simple activities, with the common material, applicable in a short time (about 15 minutes). It has normal data for American and Greek children and it is being applied in some countries in the East.

The possibility of expanding resources for speech-language diagnosis using instruments that can be shared internationally represents a significant contribution to science and relevant to clinical practice. Thus, the purpose of this study is to apply the translated version of the ToPS-R.

There is a need for internationally recognized instruments that are applicable nationally. Thus, the proposal to adapt the Portuguese ToPS-R began to be conducted by translating the test manual into Portuguese, its back-translation by two people fluent in English (a speech therapist and a non- speech therapist) and the author’s approval of the Brazilian Portuguese version. The purpose of this article is to apply the protocol with typical Brazilian children.

METHODS

This study was approved by the Research Ethics Committee of the institution under number 366/13. Those responsible for all study participants signed the Informed Consent Form agreeing with the children’s participation in the research.

Casuistry

The criteria considered for the inclusion of the participants in the research were:

• Do not have history of serious developmental changes;

• Do not have sensory losses;

• Be regularly enrolled in the public education network in São Paulo;

• Did not perform speech and/or psychotherapeutic treatment;

• Have the Portuguese language as the only language.

From these criteria, there were 60 children selected from 3:0 to 8:11 years old, with five girls and five boys in each age group. The test was applied to each participant’s school in a reserved room. Participants were divided into four groups, according to their age. The G1 consisted of five girls and five boys aged between 3:00 and 3:11 years old, from the kindergarten 1; the G2, was composed of five girls and five boys between 4:00 and 4:11 years old, from the kindergarten 2; the G3 was composed of five girls and five boys aged between 5:00 and 5:11 years old, from the preschool; the G4 was composed of five girls and five boys between 6:00 and 6:11 years old from the 1^st grade; the G5 was composed of five girls and five boys between 7:00 and 7:11 years old from the 2^nd grade; and the G6 consisted of five girls and five boys aged 8:00 to 8:11 years old from the 3^rd grade.

Material

The objects proposed by the original test were used:

• Two medium-sized cloth puppets, one male, and one female;

• Two toy telephones;

• 10 colored cubic blocks of MDF;

• A worksheet containing three printed geometric shapes;

• Pencil.

Also, the material used in this study included the test task guide and a camera, for filming the test.

Procedures

After the definition of the participants, each student was presented with a prior presentation to the groups and the classroom activities were followed for two weeks to familiarize the children with them. Also, prior to the application of the test, they would talk informally with each child to create a friendly situation.

The tasks were proposed exactly as described in the task guide. If the child failed or did not respond after the first attempt, it was repeated only once. The time of the test application varied between seven and 25 minutes. The details of the evaluative tasks and the possibilities of communicative intentions are shown in Table 1.

Table 1
Detailed evaluation tasks

TASK	TYPE OF CONTEXT	NUMBER OF TESTS	TYPES OF INTENTIONS
1	Playing with puppets	10	Greeting
			Answering
			Reporting
			Naming/Labeling
			Rejection/Denial
			Requesting information
			Reasoning
			Ending conversation
2	Playing with pencil and paper	7	Calling/Calling
			Requesting information
			Requesting Action
			Reporting
			Answering
			Rejection/Denial
			Reasoning
			Naming/Labeling
3	Playing with phones	9	Greeting
			Answering
			Reporting
			Requesting information
			Naming/Labeling
			Ending conversation
4	Playing with blocks	8	Requesting information
			Requesting action
			Rejection/negation
			Naming/Labeling
			Answering
			Reporting
	Total	34

Data analysis

Once applied, each task of the test is scored according to linguistic sophistication and how appropriate the response is. The scoring scale ranges from 0 to 5 and is described in Chart 1.

Chart 1
Scoring scale

Score	Description
0	No answer
1	Contextually inappropriate
2	Contextually appropriate answer, only gestural/nonverbal
3	Contextually appropriate answer, with one word without elaboration
4	Contextually appropriate answer, with minimal elaboration (two or three words)
5	Contextually appropriate answer, with extensive elaboration (more than three words)

After the score of each task, the Average Composite Score (ACS) is calculated, which is defined by the sum of the score of the four tasks divided by four (average). From this score, it is possible to determine the percentile of each child’s performance from the normative table according to the age group⁽¹⁸⁾.

RESULTS

Data were analyzed according to the groups in relation to the age groups. The minimum and maximum scores, mean, median and standard deviation of each of the tasks are presented below.

Task 1 consists of two puppets talking about their favorite television shows. Task 2 involves the child copying drawings in lines in three different formats. The use of communicative intentions by the child is the most important in this task and not the drawings. The drawings are only meant to facilitate the desired actions. Task 3 is intermediated by toy telephones and the examiner and the child talk. Task 4 involves 10 blocks of wood and the examiner and the child talk, a task is performed with the blocks whose goal is not the correctness of the task, but the possibility of presenting communicative initiative from the interaction. The score that reflects the performance of the groups in the 4 tasks is shown in Table 2.

Table 2
Group performance in TOPS tasks

	Task 1						Task 2						Task 3						Task 4
	G1	G2	G3	G4	G5	G6	G1	G2	G3	G4	G5	G6	G1	G2	G3	G4	G5	G6	G1	G2	G3	G4	G5	G6
Mean	29.8	36.7	31.7	33.9	34	38	20.3	25	19.1	20.1	20.7	24	24.5	31.5	25.6	29.7	32.1	32	21.3	28.9	24.1	27.1	25.2	28.2
Median	33.5	38	33	34	35	37.5	20.5	25	19	21	20	23.5	24.5	32.5	25	30	32.5	32.5	21	30.5	23.5	27.5	24.5	27.5
SD	12.2	6.4	9.4	4.8	5.9	2	5	4.1	4.3	3.2	5.4	1.6	3.6	5.1	5.4	3.2	1.8	3.2	6.1	6	6	5.3	5.1	5.3
Minimum	7	23	10	24	19	35	12	16	14	13	11	22	19	19	19	25	30	25	13	15	17	18	19	19
Maximum	50	44	41	39	40	41	29	31	26	24	31	27	30	36	35	34	35	37	29	35	35	34	36	38

The ACM was obtained from the simple mean of the scores of the four tasks and the data are presented below in Table 3, which synthesizes the score data obtained in each of the groups.

Table 3
TOPS PCM group performance

	Minimum	Maximum	Mean	Median	SD
G1	17	31	24.3	26	4.9
G2	19	36	30.9	31	4.7
G3	19	33	25.2	24	4.8
G4	22	32	27.7	27.8	2.7
G5	21	35	28.5	28	3.7
G6	28	33	30.9	31	1.9

ACM data generated the percentile of each individual’s performance from the normative table. The figure below (Figure 1) shows the performance in relation to the groups divided by the percentiles.

Figure 1
Performance of the groups in the age group according to the percentiles

Student’s t-test with a 95% confidence interval was performed to determine if the most frequent percentile was significant for each of the groups. The 99 percentile was the most frequent for G1 and G2, is significant for these age groups (p=0.001 and p<0.0001, respectively).

G3 presented the highest variability in percentiles distribution, with the percentile <10, being the most frequent, but this data was not significant (p=0156). For G4, the most frequent percentile was 50, is significant (p=0.0003) for this age group. The percentile <10 was also the most frequent in G5 and, for this age group, it was significant (p=0.005). For G6, the most frequent percentile was 25, is significant for this age group (p=0.0001).

DISCUSSION

The Test of Pragmatic Skills is able to describe the pragmatic abilities of children between 3 and 8 years old due to changes in communicative contexts. This assessment instrument was designed with the purpose of being used in several clinical conditions with suspected language alterations related to the inappropriate use or lack of communicative intention.

The objective of this research was to apply the Portuguese version of TOPS developed by Shulman⁽¹⁸⁾ in 1986 in Brazilian children. The standardization of the test was performed with middle-class American children with the numerical balance between genders and those based on American census data of the time and diversified among the geographic regions of the USA. Therefore, the objective of this work does not involve standardization and nor definition of norms of normality for the Brazilian population, but a possibility of evaluation and characterization of the pragmatic aspects of the studied population.

The performance in the tasks and in the ACM was better in the participants of groups 1 and 2 and worse in the older children (from 5 years old). Considering the date of development of the test - 1986 - it is congruent to assume that the tests performed in the tasks may be directly related to the lack of interest of the older children, as observed in the application.

Also, authors suggest⁽¹⁹⁾ that some pragmatic aspects are not measurable only in small samples, but rather need richer and broader moments of conversation.

When comparing the mean data obtained in the studied population with the mean data obtained in the normalization, we can see that the participants of groups 1 and 2 present average performance in the tasks and, in ACM, better than those obtained with American children, but the participants of groups 3, 4, 5 and 6 present worse performance than those presented in the American data.

A study by Chakrabarty et al.⁽²⁰⁾ suggested that education is an important variable that interferes with individuals’ pragmatic abilities. Another study⁽²¹⁾ found that children with learning difficulties presented lower scores than children of typical development in a test that investigates pragmatic abilities, mainly regarding the amount of speech acts, inadequate maintenance of the topic, change of shift, limitation of strategies for repairing communicative breaks, and coherence and cohesion.

Considering the educational issue and relating it to the social positioning of the individuals in this study, we are led to conclude that the educational abilities and school performance of older Brazilian children (from 5 years old) are more dissimilar than American children. This fact seems to be determinant, not exactly in the pragmatic development of language, but rather within an evaluative perspective of the pragmatic aspects, perhaps as regards sensitivity (or lack of it) in standardized instruments.

A study carried out in 2016 by Spanoudis⁽²²⁾ also concluded that the tasks presented by them did not effectively measure the intended evaluation constructs, often due to the difficulties of the tasks, and also associates with the possibility of diversification linguistic issues, which are more interfering with pragmatic skills.

CONCLUSION

Standardized and validated tests in other languages and widely used to evaluate children’s language are extremely important for clinical research and definition of specific parameters for diagnostic criteria within Brazilian Speech-Language Pathology. However, understanding the limitations and possibilities of these protocols when applied from a translation proposal is part of the process of investigating the sensitivity of the instrument to the language before its correct validation and diffusion.

When tested with TOPS, Brazilian children tend to perform better when they are younger. Several qualitative factors may be related to this performance, the interest in the activities seems to be determinant since the younger children became more interested in the materials and tests proposed by the test. On the other hand, this test seems to be more sensitive to identify the variability of the performance of children between 5 and 6 years old, but more studies are necessary to increase the number of participants in the age groups studied so these statements can be better substantiated.

Also, one of the limitations found in this study was regarding the studied population. Perhaps, expanding the sample in relation to the different social levels may be an important strategy to better characterize the Brazilian population evaluated by TOPS.

• Study carried out at Universidade de São Paulo – USP - São Paulo (SP), Brazil.
• Financial support: Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).

REFERÊNCIAS

• ¹
  Austin J. How to do things with words. Oxford: Basil Blackwell; 1962.
• ²
  Bates E. Language in context. New York: Academic Press; 1976.
• ³
  Searle JR. What is a speech act? In: Black M. Philosophy in America. London: Allen and Unwin; 1965. p. 221-239.
• ⁴
  Gallagher TM. A retrospective look at clinical pragmatics. San Diego: Springer; 1991. p. 1-10. Pragmatics of Language: Clinical Practice Issues.
• ⁵
  Murza KA, Nye C. Pragmatic language intervention for adults with asperger syndrome or high-functioning autism: a feasibility study. Contemp Issues Commun Sci Disord. 2013;40:85-97.
• ⁶
  Loukusa S, Moilanen I. Pragmatic inference abilities in individuals with Asperger syndrome or high-functioning autism: a review. Res Autism Spectr Disord. 2009;3(4):890-904. http://dx.doi.org/10.1016/j.rasd.2009.05.002
  » http://dx.doi.org/10.1016/j.rasd.2009.05.002
• ⁷
  Berko-Gleason J. The development of language. Boston: Pearson /Allyn & Bacon; 2005.
• ⁸
  Staikova E, Gomes H, Tartter V, McCabe A, Halperin JM. Pragmatic déficits and social impairment in children with ADHD. J Child Psychol Psychiatry. 2013;54(12):1275-83. PMid:23682627. http://dx.doi.org/10.1111/jcpp.12082
  » http://dx.doi.org/10.1111/jcpp.12082
• ⁹
  Russell RL. Social communication impairments: pragmatics. Pediatr Clin North Am. 2007;54(3):483-506, vi. PMid:17543906. http://dx.doi.org/10.1016/j.pcl.2007.02.016
  » http://dx.doi.org/10.1016/j.pcl.2007.02.016
• ¹⁰
  Philofsky A, Fidler DJ, Hepburn S. Pragmatic language profiles of school-age children with autism spectrum disorders and Williams syndrome. Am J Speech Lang Pathol. 2007;16(4):368-80. PMid:17971496. http://dx.doi.org/10.1044/1058-0360(2007/040)
  » http://dx.doi.org/10.1044/1058-0360(2007/040)
• ¹¹
  Abe CM, Bretanha AC, Bozza A, Ferraro GJK, Lopes-Herrera SA. Verbal communication skills in typical language development: a case series. CoDAS. 2013;25(1):76-83. PMid:24408175. http://dx.doi.org/10.1590/S2317-17822013000100014
  » http://dx.doi.org/10.1590/S2317-17822013000100014
• ¹²
  Acosta VM, Moreno A, Ramos V, Quintana A, Espino O. Avaliação do desenvolvimento pragmático. In: Acosta VM. Avaliação da linguagem: teoria e prática do processo de avaliação do comportamento lingüístico-infantil. São Paulo: Santos; 2003. p. 33-51.
• ¹³
  Adams C, Lloyd J. Elicited and spontaneous communicative functions and stability of conversational measures with children who have pragmatic language impairments. Int J Lang Commun Disord. 2005;40(3):333-47. PMid:16195192. http://dx.doi.org/10.1080/13682820400027768
  » http://dx.doi.org/10.1080/13682820400027768
• ¹⁴
  Bishop DVM, Chan J, Adams C, Hartley J, Weir F. Conversational responsiveness in specific language impairment: evidence of disproportionate pragmatic difficulties in a subset of children. Dev Psychopathol. 2000;12(2):177-99. PMid:10847623. http://dx.doi.org/10.1017/S0954579400002042
  » http://dx.doi.org/10.1017/S0954579400002042
• ¹⁵
  Branigan HP, Pickering MJ, Pearson J, McLean JF. Linguistic alignment between people and computers. J Pragmatics. 2010;42(9):2355-68. http://dx.doi.org/10.1016/j.pragma.2009.12.012
  » http://dx.doi.org/10.1016/j.pragma.2009.12.012
• ¹⁶
  Adams C, Green J, Gilchrist A, Cox A. Conversational behavior of children with Asperger syndrome and conduct disorder. J Child Psychol Psychiatry. 2002;43(5):679-90. PMid:12120863. http://dx.doi.org/10.1111/1469-7610.00056
  » http://dx.doi.org/10.1111/1469-7610.00056
• ¹⁷
  Miilher LP, Fernandes FDM. Considering responsivity: a proposal for pragmatic analysis in autism spectrum. CoDAS. 2013;25(1):70-5. PMid:24408174. http://dx.doi.org/10.1590/S2317-17822013000100013
  » http://dx.doi.org/10.1590/S2317-17822013000100013
• ¹⁸
  Shulman BB. Test of pragmatic skills: revised. Tucson: Communication Skill Builders; 1986.
• ¹⁹
  Lockton E, Adams C, Collins A. Do children with social communication disorder have explicit knowledge of pragmatic rules they break? A comparison of conversational pragmatic ability and metapragmatic awareness. Int J Lang Commun Disord. 2016;51(5):508-17. PMid:26916221. http://dx.doi.org/10.1111/1460-6984.12227
  » http://dx.doi.org/10.1111/1460-6984.12227
• ²⁰
  Chakrabarty M, Biswas A, Pal S, Chatterjee A. Development of bengali audio–visual test battery for assessment of pragmatic skills: preliminary normative data based on educational level. J Psycholinguist Res. 2017;46(2):295-309. PMid:27246521. http://dx.doi.org/10.1007/s10936-016-9438-y
  » http://dx.doi.org/10.1007/s10936-016-9438-y
• ²¹
  Kumari S, Pyata R, Afreen BK, Paithankar PS. Pragmatic skills in children with different types of learning disability: a comparative study. Language in India. 2016;16(9):224-33.
• ²²
  Spanoudis G. Theory of mind and specific language impairment in school-age children. J Commun Disord. 2016;61:83-96. PMid:27089201. http://dx.doi.org/10.1016/j.jcomdis.2016.04.003
  » http://dx.doi.org/10.1016/j.jcomdis.2016.04.003

Publication Dates

• Publication in this collection
  2018

History

• Received
  20 Feb 2017
• Accepted
  17 Aug 2017