Validity evidence based on response processes of the TRILHAR – screening tool for infant vocabulary

ABSTRACT

Purpose

To verify the validity evidence based on response processes of a vocabulary-screening tool.

Methods

This is a descriptive, cross-sectional and quantitative study, applied in a sample of 133 children between 3 and 7 years of age, divided into five groups, according to their age range. This research evaluates the instrument TRILHAR, which is a screening of receptive and expressive vocabulary, composed by ten activities for each age range. The collected data were the application time in minutes and seconds, screening instructions comprehension and behavior during the screening. The data were analyzed descriptively as mean, standard deviation and percentage using the IBM SPSS Statistics.

Results

The application time was short, with less time for the greater ages. The mean time of application was 05m19s for G1, 04m29s for G2, 04m11s for G3, 03m40s for G4 and 02m13s for G5. Only one child (4.8%) from G2 required repetition of the instructions for the receptive vocabulary, and two children (6.3%) from G4 for the expressive vocabulary. We observed little occurrence of behaviors like disinterest, inattention and agitation.

Conclusion

The application of the instrument required a short period, with a longer time for the group of 3-years children. The little necessity of repetition of the instructions indicates that children can easily comprehend the instrument.

Keywords:
Validation Studies; Mass Screening; Infant Language; Protocols; Vocabulary Tests; Vocabulary

RESUMO

Objetivo

Verificar as evidências de validade baseadas nos processos de resposta de um instrumento de triagem do vocabulário.

Método

Trata-se de um estudo descritivo, transversal e quantitativo, aplicado em uma amostra de 133 crianças entre 3 e 7 anos de idade, subdivididos em cinco grupos de acordo a faixa etária. O instrumento sobre o qual se trata a pesquisa é o TRILHAR, que visa a triagem do vocabulário receptivo e expressivo, composto por dez fichas de atividades para cada idade-alvo. Os dados coletados focaram no tempo de aplicação em minutos e segundos, compreensão das instruções do teste e comportamento do escolar durante a aplicação. Os dados foram analisados descritivamente em relação a média, desvio-padrão e porcentagem por meio do software IBM SPSS Statistics.

Resultados

O tempo de aplicação foi curto e diminuiu de acordo com a progressão da idade. O tempo médio de aplicação foi de 05m19s para o G1, 04m29s para o G2, 04m11s para o G3, 03m40s para o G4 e 02m13s para o G5. Apenas um sujeito (4,8%) do G2 necessitou de repetição das instruções no vocabulário receptivo e dois (6,3%) do G4 no vocabulário expressivo. Foi verificada pequena ocorrência de comportamentos como desinteresse, desatenção e agitação.

Conclusão

Foi verificado que o tempo de aplicação do instrumento necessita de um curto período de tempo, sendo mais dispendiosa no grupo de crianças com três anos de idade. A baixa necessidade de repetição das instruções indica que o instrumento apresenta fácil compreensão por parte da criança.

Descritores:
Estudos de Validação; Programas de Rastreamento; Linguagem Infantil; Protocolos; Testes de Vocabulário; Vocabulário

INTRODUCTION

A screening test is a procedure applied to healthy and unhealthy individuals, aiming to identify those who show signs of risk for the development of a certain condition or disease, so that early intervention brings better results⁽¹⁾. An effective screening test must be validated, simple to understand and apply, have reproducibility and accuracy, in addition to treatment opportunities for target individuals^(2-4).

The validity of response processes verifies the psychological, cognitive and social processes involved in the application of the instrument, such as operational difficulties, application time, non-verbal reactions and the tested individual's understanding of the instructions and questions⁽⁵⁾. One of the most used ways to obtain response validity is through cognitive interviews⁽⁶⁾.

In the context of Speech-Language Therapy in Brazil, research related to the instrument validation process is scarce⁽⁷⁾. Regarding the screening of children's vocabulary, this scenario is reaffirmed, indicating the need for research in this area⁽⁸⁾, as the early identification and intervention bring longitudinally positive results in the development of the individual's language⁽⁹⁾. Children with language development delays who are not subjected to intervention tend to maintain performance below their peers throughout life⁽¹⁰⁾.

Given the data presented above and the scarcity of vocabulary screening instruments in the Brazilian clinical and research setting, this study aimed to verify the evidence of validity based on the response processes of a vocabulary screening instrument for children between 3 and 7 years old.

METHODS

This is a descriptive, transversal and quantitative research, approved by the Research Ethics Committee (Comitê de Ética em Pesquisa - CEP), opinion number 2,548,341. All participants signed the Informed Consent Form (ICF) and Informed Assent Form (IAF), if necessary for children over 7 years of age.

The sample size was based on national literature, which for the validation step, according to the response processes, must be composed of at least ten individuals from each group listed below⁽⁵⁾. The convenience sample consisted of 133 children between 3 and 7 years old, enrolled in regular public education, with no complaints of changes in language development or diagnoses of syndromes and disabilities, according to the teachers' report. They were divided into the following groups, considering the age group:

• Group 1 - 38 individuals aged three years;

• Group 2 - 21 individuals with four years of age;

• Group 3 - 30 individuals with five years of age;

• Group 4 - 32 individuals with six years of age;

• Group 5 - 22 individuals with seven years of age.

This study is a stage of the validation of the receptive and expressive vocabulary screening instrument - TRILHAR⁽⁸⁾, indicated for children between three and seven years of age. The goal was to identify early preschoolers and schoolchildren with signs of risk for alterations related to the semantic level of language, especially in the educational field, aiming at early intervention and prevention of possible difficulties resulting from the restriction of vocabulary.

TRILHAR consists of ten receptive vocabulary cards, in which the child hears a word and points to the corresponding figure among 4 options offered simultaneously; and ten expressive vocabulary cards, which shows a figure for the child to name. The screening figures are specific and differentiated for each age group covered by the instrument, selected and distributed according to criteria previously described⁽⁸⁾. It consists of words from the following semantic categories: clothes, animals, food, furniture, utensils, means of transport, toys, instruments, professions, places, body parts, adjectives and verbs. The maximum score is twenty points, with ten for receptive activity and ten for expressive activity.

We contacted two public schools of infant and fundamental education to obtain informed consent (ICF) and informed assent (IAF) forms. Subsequently, the parents were invited to participate in a meeting with explanations about the research and the operation of the instrument. Finally, the ICF and IAF were signed. All screenings were applied individually, in a room with adequate acoustic and lighting conditions, at the agreed time with the teachers of each class, using the printed material of the instrument, with the support of the puppet to motivate the child. The application was started by the receptive vocabulary, followed by the expressive vocabulary.

The instruction used to sort the receptive vocabulary was as follows: “I will show you four drawings and I will ask you to point out one of them”. As for the expressive vocabulary: “I'm going to show you a picture and I want you to speak its name”. To verify the understanding of the instructions, the paraphrase strategy⁽⁵⁾ was used, which consists of the repetition of the order by the individual. After explaining the activity, the child was asked if he understood and then asked to repeat what had been said.

To complement the analysis of the understanding of the instructions, the following behaviors were recorded to indicate operational difficulties⁽⁵⁾ in performing the screening: disinterest, facial expressions of estrangement, agitation, negative response, irritation, getting up from the chair and crying.

The screening application time was recorded by using a stopwatch on the researchers' cell phones in minutes and seconds’ format. The counting started from the start of the explanation of the instructions and was completed at the end of the screening, including the receptive and expressive vocabularies.

All data were analyzed descriptively, using IBM SPSS Statistics, version 23.

RESULTS

In all groups, we observed a high rate of understanding of the instructions and a low percentage of required repetition for understanding (Table 1). The main behavioral observations were facial expressions of strangeness and agitation (Table 2).

Table 1
Percentage of understanding of orders required repetitions of screening instructions

		G1	G2	G3	G4	G5
Receptive vocabulary	Understanding	100% (n=28)	95.2% (n=20)	100% (n=30)	100% (n=32)	100% (n=22)
	Repetitions	0% (n=0)	4.8% (n=1)	0% (n=0)	3.1% (n=1)	0% (n=0)
Expressive vocabulary	Understanding	100% (n=28)	95.2% (n=20)	100% (n=30)	93.8% (n=30)	100% (n=22)
	Repetitions	0% (n=0)	0% (n=0)	0% (n=0)	6.3% (n=2)	0% (n=0)

• Caption: G1 = group 1; G2 = group 2; G3 = group 3; G4 = group 4; G5 = group 5

Table 2
Occurrence (n) of observed behavioral characteristics

	G1 (n)	G2 (n)	G3 (n)	G4 (n)	G5 (n)
Disinterest	0	3	0	1	0
Strange facial expressions	0	1	3	3	0
Agitation	4	0	4	1	0
Denied answer	1	1	1	0	0
Irritation	0	0	0	0	0
Got up from the chair	0	0	1	0	0
Cried	0	0	0	0	0

• Caption: G1 = group 1; G2 = group 2; G3 = group 3; G4 = group 4; G5 = group 5

Finally, the application time of the receptive and expressive vocabularies decreased according to the age group progression. G1 showed an application time of 05m29s (sd=01m41s), followed by G2 with 04m29s (sd=01m19s). G3 completed the test in 04m11s (sd=01m12s), G4 in 03m40s (sd=01m12s) and G5 02m13s (sd=47s).

DISCUSSION

Our results show that TRILHAR has satisfactory response processes. The observed behavioral characteristics indicate that the instrument was able to retain the child's attention, being simple to apply. Thus, these findings are important as they are consistent with the screening procedure, which must be simple and quick^(2-4), as observed in the application of the instrument in the sample. The ability to retain the child's attention is extremely important, as it reduces the chances of unfavorable results due to factors such as inattention and lack of interest. This is because the figures of the instrument are constructed aiming at playful application, as well as the use of the puppet makes the environment more interesting for the individual, favoring his active participation in the screening process.

As for the instrument instructions, only one child from G2 and two children from G4 required more information and were unable to repeat the indications in the paraphrase strategy. This is a positive characteristic of the test since quality instructions must be clear and precise for the population to whom the test is directed⁽¹¹⁾. The possibility of low performances due to faulty instructions is high, justifying careful attention to this aspect.

The average application time decreased according to the increase in the age group, which can probably be explained by the characteristics of the typical vocabulary development, that is, the older the child's lexicon is composed of more words⁽¹²⁾. This factor facilitated not only the understanding of the instructions, but also the child's response speed, thus reducing the application time. Another possible explanation is that older children have better attentional development⁽¹³⁾, thus reducing the need for additional time to respond to the test. TRILHAR was built aiming at a short application time and, therefore, it consists of 10 items of receptive vocabulary and 10 of expressive vocabulary for each age group.

A limitation of the research is the failure to explore the most qualitative data regarding validation based on the response process. Future studies should be conducted to verify whether the type of instruction offered to the child influences the quality of his responses, more specifically his test score. Another limitation refers to the study sample, as it was composed only of children students from public schools. It is also important to compare the performance of children in situations of socioeconomic vulnerability, as they may have greater difficulties in vocabulary⁽¹⁴⁾, which can directly influence the understanding of the instructions and performance in the final results of the instrument, since the sample in this article was composed of student children from a public school of reference in the region. It is also considered necessary to apply it to students from private schools, as they can present better results in vocabulary tests⁽¹⁵⁾.

CONCLUSION

From the analysis of the response processes, we found that the TRILHAR is an instrument easily understood by the child, since only one individual from G2 and two individuals from G4 had difficulties in this aspect. The test has a low application time, characteristics compatible with the definition of the screening procedure: an average of 05m19s for G1, 04m29s for G2, 04m11s for G3, 03m40s for G4 and 02m13s for G5. Future research is needed to complement the instrument with other validity data, such as internal consistency, reliability, and accuracy.

ACKNOWLEDGEMENTS

To Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes) for their support in carrying out this research.

• Study conducted at Departamento de Fonoaudiologia, in partnership with Departamento de Engenharia Biomédica, Universidade Federal do Rio Grande do Norte – UFRN - Natal (RN), Brasil.

• Financial support:

  Master’s scholarship granted byCoordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes), process nº 1728008.

REFERÊNCIAS

• ¹
  Maxim L, Niebo R, Utell M. Screening tests: a review with examples. Inhal Toxicol. 2014;26(13):811-28. http://dx.doi.org/10.3109/08958378.2014.955932 PMid:25264934.
  » http://dx.doi.org/10.3109/08958378.2014.955932
• ²
  Herman C, Gill H, Eng J, Farjado L. Fundamentals of clinical research for radiologists. AJR Am J Roentgenol. 2002;179(4):825-31. http://dx.doi.org/10.2214/ajr.179.4.1790825 PMid:12239019.
  » http://dx.doi.org/10.2214/ajr.179.4.1790825
• ³
  Bliss L, Allen D. Screening kit of language development: a preschool language screening instrument. J Commun Disord. 1984;17(2):133-41. http://dx.doi.org/10.1016/0021-9924(84)90019-4 PMid:6725626.
  » http://dx.doi.org/10.1016/0021-9924(84)90019-4
• ⁴
  Wilson J, Jungner G. Principles and practice of screening for disease. J R Coll Gen Pract. 1968;16(4):281-393. PMid:4234760.
• ⁵
  Pernambuco L, Espelt A, Magalhães HV Jr, Lima KC. Recommendations for elaboration, transcultural adaptation and valitation process of tests in speech, hearing and language pathology. CoDAS. 2017;29(3):e20160217. PMid:28614460.
• ⁶
  Padilla J, Benítez I. Validity evidence based on response processes. Psicothema. 2014;26(1):136-44. PMid:24444741.
• ⁷
  Gurgel LG, Kaiser V, Reppold TZ. A busca de evidências de validade no desenvolvimento de instrumentos em Fonoaudiologia: revisão sistemática. Audiol Commun Res. 2015;20(4):371-83. http://dx.doi.org/10.1590/2317-6431-2015-1600
  » http://dx.doi.org/10.1590/2317-6431-2015-1600
• ⁸
  Barbosa A, Soares H, Azoni C. Construção de um instrumento de triagem do vocabulário para crianças entre 3 e 7 anos. Audiol Commun Res. 2019;24:e2131. http://dx.doi.org/10.1590/2317-6431-2019-2131
  » http://dx.doi.org/10.1590/2317-6431-2019-2131
• ⁹
  Fricke S, Bowyer-Crane C, Haley AJ, Hulme C, Snowling MJ. Efficacy of language intervention in the early years. J Child Psychol Psychiatry. 2013;54(3):280-90. http://dx.doi.org/10.1111/jcpp.12010 PMid:23176547.
  » http://dx.doi.org/10.1111/jcpp.12010
• ¹⁰
  Rescorla L. Age 17 language and reading outcomes in late-talking toddlers: support for a dimensional perspective on language delay. J Speech Lang Hear Res. 2009;52(1):16-30. http://dx.doi.org/10.1044/1092-4388(2008/07-0171) PMid:18723598.
  » http://dx.doi.org/10.1044/1092-4388(2008/07-0171)
• ¹¹
  Noronha APP. Análise de testes de personalidade: qualidade do material, das instruções, da documentação e dos itens qualidade de testes de personalidade. Rev Estudos de Psicologia. 2002;19(3):55-65. http://dx.doi.org/10.1590/S0103-166X2002000300006
  » http://dx.doi.org/10.1590/S0103-166X2002000300006
• ¹²
  Cáceres-Assenço A, Ferreira S, Santos A, Befi-Lopes D. Application of a brazilian text of expressive vocabulary in European Portuguese children. CoDAS. 2018;30(2):e20170113. PMid:29791612.
• ¹³
  Luria A. Curso de psicologia geral. 1. ed. Rio de Janeiro: Civilização Brasileira; 1979.
• ¹⁴
  Jacobsen G, Moraes A, Wagner F, Trentini C. Qual é a participação de fatores socioeconômicos na inteligência de crianças. Rev Neuropsicol Latinoam. 2013;5(4):32-8.
• ¹⁵
  Brancalioni A, Zauza A, Karlinski C, Quitaiski L, Thomaz M. Desempenho do vocabulário expressivo de pré-escolares de 4 a 5 anos da rede pública e particular de ensino. Audiol Commun Res. 2018;23(0):e1836. http://dx.doi.org/10.1590/2317-6431-2016-1836
  » http://dx.doi.org/10.1590/2317-6431-2016-1836

Publication Dates

• Publication in this collection
  12 Apr 2021
• Date of issue
  2021

History

• Received
  09 Jan 2020
• Accepted
  09 Mar 2020