Developmental Stuttering Screening Instrument: development and content validation

ABSTRACT

Purpose:

to develop a screening instrument to identify the risk of developmental stuttering in preschoolers.

Methods:

the procedures were divided into two stages: The first one (theoretical), consisted of the literature review and the construction of the items that make up the instrument; the second stage (content validation), consisted of the analysis of the instrument by a judging committee formed by 10 speech-language-hearing therapists who work in the field of fluency. The data analysis was based on the calculation of the item content validity index (I-CVI) and the total number of answers (T-CVI). Also, an item reliability analysis was conducted with Cronbach’s alpha.

Results:

the Developmental Stuttering Screening Instrument (DSSI) encompassed 24 items distributed into four key categories. The I-CVI and T-CVI revealed a high agreement between the judges. The Cronbach’s alpha coefficients indicated a high internal consistency between the judges’ answers in 19 of the 24 items. The qualitative analysis pointed to the need for new adjustments. All the analyses contributed to the construction of the second version of the instrument.

Conclusion:

the proposed instrument showed content-based validity evidence that made it possible, up to the present moment, to adjust it to its construct. It is suggested that the validation process continue, employing new accuracy measures.

Keywords:
Stuttering; Child, Preschool; Mass Screening; Validation Study

RESUMO

Objetivo:

elaborar um instrumento de rastreio com fins de identificação do risco para gagueira do desenvolvimento em crianças pré-escolares.

Métodos:

os procedimentos foram divididos em duas etapas: A primeira etapa - teórica, consistiu na revisão da literatura e a construção dos itens que compõe o instrumento; A segunda etapa - validação de conteúdo, consistiu da análise do instrumento por um comitê de juízes composto por 10 fonoaudiólogas atuantes na área da Fluência. Para análise dos dados foi calculado o índice de validade de conteúdo por item (IVC-I) e pelo total de respostas (IVC-T), bem como realizada uma análise da confiabilidade dos itens por meio do alfa de Cronbach.

Resultados:

o Instrumento de Rastreio para a Gagueira do Desenvolvimento (IRGD) foi composto por 24 itens distribuídos em quatro categorias-chaves. Os índices IVC-I e IVC-T evidenciaram alta concordância entre os juízes. Os coeficientes do alfa de Cronbach indicaram alta consistência interna entre as respostas dos juízes em 19 dos 24 itens. A análise qualitativa destacou a necessidade de novos ajustes. Todas as análises possibilitaram a construção da segunda versão do instrumento.

Conclusão:

o instrumento proposto mostrou evidências de validade baseada no conteúdo que permitiram, até o momento, ajustá-lo em relação ao seu construto. Sugere-se a continuidade do processo de validação por meio do emprego de novas medidas de acurácia.

Descritores:
Gagueira; Pré-Escolar; Programas de Rastreamento; Estudos de Validação

Introduction

Stuttering is a neurodevelopmental fluency disorder that starts in childhood¹. It appears at 33 months old on average when the neural networks that sustain the fluent speech process produce unstable control signals, presenting atypical disfluencies that may be susceptible to the genetic, epigenetic, and environmental influences, characterizing it by a multifactorial, complex etiology². The incidence of stuttering among children younger than five or six years is estimated to be much higher than in later stages of life, ranging from 3% to 17% in early childhood - drawing attention to the period when preschoolers develop the disorder³.

Its primary manifestation in speech consists of the unexpected frequency of stuttering-like disfluencies - especially blocks, prolongations, and repetitions - allowing for a diagnosis that differs it from other language disorders⁴. However, there are also some not so visible characteristics with psychosocial impact, potentially interfering with the person’s quality of life in the long run. The child that starts stuttering early in life may develop social, emotional, and behavioral difficulties, such as having negative feelings towards their speech, decreasing their communicative social participation, and becoming systematically intimidated - i.e., being bullied⁵. Early detection and care may minimize speech impairment and prevent the onset of other complications. Moreover, they can keep stuttering from becoming persistent, in case it is identified soon in the initial manifestations².

The consequences of stuttering worsen as the child grows older. In its initial onset phase, at around two and a half years old, speech fluency recovery can be potentialized to 80%, whereas at seven years old stuttering tends to become chronic². This reflects directly on the clinical prognosis, as the longer a child stutters, the less likely it is for them to spontaneously recover, implying in late intervention, worse prognoses, increased clinical rehabilitation time, chronic stuttering, and fewer chances for them to become fluent adults.

Hence, the recommended professional conduct to treat stuttering is early intervention, which requires early identification⁶. This can be done with a screening instrument encompassing the main risk factors for its development in the target population. Such an instrument must be accessible to education and health professionals that work with preschoolers; when risk is present, the child must be referred for specialized assessment and diagnosis⁷.

The literature commonly points to the need for carrying out screening initiatives so children with risk for stuttering are referred for assessment as early as possible, thus avoiding the loss due to delayed initial care^8-10. Also, early childhood screening helps spread information (e.g., instructions on how to promote fluency), prevents stuttering, and opens the way to follow up the child’s speech⁷. Nevertheless, the clinical instruments in this field, besides their great variety, are mostly aimed at diagnosing the disorder, instead of screening the risk. Moreover, no instrument was found to have been administered exclusively to screen preschoolers¹¹.

To develop an instrument, it is important to respect the psychometric properties that aim to ensure the quality and effectiveness of the results with evidence of validation tests. One of the first validation tests is the content-based evidence, which begins transforming abstract concepts into actual measurable indicators - an essential stage in the process¹².

In this sense, considering the importance of the early identification of stuttering and the shortage of instruments to screen this disorder, this study aimed to develop a screening instrument with content-based validity evidence to identify the risk of developmental stuttering in preschoolers.

Methods

This research, in compliance with resolution no. 466/12 of the Conselho Nacional de Saúde (National Health Council), was approved by the Human Research Ethics Committee of the Universidade Federal de Pernambuco, state of Pernambuco, Brazil, under certificate CAAE: 00329018.1.0000.5208, and evaluation report no. 3.061.209. This study is observational, analytical, cross-sectional research with a quantitative and qualitative approach.

Participants

Ten speech-language-hearing therapists with expertise in the field of fluency participated in the study to form the judging committee. The number of analysts followed what is indicated in the literature¹³.

To select the professionals, the research was announced in digital media, particularly with the support from three entities in the field, namely: the Instituto Brasileiro de Fluência (IBF, Brazilian Fluency Institute), the Associação Brasileira de Gagueira (Abragagueira, Brazilian Stuttering Association), and the Oficina de Fluência (Fluency Workshop). They consented to participate by clicking on the statement “I have read the informed consent form (ICF) and am voluntarily willing to participate”; the ICF had been made available in an online form.

Altogether, 110 speech-language-hearing therapists were invited via e-mail to answer an online professional characterization form structured by the authors to meet the eligibility criteria (Figure 1). Besides the ICF, it contained closed- and open-ended questions, grouped in three categories: questions on their professional training; questions on the clinical assessment of and intervention on stuttering; questions on the theoretical conceptions of child stuttering.

Of the 50 professionals that answered the online form, 22 were selected to form the judging committee - those who reported more than five years of clinical experience in the field of fluency, who understood stuttering as a genetics-based neurodevelopmental disorder, and who guided their clinical practice by a neurolinguistic and motor or multidimensional approach to stuttering. Only 10 of the speech-language-hearing therapists participated in the judging committee, as they returned their evaluations by the deadline. The professionals that were no longer in the profession or the field of fluency were excluded from the selection.

Figure 1:
Methodological flowchart with the judging committee selection process. Recife, Pernambuco, Brazil, 2019

The detailed formation and characterization of the judging committee are shown in Figure 2:

Figure 2:
Characterization of the judging committee. Recife, Pernambuco, Brazil, 2019.

Materials

Developmental Stuttering Screening Instrument (DSSI)

The DSSI was developed to be administered to parents and/or guardians by trained health and education professionals who work with children aged two years to five years and 11 months. It must be administered with a set of questions, aiming to obtain objective answers and subjective considerations regarding the aspects involved in the child’s general development, communication, and speech fluency.

The first version of the DSSI considered that the main risk factors for developmental stuttering involved various dimensions, some of which were more critical and possible to be observed by a layperson. Therefore, the instrument comprised four key categories, namely: category I - General and Communication Development (containing five items); category II - Linguistic Aspects (containing eight items); category III - Motor Aspects of Speech (containing five items); and category IV - Psychosocial Aspects (containing six items). Each item was presented as questions on the risk factors, guiding the assessor in relation to the child’s speech development and behavior regarding their fluency, based on the parents’/guardians’ perception. Altogether, 24 questions were formulated. The risk factors, organized per category, are shown in Table 1.

Table 1:
Item composition and quantity distributed by category of the initial construction of the instrument (first version)

	Category 1 General and Communication Development	Category 2 Linguistic Aspects	Category 3 Motor Aspects	Category 4 Psychosocial Aspects
Composition of the Items (risk factors)	Perinatal complications	Stuttering-like disfluencies	Uncoordinated speech	Temperament
	Language development	Other disfluencies	Strained speech	Perception of stuttering
	Infections	Intelligibility of the discourse	Physical concomitants	Awareness of stuttering
	Complaint	-	Breathing	Child’s response
	Heredity	-	Speed of speech	Social reaction
	-	-	-	Communicative pressure
Total of questions	5	8	5	6

In category I, the possible answers for the questions were defined as “yes”, “no”, and “did not know how to answer”. The other categories were evaluated with the frequency in which the risk factors were presented, with a Likert scale ranging from 1 to 5 - in which number 1 indicates absence, number 5, constant presence, and the other numbers reveal a gradual perception of the frequency (rarely, sometimes, and frequently). The interpretation of the final result is that the higher the total score, the higher the risk of stuttering.

The Likert scale was chosen because of the manner how the disorder is manifested in terms of frequency and intensity. The five-point range of the scale was chosen because it makes the instrument more precise and furnishes more information to measure the construct, evidenced as the point of balance between the degree of adjustment of the model and the precision of the measurement¹⁴. This format has been recognized as the best method to early identify preschoolers at risk of developmental disorder, like autism¹⁵. Moreover, scales have been more often used to assess the severity of stuttering due to their practicality¹¹.

Checklist to Assess the DSSI

International guidelines aligned with the adequate scientific-technical advancements were followed to construct and validate the instrument and obtain evidence of its validity. One of the first validation psychometric tests is the content-based evidence¹⁶. Content validity considers data on the instrument’s content, investigating the set of items and assessing them with the degree of agreement regarding their relevance for and representativity of a given concept¹⁶.

Hence, a checklist was prepared to assess the instrument, encompassing three judging aspects: (1) judgment of the general aspects of the instrument; (2) judgment of the categories that make up the instrument; and (3) judgment of the items that make up the instrument. The first two aspects were guided by the following questions: “In general, how would you evaluate the instrument regarding the following items?”, and “Are the categories indeed adequate to their content?”. The last aspect (judgment of the items that make up the instrument) corresponded to the level of agreement in the makeup of the DSSI items, considering the adapted criteria proposed by Pasqualli (2010)¹⁷: objectivity criterion (OC), simplicity criterion (SC), clarity criterion (CC), relevance criterion (RC), precision criterion (PC), amplitude criterion (AC), modality criterion (MC), and credibility criterion (DC). In addition to these, grammatical criteria were also considered: sentence length (SL), phrasal structure (PS), and vocabulary (V).

The assessment material also had room for comments, notes, and observations, which could be used to justify items given low values in the analysis, suggest changes, or add components they considered relevant but were not included in the instrument.

Procedures

The instrument construction and validation procedure was based on the following phases: (I) investigation of the literature on the theme in question; (II) construction and organization of the items that make up the instrument (first version), described in the preceding section; (III) assessment by the judging committee; (IV) content validation (data analysis); and (V) reformulation of the second version of the instrument (final result).

Phases I and II - Investigation of the literature and construction and organization of the DSSI

The DSSI was developed based on the instrument-developing model suggested by Pasquali (2010). The author proposes that the theoretical, empirical (experimental), and analytical (statistical) stages be methodologically followed when constructing an instrument¹⁷. The present research allowed for the conclusion of the first stage of the model (the theoretical part), which encompasses from the theoretical foundation to the construction of the instrument and its content validation.

An extensive literature review, already described in an integrative review study¹¹, was used as the theoretical basis to plan and develop the first version of the instrument. The BIREME and PubMed databases were accessed to identify how stuttering preschoolers are being assessed and with what instruments. Publications of the last five years in English, Portuguese, and Spanish were selected. “Diagnosis” and “screening” were the descriptors used for the search, both crossed with “stuttering” using AND. Thus, 17 articles were selected for analysis, contributing significantly to the construction of the instrument.

Phase III - Assessment by the judging committee

The 10 selected judges were sent an invitation letter via email with the screening instrument (first version, completed after phases I and II) and a document with guidelines (checklist) to assess its items. The committee was instructed to judge the general aspects, categories, and items that make up the DSSI using the Likert scale to analyze the criteria. The score in the scale ranged from 1 to 5, which represented: 1 - I totally disagree; 2 - I partially disagree; 3 - I neither agree nor disagree, 4 - I partially agree; 5 - I totally agree.

Data analysis (Phase IV - Content validation)

The judgment of both the general aspects of the instrument and its categories were analyzed by calculating the Content Validity Index (CVI) ¹³, which measures the percentage of agreement between the judges concerning the instrument. The CVI was calculated by dividing the number of the judges’ answers given a score of 4 or 5 in the Likert scale by the total number of the judges’ answers assessing the item - i.e., CVI = number of answers with a score of 4 or 5 per item (I-CVI) divided by the total number of answers (T-CVI). The quantitative acceptance indicators between judges must be higher than 0.78 for the general aspects and categories to be considered adequate¹³.

The SPSS statistical software, version 20, was used to individually analyze each category’s items and assess their internal consistency and reliability. These variables refer to the capacity to consistently reproduce a result in time and space¹⁸. To this end, the Cronbach’s alpha (α) reliability and agreement statistical test was applied¹⁹. The minimum acceptable value for the alpha is 0.7, below which the internal consistency is considered low, indicating that the items measure different attributes or the subjects’ answers are inconsistent¹⁹.

Concerning the qualitative analysis, the content analysis technique²⁰ was used to order and organize by item everything the judges had written and suggested and thus find the information’s meaning and understand it in depth. The comments were transcribed into a table developed to verify the details not contained in the assessment material or the justifications for the negative scores of any given item. The comments were coded by central themes as they appeared. These data were interpreted by the consensus of the second group of judges formed by the researchers responsible for the present study. They analyzed whether the suggestions were fitting to be included in the screening instrument.

Results

Content Validation Evidence

The descriptive analysis, based on the measures of central tendency to judge the general aspects of the instrument and its four categories, verified that the mean values for the judges’ answers were 4.63 and 4.74, respectively, as seen in Table 2. The minimum values showed that the general items that were given a lower analysis (being therefore modified) were the following ones: “Title of the Instrument”, “Quality of the items (questions)”, “Clarity of the items (questions)”, and “Sequence in which the categories are presented”. Likewise, in judging the categories, the general analysis variables that were given minimum values were: “Relevance of categories II and III” and “Number of items that make up category II”. However, these minimum values were registered by only one judge. The items were given due attention but did not reflect on the general judgment, which was positive.

Table 2:
Measures of central tendency of the general aspects of the instrument and the general judgment of the categories

Measures of Central Tendency	General aspects of the instrument	Judgment of the categories
Mean	4.63	4.74
Median	5	4.75
Minimum	1	2
Maximum	5	5

Hence, when considering the judgments with scores of 4 and 5, the general aspects of the instrument achieved a CVI = 0.94, while the general judging of the categories achieved a CVI = 0.93. This means a high level of agreement was reached between the judges, as the index was higher than 0.78, indicating that the instrument, in the overall view, met the intended objective.

Assessing separately the content of each category’s items, the lowest and highest I-CVI are observed in Table 3. These values considered the criteria mentioned in Methods: objectivity, simplicity, clarity, relevance, precision, amplitude, modality, and credibility, as well as sentence length, phrasal structure, and vocabulary.

Table 3:
Highest and lowest item content validity index values by category

CRITERIA	CAT1		CAT2		CAT3		CAT4
	Lowest I-CVI	Highest I-CVI	Lowest I-CVI	Highest I-CVI	Lowest I-CVI	Highest I-CVI	Lowest I-CVI	Highest I-CVI
OC	I3. I4- 0.9	I1. I2. I5- 1.0	I4. 17. I8- 0.9	I1. I2. I3. I5. I6- 1.0	I5- 0.9	I1 ao I4- 1.0	I1. I6- 0.9	I2 ao I5- 1.0
SC	I3. I4- 0.7*	I2. I5- 1.0	I1. I4. I6. I8- 0.9	I2. I3. I5. I7- 1.0	-	All of them 1.0	I1- 0.8	I2. I3- 1.0
CC	I4- 0.8	I2. I5- 1.0	I1. I3. I4. I7. I8- 0.9	I2. I5. I6- 1.0	I1. I5- 0.9	I2 ao I4- 1.0	I1- 0.8	I2. I3- 1.0
RC	I4- 0.8	I1. I3. I5- 1.0	I4. I7. I8- 0.9	I1. I2. I3. I5. I6- 1.0	I1 0.89	I2 ao I5- 1.0	I2- 0.9	I1. I3 ao I6- 1.0
PC	I3- 0.7*	I2. I5 1.0	I3. I4. I7. I8- 0.9	I1. I2. I5. I6- 1.0	I1 0.8	I2 ao I4- 1.0	I1- 0.9	I2 ao I6- 1.0
AC	I3- 0.7*	I2. I5- 1.0	I2. I7. I8- 0.9	I1. I3. I4. I5. I6- 1.0	I1. I5- 0.9	I2 ao I4 1.0	I2 0.9	I1. I3 ao I6- 1.0
MC	I3. I4- 0.9	I1. I2. 15- 1.0	I7. I8- 0.8	I1 ao I6- 0.9	-	All of them 1.0	I1. I2. I4- 0.9	I3. I5. I6- 1.0
DC	I3. I4- 0.8	I1. I2. I5- 1.0	I7. I8- 0.9	I1 ao I6- 1.0	-	All of them 1.0	-	All of them 1.0
SL	I1. I3. I4- 0.9	I2. I5- 1.0	I7. I8- 0.9	I1 ao I6- 1.0	-	All of them 1.0	-	All of them 1.0
PS	-	All of them 1.0	I7. I8- 0.8	I1 ao I6- 0.9	I1 0.9	I2 ao I5- 1.0	I2. I6- 0.9	I1. I3 ao I5- 1.0
V	-	All of them 1.0	I7. I8- 0.9	I1 ao I6- 1.0	-	All of them 1.0	I2. I5. I6-v 0.9	I1. I3. I4- 1.0

• Captions: Cat1: Category 1; Cat 2: Category 2; Cat 3: Category 3; Cat 4: Category 4; I-CVI: Item content validity index; OC: objectivity criterion; SC: simplicity criterion; CC: clarity criterion; RC: relevance criterion; PC: precision criterion; AC: amplitude criterion; MC: modality criterion; DC: credibility criterion; SL: sentence length; PS: phrasal structure; V: vocabulary; I1: Item 1; I2: Item 2; I3: Item 3; I4: Item 4; I5: Item 5; I6: Item 6; I7: Item 7; I8: Item 8.

The I-CVI content validation analysis revealed a maximum degree of relevance and representativity in all the items of categories II, III, and IV, indicating that these items’ contents in the instrument properly reflect the construct that is being measured. However, in category I, two items achieved scores lower than 0.78 and needed to be reformulated. One item referred to “language development”, involving the criteria of simplicity, precision, and amplitude. The other item referred to “speech complaint”, involving the criterion of simplicity.

Internal Consistency

The internal consistency was assessed with Cronbach’s alpha to measure the homogeneity of the items of the instrument. The highest and lowest means and the highest and lowest Cronbach’s alpha are seen in Table 4, presented by the items in each category. These values considered the same criteria of the previous table, namely: objectivity, simplicity, clarity, relevance, precision, amplitude, modality, and credibility, as well as sentence length, phrasal structure, and vocabulary.

Table 4:
Highest and lowest mean and Cronbach’s alpha values of the items per category

CRITERIA	CAT1		CAT2		CAT3		CAT4
	Lowest	Highest	Lowest	Highest	Lowest	Highest	Lowest	Highest
OC	I3. I4- 4.7	I5- 5.0	I7. I8- 4.6	I1. I2. I3. I5. I6- 5.0	I5- 4.7	I2. I3. I4- 5.0	I1. I6- 4.6	I3. I4. I5- 5.0
SC	I4- 3.8*	I2. I5- 5.0	I8- 4.5	I2. I3- 4.9	I1. I4- 4.8	I2. I3. I5- 5.0	I1 4.3	I4- 4.9
CC	I4- 4.3	I5- 5.0	I7. I8- 4.5	I2. I6- 5.0	I1- 4.4	I2. I3. I4- 5.0	I1- 4.5	I3- 5.0
RC	I4- 4.4	I1. I3. I5- 5.0	I7. I8- 4.6	I2. I3. I6- 5.0	I1- 4.78	I2 to I5- 5.0	I2- 4.6	I3. to I6- 5.0
PC	I3- 4.2	I2. I5- 5.0	I7. I8- 4.6	I1. I2. I6- 5.0	I1 4.4	I2. I3. I4- 5.0	I1- 4.7	I2. I3. I4- 5.0
AC	I3- 4.2	I2. I5- 4.9	I7- 4.5	I1. I3. I4. I5. I6- 5.0	I5- 4.7	I2. I3. I4- 5.0	I2- 4.7	I3 to I5- 5.0
MC	I4- 4.6	I1. I2. I5- 5.0	I7. I8- 4.3	I1 to I6- 4.7	I1- 4.9	I2 to I5- 5.0	I1- 4.6	I3. I5- 5.0
DC	I4- 4.4	I1. I2. I5- 5.0	I7. I8- 4.6	I1 to I6- 5.0	-	All of them 5.0	I1. I2- 4.9	I3 to I6- 5.0
SL	I4- 4.5	I2. I5- 5.0	I7. I8- 4.6	I1 to I6- 5.0	I1. I4- 4.9	I2. I3. I5- 5.0	I3- 4.8	I4 to I6- 5.0
PS	I1. I4- 4.9	I2. I3. I5- 5.0	I7. I8- 4.2	I1 to I6- 4.7	I1- 4.5	I2 to I5- 5.0	I2. I6- 4.7	I1. I4 5.0
V	I4- 4.9	I1. I2. I3. I5- 5.0	I7. I8- 4.6	I1 to I6- 5.0	I1- 4.9	I2 to I5- 5.0	I2. I5- 4.6	I4 5.0
Alfa Cronbach	I2- 0.44*	I5- 0.95	I5- 0.55*	I7- 0.97	I4- 0.43*	I1. I5- 0.83	I3- 0.65*	I2- 0.90

• Captions: Cat1: Category 1; Cat 2: Category 2; Cat 3: Category 3; Cat 4: Category 4; OC: objectivity criterion; SC: simplicity criterion; CC: clarity criterion; RC: relevance criterion; PC: precision criterion; AC: amplitude criterion; MC: modality criterion; DC: credibility criterion; SL: sentence length; PS: phrasal structure; V: vocabulary; I1: Item 1; I2: Item 2; I3: Item 3; I4: Item 4; I5: Item 5; I6: Item 6; I7: Item 7; I8: Item 8.

As seen in Table 4, the results of the mean values per item demonstrated that almost all the items assessed had means equal to or higher than 4.2, indicating the good acceptance by the judges. Only one item’s mean was lower than 4.2, corresponding to the simplicity of the question on the presence or absence of the child’s speech complaint (item 4 of category I). The judges’ analysis considered that the question in this item, whose mean was 3.8, could lead to a confusing answer.

Cronbach’s alpha, on the other hand, evidenced that, of the 24 items in the instrument, 19 (80%) had high internal consistency, while only five (20%) had low internal consistency between the judges in at least one item of each category. The items on “infections” (category I), “stuttering-like disfluencies - prolongations” (category II), “stuttering-like disfluencies - sound repetition” (category II), “breathing” (category III), and “awareness of stuttering” (category IV) had an α lower than 0.7 and needed to be revised. The descriptive data made it possible to identify in which criteria these items obtained minimum values, which were the relevance (item 1 - category I); modality and phrasal structure (item 5 - category II); simplicity, modality, and phrasal structure (item 1 - category II). No minimum value was lower than 4 for the items on “breathing” and “awareness of stuttering”. It should be pointed out that in two items, both in category III, there was no variance; hence, the analysis of the α was ignored, as there was unanimous agreement between the judges on the items about “speech strain” and presence of “physical concomitants”.

Qualitative Analysis of the Instrument

The judges could also point out their considerations on all the items of each category. All the 10 judges made use of the room provided for them to make their comments on the items of the instrument. Some compliments were expressed - for instance, “succinct, enlightening, and objective material, easy for the parents to access and for the professional to interpret” (judge 7). Nonetheless, the qualitative analysis focused on the suggestions of changes in the instrument. The most important notes that were accepted as a consensus by the second group of judges, formed by the researchers responsible for the present study, are shown below, in Figure 3.

Figure 3:
Transcribed notes of the judging committee classified by theme. Recife, Pernambuco, Brazil, 2020

Most of the comments (six) referred to the need for changes in the vocabulary used in the instrument, to make the language more accessible to the parents and/or guardians, to whom the questions will be submitted. This is so because socioeconomic aspects may influence their understanding of what is being proposed. Other theme classifications were also brought up, such as content (four times), clarity of the items (three times), syntax (once), and phrasal structure (once). All the said aspects were changed in the respective items of the instrument.

Considering that the qualitative assessment revealed important aspects, especially on the vocabulary, it was decided to provide further details of the quantitative analysis regarding this aspect. Thus, Table 5 presents the means of the answers of all the judges regarding the vocabulary in the judgment of all the items in each category.

Table 5:
Means of the judges’ answers for the judgment of the vocabulary per item in each category

	Category I	Category II	Category III	Category IV
Item 1	5.0	5.0	4.9	4.9
Item 2	5.0	5.0	5.0	4.6
Item 3	5.0	5.0	5.0	4.9
Item 4	4.9	5.0	5.0	5.0
Item 5	5.0	5.0	5.0	4.6
Item 6	-	5.0	-	4.7
Item 7	-	4.6	-	-
Item 8	-	4.6	-	-

• Captions: Item 1: perinatal complications (category I), sounds repetition (category II), motor incoordination of speech (category III), temperament (category IV); Item 2: infections (category I), syllable repetition (category II), speech strain (category III), perception of stuttering (category IV); Item 3: language development (category I), monosyllabic word repetition (category II), physical concomitants (category III), awareness of stuttering (category IV); Item 4: complaint (category I), non-monosyllabic word repetition (category II), breathing (category III), response to stuttering (category IV); Item 5: heredity (category I), prolongations (category II), speed of speech (category III), social reaction (category IV); Item 6: block (category II), communicative pressure (category IV); Items 7 and 8: intelligibility of the discourse (category II).

The results obtained from calculating the means of the judges’ answers regarding the vocabulary employed in each category revealed that all the categories achieved a mean higher than 4.6. This means that there was a positive assessment of the vocabulary, despite the comments suggesting possibly necessary changes. The lowest means are those of the vocabulary in the questions on “intelligibility of the discourse” (category II), “awareness of stuttering (category III), and “social reaction” (category III). All these items were given due attention, being adjusted accordingly.

The makeup of the instrument in relation to the risk factors assessed by categories after being judged by the judging committee is seen in Table 6.

Table 6:
Item composition and quantity distributed by category in the second version of the instrument (post-assessment)

	Category 1 General and Communication Development	Category 2 Linguistic Aspects	Category 3 Motor Aspects of Speech	Category 4 Psychosocial Aspects
Composition of the Items (risk factors)	Heredity (reallocated within the category)	Stuttering-like disfluencies (more examples added)	Uncoordinated speech (more examples added)	Temperament (reformulated as “Child’s response”)
	Peri- and postnatal complications (reallocated within the category and reformulated sentence)	Other disfluencies	Strained speech (reallocated within the category and more examples added)	Perception of stuttering (reformulated sentence)
	Language development (reformulated sentence)	Intelligibility of the discourse (reformulated sentences)	Physical concomitants	Awareness of stuttering (reformulated sentence and more examples added)
	Complaint (reallocated to the “Identification” sector)	-	Breathing (reformulated sentence and more examples added)	Child’s response (reformulated as “Child’s attitude”)
	Infections (excluded)	-	Speed of speech (reformulated sentence)	Social Reaction
	Allergies and sleep (added)	-	-	Communicative Pressure (reformulated sentence)
Total of questions	5	8	5	6

Reformulation - Second Version of the Instrument

After the analysis and suggestions of the judging committee and the consensus of the research group, some changes were made in the questions of the instrument as initially proposed. The number of items remained the same (24); however, some items were excluded, while others were added, reallocated, or reformulated. Also, additional examples were given to make the questions in the categories easier to understand. The changes resulted in the second version of the instrument, which is available in Appendix A.

Discussion

In the field of speech-language-hearing, there has been a tendency in recent years to construct and improve clinical instruments. Nevertheless, most of these instruments deal with the assessment, diagnosis, and monitoring process of the therapeutic intervention. Few studies have sought to identify the risk of a communication disorder²¹. It was verified that, in Brazil, the psychometric principles are scarcely or partially applied in the speech-language-hearing instruments^21,22. Both the diagnosis and the screening tests lack the validation for the populations to which they are intended. Moreover, the methodological path needs to be systematized to generate valid and reliable interpretations of their results²². National and international studies highlight the importance of early identification screening initiatives leading to a therapeutic prognosis of developmental stuttering^23-25. Even so, no validated instruments with such a purpose were found.

To obtain psychometric evidence, one of the first procedures is that which critically assesses whether the components of the instrument are related to the attributes that need to be measured - i.e., content validation¹². Concerning this study’s content validity, the judges’ analysis with the CVI calculation revealed a positive degree of agreement (> 0.8) in most of the items, which proved to accurately reflect the construct that is being measured. In other words, the sample of items was representative of the content of the categories, according to the judgment of the professionals with experience in the field^13,18. A Screening Tool for Oropharyngeal Dysphagia in Stroke (RADAVE)²⁶ also followed the international guidelines, beginning the psychometric analysis with the content-based validity evidence. The judging committee noticed the need for changes in RADAVE after realizing the minimum I-CVI had not been achieved in some items, despite the strong overall agreement. The same occurred with the present study, as the I-CVI in two items of category I was below the ideal value, while the general analysis of the categories was higher than 0.9.

Regarding the reliability validity evidence, the internal consistency analyzed with Cronbach’s alpha coefficient proved to be adequate (> 0.7). Only five items presented low homogeneity, and these were reformulated. This indicates satisfactory reliability of the items that make up the present instrument to assess the risk of developmental stuttering, as it assesses a single construct through a variety of items (24), which is considered appropriate when applying internal consistency measures¹⁸. Other national instruments that assess aspects of language also analyzed the internal consistency with the Cronbach’s alpha coefficient. The internal consistency of INFONO²⁷, a phonological assessment instrument, was analyzed regarding the precision of its scores. It obtained a median of 0.8, demonstrating strong trustworthiness evidence. The IRC-36, a screening instrument to identify communication development milestones in 0- to 36-month-old children²⁸, also achieved results that indicated high internal reliability (0.9) for the scores obtained.

The Protocol of Risk of Developmental Stuttering (PRGD)²⁹, which assesses the degree of risk of stuttering as low, medium, and high, is currently the most used clinical instrument in Brazilian research on child stuttering^30,31. Nevertheless, it has not yet provided content and trustworthiness validity evidence. Its construction underwent the theoretical stage of literature review and structuring, then moving on to the empirical stage, administering the instrument to define the scores²⁹. In other words, there was no psychometric test analysis, which is recommended when developing instruments. Consequently, it is impossible to make comparisons with the instrument in the present study. Therefore, this study is relevant to strengthen the field of fluency in the country, as it contributed to the development of a screening instrument that was submitted to rigorous validation and trustworthiness methodological stages so it could be used to identify children at risk of developmental stuttering.

In the present study, the assessment by a committee formed by speech-language-hearing therapists trained and/or experienced in the field enabled the first version of the instrument to be analyzed, and the second one to be proposed. Based on the mean values, I-CVI, Cronbach’s alpha, and the judges’ qualitative comments/observations, some changes were made, such as excluding, adding, reallocating, and reformulating items, especially regarding content and vocabulary.

Some highlighted changes made in content include the risk factors in category I (general and communication development). Recent studies reveal that atopic diseases - e.g., allergic rhinosinusitis - are associated with neurodevelopmental disorders, such as stuttering³². It was also identified that stuttering children are more likely to have insomnia, difficulties to fall asleep, somnolence, and fatigue throughout the day - symptoms that persist from early childhood to adolescence³³. Thus, based on the judges’ suggestions, two questions were added in category 1, one related to the presence of allergies and the other, sleep problems.

Further on category I, it was decided to exclude the question on the presence of infection because, if it were confirmed when the disfluencies first manifested, it would not be developmental stuttering. Instead, this stuttering would be due to an autoimmune response, caused by the tonsillitis type A Streptococcus (PANDAS syndrome)³⁴, the H. pylori bacteria³⁵, or the Rotavirus³⁶. This aspect needs to be investigated by a specialized speech-language-hearing therapist to reach a differential diagnosis of fluency disorders. The question about the concern with the child’s speech obtained a low score in the mean and I-CVI. Hence, it was decided not to dismiss it but reallocate it, as the complaint is a risk factor for stuttering². This item was transferred to the part of the instrument with identification data, and it is suggested that it have a weighted score in the final result.

One of the things to be attentive to when developing an instrument is its cultural adequacy³⁷. The qualitative analysis of the present instrument made it possible to explore aspects important to vocabulary changes in some items of each category. The Cronbach’s alpha coefficients also corroborated this aspect, as the items with low internal consistency were mostly related to simplicity and phrasal structure. Considering that these grammatical criteria ensure accessibility, respecting the different regional and socioeconomic standards, and making the content easier to understand¹⁷, the suggestions to replace certain words and terminologies were accepted. The Compasso protocol³⁷, which aims to promote adherence to self-care practices in diabetes, was likewise submitted to the stage of cultural adequacy when it was being constructed, with the participation of specialists, which allowed for the development of a more functional protocol to the target population. Hence, it is expected that the reformulations to which the instrument of the present study was subjected will bring it closer to local vocabulary, adapting sentences, and replacing technical terms with simpler words. These are meant to make the items clearer in conformity with the characteristics of the target culture³⁷. This criterion must be better investigated in future validation processes.

Despite the interest in proposing an instrument to screen developmental stuttering with content validity psychometric evidence, the present study has some limitations inherent to the very instrument-constructing process, as it is still in the early phase of the overall validation process. Content validity evidence is essential but not sufficient. Further studies in the continuity of this research must present the differentiation sensitivity analysis of the target population with the threshold risk values of developmental stuttering. They must also show other validity evidence, such as that of criterion, the concurrent, the convergent, and so forth¹⁸.

The development of this instrument has important clinical implications for the early identification of stuttering in the country - it is the first standardized instrument with content-based validity evidence. It was developed to be administered by any education and health professional trained to screen the risk of developmental stuttering in preschoolers. When the result comes out positive for risk, they must refer such children for specific speech fluency assessment, which must be performed by a specialized speech-language-hearing therapist.

Conclusion

A screening instrument was developed to identify the risk of developmental stuttering in preschoolers with content-based validity evidence. The validity evidence studied up to this moment allowed for relevant changes and the proposal of the second version of the instrument with a more adequate construct. It is recommended that the psychometric analyses be continued, employing new accuracy measures. The instrument proposed in this study is expected to promote the practice of preventive speech-language-hearing health in the early identification of stuttering, as a starting point for specific interventions in child communication health.

Acknowledgments

Gratitude is extended to FACEPE - Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco. Also, this study was financed in part by the CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil - Finance Code 001.

REFERENCES

• ¹
  Micai M, Fulceri F, Caruso A, Guzzetta A, Gila L, Luisa M. Early behavioral markers for neurodevelopmental disorders in the first 3 years of life: an overview of systematic reviews. Neurosci Biobehav Rev. 2020;116:183-201.
• ²
  Smith A, Weber C. How stuttering develops: the multifactorial dynamic pathways theory. J Speech Lang Hear Res. 2017;60(9):2483-505.
• ³
  Kefalianos E, Onslow M, Packman A, Vogel A, Pezic A, Mensah F et al. The history of stuttering by 7 years of age: follow-up of a prospective community cohort. J Speech Lang Hear Res. 2017;60(10):1-12.
• ⁴
  Alencar PBA, Palharini TA, Silva LM, Oliveira CMC, Berti LC. Indicators of speech fluency in stuttering and in phonological disorder. CoDAS. 2020;32(2):e20190002.
• ⁵
  McAllister J. Behavioural, emotional and social development of children who stutter. J Fluency Disord. 2016;50:23-32.
• ⁶
  Sander RW, College M, Wisconsin W. Stuttering: understanding and treating a common disability. Am Fam Physician. 2019;100(9):556-60.
• ⁷
  ASHA: American Speech-Language-Hearing Association. Preferred practice patterns for the profession of speech-language pathology. Speech-Language Screening-Children. Rockville: American Speech-Language-Hearing; 2004.
• ⁸
  Mohamadi O, Rahimi-Madiseh M, Sedehi M. The prevalence of stuttering, voice disorders, and speech sound disorders in preeschoolers in Shahrekord Iran. Int J Child Youth Family Stud. 2016;7(3-4):456-71.
• ⁹
  Thapa KB, Okalidou A, Anastasiadou S. Teachers' screening estimations of speech-language impairments in primary school children in Nepal. Int J Lang Commun Disord. 2016;51(3):310-27.
• ¹⁰
  Shimada M, Toyomura A, Fujii T, Minami T. Children who stutter at 3 years of age: A community-based study. J Fluency Disord. 2018;56:45-54.
• ¹¹
  Lima MMO, Cordeiro AAA, Queiroga BAM. Instruments of early identification of stuttering in preschool children: an integrative literature review. CoDAS. No prelo 2020.
• ¹²
  Echevarría-Guanilo ME, Gonçalves N, Romanoski PJ. Psychometric properties of measurement instruments: conceptual basis and evaluation methods - Part II. Texto Contexto Enferm. 2019;28:e20170311
• ¹³
  Coluci MZO, Alexandre MMC, Milani D. Construction of measurement instruments in the area of health. Cien Saude Colet. 2015;20(3):925-36.
• ¹⁴
  Aval FJP, Auné SE, Attorresi HF. Variation in Likert scale of the mathematics usefulness test. Interacciones. 2018;4(3):177-89.
• ¹⁵
  SBP: Sociedade Brasileira de Pediatria. Manual de Orientação: Transtorno do Espectro do Autismo. 2019;00(5):1-24.
• ¹⁶
  Andrade JM, Valentini F. Guidelines for the construction of psychological tests : regulation CFP No: 009/2018 in Highlight. Psicol. cienc. prof. 2018;38(spe):28-39.
• ¹⁷
  Pasquali L. Instrumentação psicológica: fundamentos e práticas. 1.ed. Porto Alegre: Artmed, 2010.
• ¹⁸
  Souza AC, Alexandre NMC, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol. Serv. Saude. 2017;26(3):649-59.
• ¹⁹
  Cunha CM, Almeida Neto OP, Stackfleth R. Main psychometric evaluation methods of measuring instruments reliability. Rev Atenção Saúde. 2016;14(49):98-103.
• ²⁰
  Vosgerau DSR, Pocrifka DH, Simonian M. Association between the technical analysis of the content and the cycles of codification: possibilities from Atlas.ti software. RISTI. 2016;19(09):93-106.
• ²¹
  Gurgel LG, Kaiser V, Reppold TZ. The search for validity evidence in the development of instruments in speech therapy: a systematic review. Audiol. Commun. Res. 2015;20(4):371-83.
• ²²
  Goulart BNG, Chiari BM. Screening versus diagnostic tests: an update in the speech, language and hearing pathology practice. Pró-Fono R. Atual. Cient. 2007;19(2):223-32.
• ²³
  Ward D. Risk factors and stuttering: evaluating the evidence for clinicians. J Fluency Disord. 2013;38(2):134-40.
• ²⁴
  Howell P. Screening school-aged children for risk of stuttering. J Fluency Disord. 2013;38(2):102-23.
• ²⁵
  Oliveira CMC, Souza HA, Santos AC, Cunha D, Giacheti CM. Fatores de risco na gagueira desenvolvimental familial e isolada. Rev. CEFAC. 2011;13(2):205-13.
• ²⁶
  Almeida TM, Cola PC, Pernambuco LA, Magalhães Junior HV, Magnoni CD, Silva RG. Screening tool for oropharyngeal dysphagia in stroke - Part I: evidence of validity based on the content and response processes. CoDAS. 2017;29(4):e20170009.
• ²⁷
  Ceron MI, Gubiani MB, Oliveira CR, Keske-Soares M. Evidence of validity and reliability of a phonological assessment tool. CoDAS. 2018;30(3):e20170180.
• ²⁸
  Queiroga CAM, Queiroga BAM, Cordeiro AAA. Communication screening instrument for children from 0 to 36 months (IRC-36): preparation and evidence of contente validation. CoDAS. No prelo 2020.
• ²⁹
  Andrade CRF. Gagueira Infantil: risco, diagnóstico e programas terapêuticos. Barueri, São Paulo: Pró-fono, 2012.
• ³⁰
  Oliveira CMC, Cunha D, Santos AC. Risk factors for stuttering in disfluent children with familial recurrence. Audiol. Commun. Res. 2013;18(1):43-9.
• ³¹
  Oliveira CMC, Nogueira PR. Prevalence of risk factors for stuttering among boys: analytical cross-sectional study. Sao Paulo Med J. 2014;132(3):152-7.
• ³²
  Ajdacic-Gross V, Rodgers S, Müller M, Känel R, Seifritz E, Castelao E et al. Hay fever is associated with prevalence, age of onset and persistence of stuttering. Adv Neurodev Disord. 2020;4(1):67-73.
• ³³
  Merlo S, Briley PM. Sleep problems in children who stutter: evidence from population data. J Commun Disord. 2019;82:1-11.
• ³⁴
  Maguire GA, Nguyen DL, Simonson KC, Kurz TL. The Pharmacologic treatment of stuttering and its neuropharmacologic basis. Front Neurosci. 2020;14:1-8.
• ³⁵
  Tsao JW, Shad JA, Faillace WJ. Tremor, aphasia, and stuttering associated with helicobacter pylori infection [letter]. Am J Med. 2004;116(3):211-2.
• ³⁶
  Theys C, Wieringen AV, Tuyls L, Nil LD. Acquired stuttering in a 16-year-old boy. J Neurolinguistics. 2009;22(5):427-35.
• ³⁷
  Fernandes BS, Reis IA, Pagano AS, Ceciclio SG, Torres HC. Development, validation and cultural adaptation of the Compasso protocol: Adherence to self-care in diabetes. Acta Paul Enferm. 2016;29(4):421-9.

• Research support source: Fundação de Amparo à Ciência e Tecnologia de Pernambuco - FACEPE and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil - CAPES.

Publication Dates

• Publication in this collection
  04 Dec 2020
• Date of issue
  2021

History

• Received
  12 Aug 2020
• Accepted
  20 Oct 2020