Internal consistency and latent factors in the variability of syntactic competence in elementary school students

ABSTRACT

Purpose

Present a study on the internal consistency and latent factors that promote response variability in an assessment task of syntactic competence applied to Elementary School students.

Methods

An oral sentence complementation task containing 30 main adverbial subordinate clauses was prepared. Participants were 113 students with typical development (55.7% girls), aged 9 years to 11 years and 1 month (M=9.5; SD=0.6), regularly enrolled in the 4^th (59.3%) and 5^th (40.7%) grades of a public Elementary School. The schoolchildren were evaluated with respect to auditory discrimination and processing (simplified assessment), phonological short-term and working memory, expressive vocabulary, and oral sentence complementation. Data were analyzed using the Fleiss’ Kappa (α=0.6) and Cronbach's Alpha (α=0.4) coefficients. Phi coefficient (α=0.7) factor analysis with the principal components method was also performed.

Results

Inter-rater concordance between the three judges was α=0.61<x>1 for six clauses. Concordance between the two speech-language pathologists was perfect for two clauses and k=0.69<x>0.89 for 23 clauses. In the internal consistency analysis, the condition and purpose subordinate clause categories showed α=0.43 and α=0.47, respectively. Of the 13 latent factors identified, only one (12%) promoted total response variance.

Conclusion

Inter-rater concordance between the speech-language pathologists was significant, task items were not correlated within the same grammatical category, and the use of the subjunctive mood in the complementation response to the main clause seems to be a latent factor promoting the response variability of students.

Keywords:
Language; Linguistics; Speech, Language and Hearing Sciences; Validation studies; Statistical analysis

RESUMO

Objetivo

Apresentar o estudo de consistência interna e de fatores latentes responsáveis pela variabilidade de respostas a uma tarefa de avaliação de competências sintáticas, aplicada em crianças escolares do 2º ciclo do ensino fundamental I.

Métodos

Elaborou-se uma prova de complementação oral de sentenças, a partir de 30 orações subordinadas adverbiais. Participaram 113 escolares típicos (55,7% sexo feminino), entre 9 anos e 11 anos e 1 mês (média de idade = 9,5; desvio padrão = 0,6), matriculados no 4º (59,3%) e 5º (40,7%) anos do ensino fundamental da rede pública. Foram avaliados quanto à discriminação e processamento auditivo, memória fonológica, vocabulário expressivo e complementação oral de sentenças. Três juízes (dois fonoaudiólogos e um linguista) avaliaram 3390 respostas. Analisou-se por coeficiente Kappa de Fleiss (α=0,6), coeficiente de Alpha de Cronbach (α=0,4) e realizou-se análise fatorial com coeficiente Phi (α=0,7), com método de componentes principais.

Resultados

A concordância entre os três juízes mostrou k=0,61<x>1 para 6 itens. A análise dos dois fonoaudiólogos mostrou concordância perfeita para 2 itens e k=0,69<x>0,89 para 23 itens. Na análise da consistência interna, as categorias condicional e final mostraram α=0,43 e α=0,47, respectivamente. As demais categorias não mostraram consistência. Dos 13 fatores latentes identificados, apenas 1 mostrou ser responsável (12%) pela variância total de respostas.

Conclusão

A concordância entre juízes fonoaudiólogos foi substancial, os itens de teste não se mostraram correlacionados dentro da mesma categoria gramatical. O uso do modo subjuntivo na resposta complementar à oração principal parece ser um fator latente, condutor da variabilidade de resposta dos escolares.

Palavras-chave:
Linguagem; Linguística; Fonoaudiologia; Estudos de validação; Análise estatística

INTRODUCTION

Factors predictive of school performance, as well as punctual determinants of reading and writing performance, have been considerably investigated in the fields of Cognitive Psychology, Psycholinguistics, and Speech-language Pathology (SLP) through the study and design of cognitive processes involved in learning^(1-5). Research has increasingly focused on the investigation of cognitive functions and measures of mechanisms and structures of oral language that can correlate with the skills required for reading comprehension and writing coherence and cohesion^(6,7).

Reading comprehension requires coordination of different cognitive and language levels. Reading can be understood as a complex activity that involves the operation of basic and automatic mechanisms used in the processing of linguistic information and metalinguistic skills, which allow reflection on what has been read^(8,9). Lexical retrieval and syntactic parsing⁽¹⁰⁾ are among the first cognitive processes (considered as of lower order) included in the information conveyed in the text, leading to the creation of mental representation of the situation model presented. In order to understand it, the reader initially needs to identify the syntactic and/or semantic structures among the textual constituents so that a sense unit can be constructed⁽¹¹⁾.

Therefore, good syntactic information processing skills are essential for the processing of reading comprehension. Researchers have warned that little is known about the grammatical skills, especially morphosyntactic abilities, of readers who present difficulty understanding what they read. According to some authors, common sense suggests that these difficulties are determined by deficits in semantics or in the high-order cognitive mechanisms of oral language⁽¹²⁾. In fact, the literature indicates that text information integration mechanisms, necessary to connect words within sentences and sentences within paragraphs, are directly associated with the correct use of closed class words⁽¹³⁾.

Thus, for readers to meet the expected cognitive demand involved in these mechanisms so that they can understand the text, skills that certainly relate to linguistic competence and grammatical structures are needed. Linguistic skills of syntactic resolution and combination of words juxtaposed in a sentence and sentences in a text, during reading, are examples of tools necessary for the sedimentation of grammatical structures, analysis, and reading comprehension⁽¹⁴⁾.

Although the importance of these language automatisms in the use of grammar and of the relevance of syntactic competence is recognized for the development of written language^(8,10,12,15), there are few instruments, especially of clinical nature, capable of evaluating Brazilian students whose syntactic competence performance has not yet been standardized. This performance has been assessed through the analysis of spontaneous speech samples, imitation, and induced production, whose tasks seek to verify the frequency of the types of syntactic constructions used, complexity of nominal and verbal phrases, sentences with syntactic errors or omissions, internal structure of sentences (coordination or subordination), and quantitative verification of the use of formal classes of words: agreement and gender, number and grade markers^(6,14,16,17).

The Oral Sentence Complementation Task, a sub-test of the TIPITI Language Evaluation Exam⁽¹⁶⁾, which aims to clinically assess syntactic competence according to age group, ranging from 3 to 15 years (being also referred to evaluate older individuals), has not yet been analyzed psychometrically and, therefore, lacks standardized/normalized scores. In contrast, although the Syntactic Awareness Test has been previously standardized and validated⁽¹⁷⁾, it is intended to evaluate syntax at the metalinguistic level. An increased number of adaptations and cross-cultural translations of foreign tests, organized in languages with structures divergent from Brazilian Portuguese, have been observed in the Brazilian academic context⁽¹⁸⁾. One of the advantages of these newly investigated tests is the possibility of measuring receptive and expressive aspects of language so that possible discrepancies between them can be identified, which optimizes the definition of clinical diagnosis of oral and written language disorders^(6,19).

Reading is a form of communication: when the reader understands what the author meant, they both have performed their sociocultural roles satisfactorily. SLP, attentive to the communicative, educational and social demands of human beings, should include the treatment of communicative disorders in its practice. SLP should make use of instruments with diagnostic accuracy to favor clinical interventions and good clinical and educational outcomes, also when reading comprehension was not achieved.

In this context, given the recognized importance of syntax for written comprehension, this study intends to contribute, by means of a proposal to construct a syntactic competence assessment task, to the improvement of scientific research and clinical SLP evaluation on issues associated with syntax. To this end, main adverbial subordinate clauses were prepared to be complemented by 113 students with typical development enrolled in the 4^th and 5^th grades of Elementary School.

It was assumed that language manifestations can be observed and analyzed with regards to performance and competence, despite the fact that expressive communication does not always correspond to the level of representational organization^(7,14), and that, in this age group, schoolchildren are expected to have sufficient knowledge of grammatical structures to support the learning through textual genres, in narratives and fables, required by the Brazilian National Curriculum Parameters for the 4^th and 5^th grades of Elementary School. Moreover, the present study was also based on the hypothesis that this type of evaluation will determine a wide variety of syntactic structures (such as agrammatism, realization of referential dependencies - sentence parsing) because of the linguistic competence of the participants⁽¹²⁾.

Therefore, in an attempt to reduce the already expected noisy diversity for analysis and measurement, this study predicted the structured use of specific linguistic categories. Thus, structure of the task items was delimited, and a choice was made for adverbial subordinate clauses that enabled positioning the main clause at the beginning of the sentence, according to the categorization researched⁽²⁰⁾.

Analysis of the consecutive oral complementation of the main clauses intended to collect occurrences^(21,22) that would enable a validation study of the proposed task by analyzing the internal consistency of the suggested items. Therefore, this study aimed to present an evaluation of the internal consistency and latent factors that promote response variability in a syntactic competence assessment task applied to Elementary School students.

METHODS

This prospective, cross-sectional, quantitative study was approved by the Research Ethics Committee of the Federal University of Sao Paulo – CEP/UNIFESP (CAAE: 62553916.8.0000.5505). The evaluations began after the public school selected (Basic Education Development Index (IDEB) 2015=6.24) authorized the data collection and signed an Institutional Consent Form. Informed Consent (ICF) and Child Assent (CAF) Forms were also signed by all participants voluntarily involved in the survey.

Sample selection

The initial inclusion criteria comprised regular enrollment in the 4^th or 5^th grades of Elementary School; absence of grade repetition, related complaints or indicators (uncorrected) of hearing and visual impairments, and neurological, behavioral or cognitive disorders; signing of an ICF by the parents and/or legal guardians and of a CAF by the participating schoolchildren. Observance of these criteria was possible with the support of the school, teachers, and parents.

Two hundred five schoolchildren complied with these inclusion criteria. The following tests were used to assess all study participants: Simplified Auditory Processing Assessment⁽²³⁾; Auditory Discrimination Test⁽²⁴⁾; Child Expressive Vocabulary Test⁽²⁵⁾; Phonological Working Memory: Digit Span of the Wechsler Intelligence Scale for Children – 3^rd edition (WISC-III)⁽²⁶⁾. The tests used to select the study sample were aimed at evaluating skills considered correlated to the task of oral sentence complementation^(2,3,7,19).

Evaluations occurred in the school premises at times previously arranged with the school teachers, coordination, and board. The 30-min tasks were individually applied in a quiet room. For the Auditory Discrimination Test, a computer was placed in front of the child, at a distance of 50 cm, and the examiner remained seated next to it, whereas for the Simplified Auditory Processing Assessment, the instruments were placed on a foam-lined table and the child remained standing.

Even after being referred by their parents and teachers, based on the inclusion criteria, 92 students of the initial sample of 205 were excluded because they showed deficits in one or more of the applied assessments: vocabulary below average for age in the Child Expressive Vocabulary Test; lower rank in either the forward or backward Digit Span test; absence of cochleopalpebral reflex in the Simplified Auditory Processing Assessment.

Thus, the final sample was composed of 113 schoolchildren (55.7% girls), aged 9 years to 11 years and 1 month (mean=9.5; standard deviation (SD)=0.6; minimum=9; maximum=11), regularly enrolled in the 4^th (59.3%) and 5^th (40.7%) grades of Elementary School.

Procedures for the preparation of the oral sentence complementation task

For the preparation of the sentences (main clauses), the coordination context was discarded, as it presents grammatically equivalent combined units without reciprocal determination. The subordination context is differentiated due to possibility of resumption or anticipation of the grammatical stratum. Thus, adverbial subordinate clauses were selected because they can more clearly emphasize their dependence on adequate connection, so that one clause determines the other^(20,27). The task of oral complementation of spoken sentences was conducted based on the proposal of the TIPITI Language Evaluation Exam⁽¹⁶⁾. With the help of a linguist, 30 main clauses were prepared, which differed according to the syntactic demand of the grammatical structure strata of Portuguese, considering Brazilian Portuguese^(20,27), selected from the following types of subordinate adverbial clauses of reason, purpose (consequence and result), condition, concession, proportion, and time. The following inclusion criteria were applied to compose the items: (a) Main clause at the beginning of the sentence to elicit the student's response; (b) subordinate adverbial clauses that would enable occurrence of the main clause at the beginning of the sentence; (c) clause with no reversibility effect; (d) vocabulary appropriate to the students’ age; (e) varied sentence length.

We selected seven categories of subordinate adverbial clauses that would enable the main clause to be placed at the beginning of the sentence spoken by the examiner. For each category, we sought to obtain four or five more commonly used conjunctions in Brazilian Portuguese (Chart 1).

Chart 1
Subordinate adverbial clauses: related categories and conjunctions

CATEGORIES	CONJUNCTIONS
Reason	Because	That	Given that	Since	As
Purpose (consequence)	So that	Such that	Lest	In order that	That
Condition	If	Unless	Provided (that)	As long as	Whether
Purpose (result)	Such that	So that	In order to	For	--
Concession	(Al)though	Despite	Even though	While	In spite of
Proportion	As	To the extent	The … the (correlative)	As … so	--
Time	When	While	Before	After	As soon as

A training sentence “Eu caí porque havia um buraco” (“I fell because there was a hole”) and 30 items including seven interspersed categories were prepared in order to avoid generalization of a single circumstantial aspect in response production.

A pilot study was conducted with 10 children (both genders), aged 9-11 years, selected voluntarily in SLP clinics in Sao Paulo, with or without oral language disorder complaints. All children signed a CAF and their parents and/or legal guardians signed an ICF prior to study commencement. The main purpose of the pilot study was to verify whether the task items would elicit expected responses for each main clause and be subject to analysis.

Assessment procedures

Students were assessed individually in the school's computer room, which was made available by the school board for the application of the tasks because it is a quiet environment. The examiner sat down in front of the student, set the computer on a table to the left, and read the task instructions.

The schoolchildren were told to complete the main (initial) clause as accurately and quickly as possible, without repeating the clause. Aiming to ensure homogeneity in the task application, both the instructions and the training and task items were applied by a single examiner. After reading the instructions to each task, the examiner verified whether the students had understood them by requesting that they explain, in their words, what they were supposed to do.

If the student did not understand (even if partially) the instructions, they were resumed and, after that, the training items were presented and the responses recorded on a SONY ICD-PX240 recorder positioned 10 cm from the student's mouth. The responses were recorded, canonically transcribed, and later analyzed by a panel composed of three judges (two speech-language therapists and one linguist). All judges were given a manual with explanatory parameters on oral productivity, so that they used the same response analysis criterion for the task items aiming to verify whether the syntactic resolution expressed by the student was adequate to the stimulus provided. Owing to the oral nature of the task, and with the objective of measuring linguistic competence, it was deemed essential not to consider the presence of some orality markers in the students’ speech, such as reductions, hesitations, omissions, and repetitions, according to the Oral Productivity Analysis Parameter, which guided the panelists on their analyses of the responses to the initial clauses (main clauses), without grammatically indicating the subordinate categories used for each clause. The panelists analyzed the 3,390 clauses produced by the schoolchildren.

Statistical analysis

The study sample was characterized with regards to age, gender, schooling, and performance in the forward and backward Digit Span and Expressive Vocabulary tests. Application of the Kolmogorov-Smirnov and Shapiro-Wilk tests showed that the sample distribution was not normal.

Interrater concordance analysis between the judges was performed from the distribution of frequencies and joint and marginal percentages of their evaluations, two-paired, observing the values of the Cohen’s Kappa coefficient for the concordance rate for each pair of judges, and the value of the Fleiss’ generalized Kappa coefficient for the concordance rate between the three judges (two speech-language therapists - one audiologist and one linguist), for clauses 1 to 30 (a total of 30 task clauses). Internal consistency analysis of the items in each category of subordinate adverbial clauses of reason, purpose (consequence and result), condition, concession, proportion, and time was conducted using the Cronbach’s Alpha coefficient, through intra-item covariance study and in the proposed categories. The Kuder-Richardson Formula 20⁽²¹⁾ coefficient was calculated for the binary data. For this analysis, the responses of the speech-language pathologist with 20 years of clinical experience were chosen, because the instrument was designed for application in language practice.

In order to identify the latent factors that promote response variability in the 30 items (main clauses)⁽²²⁾, factor analysis was conducted, because it is a size-reduction technique, to verify the possibility of exclusion of some clauses and observe how all the items were grouped without the categories of subordinate adverbial clauses proposed. The Phi coefficient⁽²⁸⁾ was considered as a measure of correlation between the items. The principal components method was adopted.

Statistical analyses were processed in Minitab 17, SPSS 18, and R 3.2.0 software.

RESULTS

Interrater concordance analysis between the judges showed the distribution of frequencies and joint and marginal percentages of their evaluations, two-paired, from the values of the Fleiss’ Kappa coefficient (α=0.6) and concordance rate for each pair of judges, and the value of the Fleiss’ generalized Kappa coefficient for the concordance rate between the three judges in clauses 1 to 30. The Kappa coefficient values observed in the concordance analysis between judges 1 and 3, both speech-language therapists, were higher than those observed in the analysis between judges 1 and 2 and 2 and 3, speech-language therapist and linguist.

Concordance between the two speech-language pathologists was perfect for clauses 1 and 21 and significant for clauses 2, 3, 4, 5, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, and 30. Between the three judges, agreement was low for clauses 3, 14, and 18 and high for clauses 2,5, 6, 16, 21, and 25 (Table 1).

Table 1
Distribution of the values observed by the judges, two-paired, with Kappa coefficient, and concordance rate for items 1 to 30

Clause	Judge 1 × Judge 2			Judge 1 × Judge 3			Judge 2 × Judge 3			All judges
	Kappa	SD	% con	Kappa	SD	% con	Kappa	SD	% con	Kappa	SD	% conc
1	0.18	0.141	88.5	1	0.00	100	0.18	0.140	88.5	0.49	0.054	88.5
2	0.79	0.06	90.1	0.77	0.062	89.2	0.80	0.059	90.2	0.79	0.055	84.8
3	0.18	0.054	55.7	0.13	0.057	53.1	0.64	0.109	92	0.16	0.055	50.4
4	0.68	0.077	87.6	0.69	0.073	86.7	0.71	0.073	88.5	0.69	0.054	81.4
5	0.87	0.071	97.3	0.73	0.090	92.9	0.77	0.084	93.8	0.78	0.05	92
6	0.84	0.065	94.7	0.86	0.060	95.6	0.92	0.046	97.4	0.87	0.054	93.8
7	0.26	0.089	71.6	0.47	0.082	74.3	0.30	0.07	67.3	0.32	0.054	56.6
8	0.65	0.163	96.4	0.38	0.143	90.2	0.53	0.134	92	0.50	0.054	89.4
9	0.72	0.064	85.8	0.58	0.070	78.8	0.62	0.075	82.3	0.64	0.0543	73.5
10	0.15	0.093	83.2	0.83	0.069	94.7	0.15	0.093	83.2	0.43	0.054	80.5
11	0.22	0.06	56.6	0.72	0.640	85.8	0.33	0.072	67.3	0.38	0.0542	54.9
12	0.23	0.061	63.7	0.69	0.066	85	0.29	0.085	75.3	0.40	0.054	62.8
13	0.22	0.049	53.1	0.57	0.071	77.8	0.34	0.073	68.1	0.32	0.054	49.6
14	0.00	0.021	30.1	0.84	0.056	92.9	0.03	0.015	35.4	0.04	0.055	28.3
15	0.57	0.077	80.5	0.89	0.044	94.7	0.68	0.072	85.9	0.71	0.055	80.5
16	0.80	0.063	92	0.81	0.061	92.1	0.78	0.065	91.2	0.80	0.054	87.6
17	0.11	0.102	87.6	0.79	.0.90	95.5	0.14	0.126	90.3	0.42	0.0543	86.7
18	0.14	0.048	52.2	0.83	0.085	96.4	0.18	0.048	54	0.17	0.0544	51.3
19	0.34	0.078	73.4	0.84	0.079	96.4	0.44	0.08	77	0.46	0.054	73.5
20	0.53	0.068	75.2	0.69	0.064	84.1	0.78	0.06	89.4	0.65	0.054	74.3
21	0.91	0.041	95.6	1	0.000	100	0.91	0.041	95.6	0.94	0.0542	95.6
22	0.57	0.072	79.7	0.89	0.050	95.6	0.63	0.07	82.3	0.68	0.0542	78.8
23	0.47	0.068	72.5	0.77	0.058	88.4	0.45	0.08	73.4	0.55	0.0542	67.3
24	0.35	0.069	69	0.73	0.063	86.7	0.51	0.081	80.5	0.52	0.054	68.1
25	0.75	0.064	88.5	0.77	0.062	89.4	0.94	0.032	97.4	0.82	0.0543	87.6
26	0.24	0.089	78.7	0.66	0.083	87.6	0.21	0.097	80.5	0.38	0.0542	73.5
27	0.33	0.125	87.6	0.82	0.071	94.6	0.26	0.108	84	0.50	0.0543	83.2
28	0.10	0.095	86.7	0.85	0.071	96.5	0.10	0.095	86.7	0.43	0.0543	85
29	0.13	0.081	80.6	0.83	0.062	93.9	0.10	0.066	76.1	0.39	0.0543	75.2
30	0.16	0.071	70.8	0.78	0.063	90.2	0.20	0.072	71.7	0.38	0.0544	66.4

• Fleiss’s Kappa Coefficient (α=0.6; Kappa)
• Subtitle: SD = standard deviation; con = concordance

Internal consistency analysis of the items (clauses) in each category of subordinate clauses showed values of the Cronbach’s Alpha coefficient (α=0.4), in each of the seven categories of subordinate adverbial clauses (reason, purpose (consequence and result), condition, concession, proportion, and time), calculated considering all items of the category and, then the omission of each of the items. Low values (<0.7) were observed in this analysis, indicating no internal consistency in each category of subordinate clauses. The highest Cronbach’s Alpha coefficient values were verified in the subordinate adverbial clauses of condition and purpose (result), and little or no gain was obtained when one of the clauses was excluded from these categories of clauses. This analysis showed the need to maintain the 30 task items, considering that one could not replace the other, even being part of the same grammatical category. Table 2 shows the internal consistency analysis of the items (clauses) in each category of subordinate clauses.

Table 2
Internal consistency analysis of the items in each category of subordinate clauses

Adverbial subordinate clause	Cronbach’s α	Omitted clause	Cronbach’s α after clause omission
Reason	0.00	1	0.02
		2	0.00
		3	0.12
		4	-0.15
		5	-0.04
Purpose (consequence)	0.08	6	0.20
		7	-0.11
		8	0.10
		9	-0.02
Condition	0.43	10	0.48
		11	0.20
		12	0.28
		13	0.42
Purpose (result)	0.47	14	0.39
		15	0.34
		16	0.40
		17	0.45
Concession	0.38	18	0.27
		19	0.24
		20	0.31
		21	0.46
Proportion	0.37	22	0.29
		23	0.28
		24	0.21
		25	0.41
Time	0.25	26	0.11
		27	0.09
		28	0.18
		29	0.30
		30	0.35

• Cronbach’s Alpha Coefficient (α=0.4)

Aiming to investigate the existence of some latent factor that promoted the variability observed in the responses evaluated by the judges, a factor analysis study was performed and showed factor loadings >0.40, which are highlighted in bold in Table 3 to facilitate their interpretation. Thus, the first loading factor was dominated by the following clauses: 11, 12, 14, 15, 17, 18, 19, and 28, which explains the higher variability percentage (12%). This result demonstrates that the clauses highlighted in the first loading factor are in the same direction, with a common performance trend. The main clause with the highest loading on the second factor was clause 26, and clauses 6 and 22 also showed high loadings and opposite sign. However, there is no interpretation for the opposite sign. Therefore, the second loading factor was a contrast between clause 26 and clauses 6 and 22.

Table 3
Factor loadings, commonalities, and percentages of total variability explained by the factors obtained with the principal components method

Clause\Factors	F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	Commonality
Clause 1	0.39	-0.17	0.19	-0.20	0.39	0.05	0.10	0.04	0.12	-0.02	0.08	0.02	0.17	0.48
Clause 2	0.29	0.31	-0.27	0.06	-0.15	0.08	-0.60	0.07	0.01	-0.14	-0.05	-0.01	-0.25	0.74
Clause 3	-0.03	0.37	0.06	0.03	-0.46	-0.38	0.28	0.15	0.10	-0.11	-0.14	0.07	0.17	0.68
Clause 4	0.31	-0.07	0.22	-0.48	-0.29	-0.13	0.15	-0.27	-0.36	0.11	-0.06	0.06	-0.12	0.74
Clause 5	0.28	-0.04	-0.27	0.02	0.47	-0.36	-0.21	-0.09	-0.18	-0.18	-0.19	0.02	-0.18	0.70
Clause 6	-0.20	-0.46	0.17	-0.04	0.10	0.16	-0.01	-0.19	0.13	0.20	-0.44	0.01	0.08	0.61
Clause 7	0.33	0.23	0.35	-0.31	-0.28	0.30	-0.12	-0.18	-0.29	-0.09	-0.23	-0.02	-0.21	0.78
Clause 8	0.19	-0.04	-0.26	-0.46	0.04	0.19	0.15	-0.09	-0.06	-0.31	0.14	-0.17	0.36	0.67
Clause 9	0.36	0.01	0.43	0.04	0.06	-0.19	-0.02	0.16	-0.34	0.27	0.14	0.21	-0.02	0.64
Clause 10	0.25	0.21	0.31	0.24	0.29	0.23	-0.12	-0.39	0.16	-0.11	-0.28	-0.25	0.15	0.77
Clause 11	0.59	-0.20	-0.04	0.04	-0.25	0.10	-0.24	0.26	0.14	0.08	-0.24	-0.21	0.15	0.73
Clause 12	0.46	0.17	0.01	0.18	-0.06	0.11	0.18	0.22	0.03	0.00	-0.53	0.29	-0.05	0.75
Clause 13	0.26	0.09	-0.09	0.00	0.01	0.00	0.13	0.69	-0.16	0.11	-0.05	-0.41	0.10	0.79
Clause 14	0.50	0.00	-0.05	0.03	-0.18	0.00	-0.01	-0.23	0.45	0.25	0.10	-0.34	-0.14	0.76
Clause 15	0.51	-0.10	-0.32	-0.30	-0.34	-0.01	0.03	-0.08	0.06	-0.06	-0.07	0.23	0.11	0.66
Clause 16	0.20	0.14	-0.52	-0.47	0.19	-0.24	0.07	-0.05	0.15	0.04	-0.08	0.12	-0.03	0.70
Clause 17	0.45	-0.02	0.19	-0.28	0.21	0.16	0.18	-0.01	0.19	-0.07	0.07	0.01	0.16	0.49
Clause 18	0.42	0.07	-0.29	0.36	-0.07	0.32	0.23	-0.03	0.11	-0.20	0.18	0.08	-0.26	0.71
Clause 19	0.46	-0.36	-0.31	0.06	-0.03	0.04	0.08	-0.08	-0.03	0.28	0.17	-0.05	-0.21	0.62
Clause 20	0.35	0.03	0.13	0.17	-0.05	0.17	0.56	-0.04	0.05	0.04	0.09	-0.02	-0.27	0.61
Clause 21	0.38	-0.31	0.05	0.03	0.03	0.00	-0.35	0.13	0.13	0.41	0.12	0.33	0.18	0.72
Clause 22	0.34	-0.46	0.18	0.09	0.02	0.15	-0.04	0.20	-0.33	-0.29	0.07	-0.18	-0.01	0.66
Clause 23	0.18	-0.39	0.13	0.15	-0.03	-0.45	0.07	0.04	0.18	-0.47	-0.11	0.03	-0.01	0.70
Clause 24	0.29	-0.26	-0.01	0.43	-0.17	-0.24	0.03	-0.33	-0.23	-0.10	0.19	0.03	0.13	0.65
Clause 25	0.40	0.32	0.05	0.32	-0.14	0.09	-0.11	-0.09	0.07	-0.10	0.17	0.28	0.43	0.72
Clause 26	0.26	0.63	0.08	-0.04	0.31	0.09	-0.05	-0.03	-0.15	0.12	0.20	-0.05	0.07	0.66
Clause 27	0.15	0.21	-0.09	0.22	0.17	-0.41	0.29	-0.15	-0.05	0.33	-0.21	-0.17	0.10	0.62
Clause 28	0.55	0.09	0.17	-0.04	0.44	-0.25	-0.03	0.11	0.06	-0.16	0.02	0.12	-0.19	0.69
Clause 29	0.21	0.11	0.30	-0.13	-0.28	-0.49	-0.24	-0.09	0.15	-0.02	0.16	-0.33	-0.02	0.71
Clause 30	0.18	-0.03	-0.46	0.23	0.02	0.01	0.00	-0.21	-0.50	0.10	-0.16	-0.17	0.24	0.71
Variance	3.69	1.91	1.74	1.68	1.62	1.54	1.39	1.30	1.27	1.17	1.11	1.04	1.00	20.44
Variance %	0.12	0.06	0.06	0.06	0.05	0.05	0.05	0.04	0.04	0.04	0.04	0.04	0.03	0.681

• Phi Coefficient (α=0.7)
• Subtitle: F = factor

The number of latent factors was determined considering those whose corresponding eigenvalues were ≥1. These factors explained 68.1% of the total variability of the data. The commonalities presented in Table 3 correspond to the variance proportion of each of the clauses represented by the 13 factors. Thus, clause 13 is the best explained by the 13 factors, whereas clause 1 is the worst explained.

DISCUSSION

In the current scientific literature, aspects of oral language are described as determinants of reading and writing competence. Domain of morphosyntax, whether at the linguistic or metalinguistic levels, is present as an endophenotype capable of justifying the complexity of the mechanisms that mediate the genetic and environmental determinants on full development in reading^{(2,3,8-12,29,30)}.

There is increasing concern with the pre-school period, which favors longitudinal studies that can anticipate losses⁽³⁾. However, research on this theme has turned its attention to the investigation of linguistic conditions, which are sufficiently developed and available, that will enable achievement of more abstract levels of reasoning in high-order cognitive tasks.

Assessment of the subordinate adverbial clauses allowed observation of syntactic-semantic deviations, according to the rupture in the relation of the syntactic dependence expected. The use of subordinate conjunctions and conjunctive phrases was determined by the fact that they are free, and sometimes can perform two functions; however, assessment was based on the relationship of subordination (subjunctive mood), and not of the conjunctions.

Interrater concordance between the three judges showed k=0.61<x>1 for six items, which was considered moderate. Matching for concordance between the two speech-language pathologists showed perfect agreement for two clauses and significant concordance (k=0.69<x>0.89) for 23 items. This result indicated the possibility of task reproducibility by another speech-language therapist.

Nevertheless, internal consistency analysis of the clauses in each category (categories of subordinate adverbial clauses) indicated that the condition and purpose (result) categories showed weak consistency: α=0.43 and α=0.47, respectively. No consistency was observed for the other categories. This result revealed that conjunctions of the same category do not measure the same factor. It is known that oral languages, such as Brazilian Portuguese, present variability of structure and syntactic expressions that provide the speaker with large space for grammatical formulations^(19,27).

Internal consistency analysis demonstrated that items in the same subordinate clause category showed low correlation, and an inverse correlation may occur between the items, which results in a negative Cronbach’s Alpha coefficient, as in the reason and purpose (consequence) adverbial clause categories, clauses with subordinate conjunctive phrases. This finding, especially in relation to the reason category, which showed no correlation between clauses 1 to 5, seems to confirm the wide diversity of responses elicited by the task and observed by the judges. Syntactic construction depends on grammatical knowledge, vocabulary dimension, and memory^(3,12,15,30).

Internal consistency of the proposed clauses was also assessed through item-total correlation and the Alpha coefficient. Based on the set of 30 items (clauses), the Alpha value was <0.7, which defined the necessity to maintain the 30 task items and conduct factor analysis in order to understand the high variability between the judges' responses.

From this new analysis, 13 latent factors were identified, and only one factor (latent factor 1) was responsible for the variance (12%) of the total response variability. Although no strong correlation between clauses 11, 12, 14, 15, 17, 18, 19, and 28 was identified regarding the grammatical category, they showed the same orientation of response variability.

The following clauses showed the highest correlation in factor 1: 11. “Luiza vai ser suspensa da escola caso_” (Luiza will be suspended from school if_); 12. “Você ficará com o cachorro conquanto que_” (You will stay with the dog as long as_); 14. “Meu amigo me deu um ingresso para que_” (My friend gave me a ticket so that_); 15. “Mamãe ninou o bebê no colo a fim de que_” (Mom cradled the baby so that_); 18. “Aquele nadador participou da competição embora_” (That swimmer participated in the competition although_); 19. “Eu brinquei no parque ainda que_” (I played in the park even though_); 28. “Vou voltar para casa antes que_” (I am going to go home before_). These clauses demonstrated that there is correlation, from a linguistic point of view, regarding the demand for use of the subjunctive mood in the complementation response to the main clause after the proposed conjunction. Clause 17, “As crianças fizeram silêncio para_” (The children were silent in order to_) did not show the same linguistic demand, but was directed to the circumstance of purpose, which allowed association between the main clause and the subjunctive mood: both should convey a hypothetical event that can possibly occur.

The other factors can be similarly interpreted, but they showed variability percentages <6%, reinforcing the relevance of studying the first factor. Principal component factor analysis with Varimax rotation was also performed, but this rotation did not improve interpretation of the factors.

Clauses 11, 28, 12, 19, 14, 15, and 18 stood out because they demand, strictly, the subjunctive mood in the subordinate clause (complementation of the main clause). These items seamed to explain latent factor 1 (with 12% variability) by indicating that commonality between the clauses lied on domain and use of the subjunctive mood and on domain of the semantic-lexical aspect/meaning of the words.

Other clauses such as “O cachorro fugiu porque_” (The dog ran away because_); “Vou ao teatro já que_” (I am going to the theater since_); “Maria reclamou tanto que_” (Maria complained so much that_); “Maria pode ir nadar na piscina se_” (Maria can swim in the pool if_); “O atleta ganhará a competição conquanto que_” (The athlete will win the competition as long as_); “Ela vai comer quiabo apesar de_” (She is going to eat okra despite_), and “O menino suava a proporção que_” (The boy sweated as_) showed no correlation with any of the other clauses in any of the 13 factors analyzed, which may have occurred because of the low correlation between the 30 items, the low power of the sample or, theoretically, of the diversity of factors involved in language processing in sentence parsing (syntactic resolution of the sentences).

It seems appropriate to comment on the analysis of error typologies to understand the specificities of spoken language and occurrences inherent in the task nature, as it seems to be the presence of dialogism in the relationship between examiner and examinee. However, it is worth emphasizing the importance of observing the subcategories found in the students’ responses: (a) The use of verbal phrasing and the infinitive, e.g., 11. “Luiza vai ser suspensa da escola caso continue a fazer bagunça” (Luiza will be suspended from school if she keeps on messing up) and 28. “Vou voltar para casa antes que minha mãe chegue” (I am going to go home before my mother arrives); (b) Marking of the discursive subject, as in claus 12. “Você ficará com o cachorro desde que cuide bem dele” (You will stay with the dog as long as you take good care of it) and 19. “Eu brinquei no parque ainda que estivesse chovendo” (I played in the park even though it was raining); (c) The subordinate adverbial clause of purpose (result), e.g., 14. “Meu amigo me deu um ingresso para que fôssemos ao cinema” (My friend gave me a ticket so that we went to the movies), 15. “Mamãe ninou o bebê no colo a fim de que dormisse” (Mom cradled the baby so that he slept), and 17. “As crianças fizeram silêncio para ouvir a professora falar” (The children were silent in order to hear the teacher speak); (d) Domain of the subjunctive mood, as in clause 18. “Aquele nadador participou da competição, embora não tenha ganhado em primeiro” (That swimmer participated in the competition although he did not win first place).

Factor analysis showed that the main clause best explained by the 13 factors was “O atleta ganhará a competição conquanto que_” (The athlete will win the competition as long as_). In particular, this clause should be in the category loaded by factor 1, because it is the only one that requires the use of the subjunctive mood, which was not included in the category. From the students' responses, clause 13 suggests two limitations to the test clause: 1) Variations in the examiner’s prosody: “O atleta ganhará a competição conquanto que com vinte pontos” (The athlete will win the competition as long as with 20 points); 2) ignorance about the conjunction: “O atleta ganhará a competição conquanto que que vai ganhar” (The athlete will win the competition as long as he wins). A response such as the one selected ahead shows how a possible pause after the word “competition” misled the student to understand it in the interrogative mode, “how much?”. (In Portuguese, the conjunction “conquanto”, meaning as long as, can be understood in spoken language as “com quanto”, meaning how much.) Example: “O atleta vai ganhar a competição com 20 pontos” (The athlete will win the competition with 20 points).

Factor analysis also showed that the main clause worst explained by the 13 factors was “O cachorro fugiu porque” (The dog ran away because_). From the students’ responses, it was possible to identify that the polysemy of the term “porque/because” allowed the student to understand it either as a question or as a conjunction that demanded complement. (In Portuguese, the term “porque” means why, whereas the term “por que” means because, and they are both pronounced in the same way.) The responses varied between the circumstances of purpose (result) and reason.

These are preliminary data. Disagreement between the judges was an unexpected finding, but it can be explained by the linguistic competence of the study participants. This was ensured by the screening performed, which selected, minimally, schoolchildren with expressive vocabulary appropriate to their age groups. This linguistic competence presupposes variability of utterances in the individual, a human characteristic that should be considered in all studies on language and its forms of evaluation, in all its modalities. Identification of the latent factor, associated with production of the subjunctive mood, demonstrates the need for attention to the grammar of spoken Portuguese. Although the use of the subjunctive mood is of low frequency in spoken Portuguese, one can ask if this would not be the differentiating factor of the linguistic competences. This finding is an important contribution of this study. In contrast, failure to perform correlation analysis of the responses between a control group (memory and vocabulary appropriate for age group) and study group (schoolchildren who failed the vocabulary and memory tests) certainly limited possible interpretations of the performance of these schoolchildren and the adequacy to the clinical use of the prepared task. We believe that this addition should enable performance of criterion validation studies that could indicate ways to analyze discourse and correlate oral narratives.

CONCLUSION

Interrater concordance between the speech-language pathologists was significant, task items were not correlated within the same grammatical category, and the use of the subjunctive mood in the complementation response to the main clause seems to be a latent factor promoting the response variability of students.

• Study carried out at Programa de Pós-graduação em Distúrbios da Comunicação Humana (Fonoaudiologia), Universidade Federal de São Paulo – UNIFESP – São Paulo (SP), Brasil.
• Funding: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Brazil – Code 001.

REFERÊNCIAS

• ¹
  Chiu YD. The simple view of reading across development:the prediction of grade 3 reading comprehension by prekindergarten skills. Remedial Spec Educ. 2018;39(5):289-303. http://dx.doi.org/10.1177/0741932518762055
  » http://dx.doi.org/10.1177/0741932518762055
• ²
  Hulme C, Snowling MJ. The interface between spoken and written language: developmental disorders. Biological Sciences. 2013;369(1634):1-8. PMid:24324239.
• ³
  Snowling MJ. Dyslexia: a language learning impairment. Br J Biomed Sci. 2014;2(1):43-58. PMid:24693575.
• ⁴
  Lúcio PS, Moura RJ, Nascimento E, Pinheiro ÂMV. Estudo de fidedignidade do avaliador em provas de compreensão leitora e oral. Aval Psicol. 2016;(2):161-7.
• ⁵
  Lúci PS, Moura RJ, Nascimento E, Pinheiro AMV. Construção de uma tarefa de leitura em voz alta de palavras: análise psicométrica dos itens. Psicol Reflex Crit. 2012;25(4):662-70. http://dx.doi.org/10.1590/S0102-79722012000400005
  » http://dx.doi.org/10.1590/S0102-79722012000400005
• ⁶
  Rossi NF, Lindau TA, Gillam RB, Giacheti CM. . Cultural adaptation of the Test of Narrative Language (TNL) into Brazilian Portuguese. CoDAS. 2016;28(5):507-16. http://dx.doi.org/10.1590/2317-1782/20162016018 PMid:27683830.
  » http://dx.doi.org/10.1590/2317-1782/20162016018
• ⁷
  Van Silfhout G, Evers-Vermeul J, Sanders T. Connectives as processing signals: how students benefit in processing narrative and expository texts. Discourse Process. 2015;52(1):47-76. http://dx.doi.org/10.1080/0163853X.2014.905237
  » http://dx.doi.org/10.1080/0163853X.2014.905237
• ⁸
  Elbro C, Oakhill J, Megherbi H, Seigneuric A. Aspects of pronominal resolution as markers of reading comprehension: the role of antecedent variability. Read Writ. 2017;30(4):813-27. http://dx.doi.org/10.1007/s11145-016-9702-4
  » http://dx.doi.org/10.1007/s11145-016-9702-4
• ⁹
  Teixeira BS, Schiefer AM, Carvalho CAF, Avila CRB. Compreensão oral e leitora e consciência sintática nas alterações de leitura e escrita. Rev CEFAC. 2016;18(6):1370-8. http://dx.doi.org/10.1590/1982-021620161862216
  » http://dx.doi.org/10.1590/1982-021620161862216
• ¹⁰
  Landi N, Ryherd K. Understanding specific reading comprehension deficit: a review. Lang Linguist Compass. 2017;11(2):e12234. http://dx.doi.org/10.1111/lnc3.12234 PMid:30034511.
  » http://dx.doi.org/10.1111/lnc3.12234
• ¹¹
  Kintsch W, Mangalath P. The construction of meaning. Top Cogn Sci. 2011;3(2):346-70. http://dx.doi.org/10.1111/j.1756-8765.2010.01107.x PMid:25164299.
  » http://dx.doi.org/10.1111/j.1756-8765.2010.01107.x
• ¹²
  Adlof SM, Catts HW. Morphosyntax in poor comprehenders. Read Writ. 2015;28(7):1051-70. http://dx.doi.org/10.1007/s11145-015-9562-3 PMid:27397969.
  » http://dx.doi.org/10.1007/s11145-015-9562-3
• ¹³
  Kida AS, Ávila CR, Capellini SA. Marcadores sintáticos no reconto oral de escolares disléxicos. CoDAS. 2015;27(6):557-64. http://dx.doi.org/10.1590/2317-1782/20152015106 PMid:26691620.
  » http://dx.doi.org/10.1590/2317-1782/20152015106
• ¹⁴
  Kida ASB, Bueno GJ, Lima VLCC, Rossi SG, Nepomuceno PF, Martin MMM, et al. Influência da modalidade de reconto na avaliação do desempenho de escolares em compreensão leitora. Estud Psicol. 2015;32(4):605-15.
• ¹⁵
  Deacon SH, Kieffer M. Understanding how syntactic awareness contributes to reading comprehension: Evidence from mediation and longitudinal models. Br J Educ Psychol. 2018;110(1):72-86. http://dx.doi.org/10.1037/edu0000198
  » http://dx.doi.org/10.1037/edu0000198
• ¹⁶
  Braz HA, Pellicciotti THF. Exame de linguagem TIPITI. São Paulo: MJN; 1981.
• ¹⁷
  Capovilla FC, Capovilla AGS. Prova de consciência sintática: normatizada e validada. 1. ed. São Paulo: Memnon; 2006. 73 p.
• ¹⁸
  Giusti E, Befi-Lopes DM. Tradução e adaptação transcultural de instrumentos estrangeiros para o Português Brasileiro (PB). Pro Fono. 2008;20(3):207-10. http://dx.doi.org/10.1590/S0104-56872008000300012 PMid:18852970.
  » http://dx.doi.org/10.1590/S0104-56872008000300012
• ¹⁹
  Costa J, Lobo M, Silva C, Ferreira E. Produção e compreensão de orações relativas em português europeu: dados do desenvolvimento típico, de PEDL e do agramatismo. In: Anais do XXIV Encontro Nacional da Associação Portuguesa de Linguística; 2009; Lisboa. Lisboa: APL/Edições Colibri; 2009.
• ²⁰
  Bechara E. Moderna gramática portuguesa. 38. ed. Rio de Janeiro: Nova Fronteira; 2015. 689 p.
• ²¹
  Kuder GF, Richardson MW. The theory of the estimation of test reliability. Psychometrika. 1937;2(3):151-60. http://dx.doi.org/10.1007/BF02288391
  » http://dx.doi.org/10.1007/BF02288391
• ²²
  Onwuegbuzie AJR, Johnson RB, Collins KMT. Assessing legitimation in mixed research: a new framework. Int J Soc Res Methodol. 2011;45(6):1253-71.
• ²³
  Pereira LD, Schochat E. Processamento auditivo central: manual de avaliação. São Paulo: Lovise; 1997. 82p.
• ²⁴
  Appezzato MM, Hackerott MMS, Avila CRB. Tarefa de discriminação de fala com pseudopalavras. CoDAS. 2018;30(2):e20170030. http://dx.doi.org/10.1590/2317-1782/20182017030 PMid:29791617.
  » http://dx.doi.org/10.1590/2317-1782/20182017030
• ²⁵
  Seabra AG, Trevisan BT, Capovilla FC. Teste infantil de nomeação. In: Seabra AG, Dias NM, organizadores. Avaliação neuropsicológica cognitiva: linguagem oral. São Paulo: Memnon; 2012. p. 54-82.
• ²⁶
  Wechsler D. Wechsler intelligence scale for children. 3rd ed. San Antonio: The Psychological Corporation; 1991.
• ²⁷
  Neves MHM. Gramática do português falado: novos estudos. Campinas: Unicamp; 1999. 742 p. (vol. VII).
• ²⁸
  Kaltenhauser J, Lee Y. Correlation coefficients for binary data in factor analysis. Geogr Anal. 2010;8(3):305-13. http://dx.doi.org/10.1111/j.1538-4632.1976.tb00538.x
  » http://dx.doi.org/10.1111/j.1538-4632.1976.tb00538.x
• ²⁹
  Hagoort P. Nodes and networks in the neural architecture for language: Broca’s region and beyond. Curr Opin Neurobiol. 2014;28(1):136-41. http://dx.doi.org/10.1016/j.conb.2014.07.013 PMid:25062474.
  » http://dx.doi.org/10.1016/j.conb.2014.07.013
• ³⁰
  Cain K. Text comprehension and its relation to coherence and cohesion in children’s fictional narratives. Br J Dev Psychol. 2013;21(3):335-51. http://dx.doi.org/10.1348/026151003322277739
  » http://dx.doi.org/10.1348/026151003322277739

Publication Dates

• Publication in this collection
  30 May 2019
• Date of issue
  2019

History

• Received
  19 Apr 2018
• Accepted
  21 Nov 2018