Auditory-perceptual training for the assessment of hypernasality in cleft palate speakers: an integrative literature review

ABSTRACT Purpose: to describe the auditory-perceptual training for the assessment of hypernasality in individuals with cleft lip and palate. Methods: an integrative literature review in the databases Virtual Health Library, SciELO, and PubMed, aimed to answer the following guiding question: 1) What are the characteristics of auditory-perceptual training to assess hypernasality in individuals with cleft lip and palate? Articles in Portuguese and English, available in full access, without the restriction of the publication date, which presented programs of training for speech hypernasality, unprecedented, adapted, or replicated, were included. The pursuit of descriptors, selection, extraction, and synthesis of data was performed by three independent evaluators. Literature Review: 10 articles were included in this study, based on established criteria. Five articles investigated the effectiveness of training on speech analysis by listeners, regardless of experience level. Another five articles pertained to training when validating speech assessment protocols. Consensus analyses and reference samples were the most used training reported. Perceptual rating of phrases, using the equal appearance scale and in person training, was the most reported one. Conclusions: the auditory-perceptual training of listeners to identify hypernasality showed variability in the proposed strategies, particularly when proposed for non-experienced listeners. The difficulty in maintaining acquired skills in the long term is pointed out.


INTRODUCTION
Auditory-perceptual assessment is considered the gold standard in the identification and assessment of speech disorders in individuals with cleft lip and palate and/or velopharyngeal dysfunction.Among these disorders, speech hypernasality is the most frequent one, characterized by an excessive nasal resonance that occurs during the production of oral sounds 1,2 .Identifying the presence of hypernasality through auditory-perceptual assessment is essential for the initial diagnosis of velopharyngeal dysfunction and for the assessment of the effectiveness of the treatment, even if subjectively 3 .Instrumental measures (nasoendoscopy, videofluoroscopy, nasometry, and pressureflow technique) are commonly used to complement the diagnosis, as they offer valuable information with perceptive findings 4 .
Although essential for clinical purposes, the auditory-perceptual assessment of the speech is subjective and, therefore, can be influenced by internal factors (listeners), external factors (tasks), and the interaction of these factors 5 .The factors related to listeners include the evaluator's degree of experience, habits and errors, perceptive sensitivity, fatigue, lapses, and attention deficits 5 .Speech therapists with experience in evaluating the speech of individuals with cleft lip and palate are preferable for clinical, research, and auditing purposes 6 , even an isolated experience does not guarantee sufficient levels of intra-and inter-rater agreement.Evaluators' internal standards can be unstable over time even for an experienced evaluator 7 .
The factors external to the evaluator include, for example, the type and extent of the sample being analyzed 8 , the phonetic context 8,9, and the presence of coexisting speech disorders, such as compensatory articulations 10 and dysphonia 11 .Perceptual assessment of a speech aspect can be affected by concurrent changes in other speech subsystems.Scholars point out that listening to several dimensions in the speech signal at the same time can be a difficult task 12 and, therefore, the possible effect of the coexistence of articulatory disorders 10 and dysphonia 11 in the auditory-perceptual analysis of hypernasality 10 must be considered.
Considering that the variability in the auditoryperceptual assessment may be due to variations in the listeners' perception and also in the assessment tasks 13 , achieving a high-reliability index can be difficult, even for experienced evaluators 14,15 .To minimize errors and biases in the auditory-perceptual assessment and to increase the reliability of this type of assessment, scholars search for strategies that optimize the auditory-perceptual assessment of speech, including speech hypernasality.
According to some scholars 13 , the reliability of the analyses concerns the extension of a method used for the assessment, which offers the same result when measurements are taken repeatedly.As summarized by these scholars 13 , reliability can be achieved through strategies that include speech recordings in audio and/or video with quality equipment for analysis by multiple evaluators; definition of terms, standardization of materials, use of reference samples, and particularly, auditory-perceptual training.
In previous studies, training has been shown to increase the reliability of perceptual analyses by listeners without 16 and with experience in the assessment of speech hypernasality 14 .Some authors also suggest that inter-rater agreement rates may increase after a systematic training program for the rater 17 .Others pointed out improvement in the concordance indices after training but with no significant difference between the control (without training) and experimental (with training) groups 18 .
The importance of describing and carefully analyzing the auditory-perceptual training programs used in research is emphasized in the literature, since this analysis can offer subsidies to improve the training of listeners, both for clinical use and in research 19 .In this sense, it is necessary to expand knowledge about auditory-perceptual training for speech hypernasality and, above all, to analyze the training offered in terms of training programs and, particularly, the characteristics of this training (types, duration, and modality of training, speech stimuli used for judgments, scales used to assess hypernasality, and trained listeners).Thus, this study aimed to describe the characteristics of auditory-perceptual training for the assessment of speech hypernasality in individuals presented with cleft lip and palate.

METHODS
This study is an integrative literature review that aims to contribute to the knowledge of the characteristics of auditory-perceptual training offered to favor auditory-perceptual analysis of speech hypernasality.The question was built based on the acronym P -population, C -concept, and C -context.Thus, in this research, P was the individuals with cleft lip and palate or velopharyngeal insufficiency with speech hypernasality, C was the auditory-perceptual training programs, and C was the listeners and other training characteristics, as reported in the methodology.The study aims to answer the following guiding question: 1) What are the characteristics of auditory-perceptual training to assess hypernasality in individuals with cleft lip and palate?The selection of studies contemplating auditory-perceptual training for the assessment of speech hypernasality was carried out through a search in the national and international scientific literature contemplating auditory-perceptual training for the assessment of speech hypernasality in specialized journals available in three databases: Biblioteca Virtual da Saúde (BVS), Scientific Electronic Library Online (SCIELO), PubMed and, later, Google Scholar.
The health descriptors (DeCS) and the corresponding Medical Subject Headings (MeSH) used in the search were fissura palatina (cleft palate), insuficiência velofaríngea (velopharyngeal insufficiency), fala (speech), distúrbios da fala (speech disorders), educação (education), julgamento (judgment) and treinamento (training).The keyword treinamento auditivo (listener training) was used to help in searches.The identification of these descriptors and keywords was carried out by three researchers, independently, from November 2021 to June 2022.
Afterward, a search in the literature was performed using a combination of descriptors.There was no restriction on period, language, or nationality.The Boolean operators "e" (Portuguese) "AND" (English) and "ou" (Portuguese) "OR" (English) were used.An article found through a search on the Google Scholar platform, gray literature, was also included, as it met the established criteria, through the descriptors e-learning AND perceptual assessment.

Selection criteria
Original articles in the searched databases and available in full, without any restriction of publication dates, that presented training programs for speech hypernasality and were unpublished, adapted, or replicated, were included.Repeated articles in the databases, theses, or publications that were not in Portuguese or English, as well as studies with another proposal on the topic of interest or involving other speech disorders other than hypernasality, studies in which participants were only instructed to perform the experimental task, without the description of the training program and studies without training details, were excluded.
database, using descriptors in Portuguese and English, three of which were duplicates.After checking the titles, a single study was selected for reading the abstract and, later, this study was read in full, as it dealt specifically with training evaluators to classify hypernasality, with a detailed description of the proposed training.
Another 14 articles, one of which was duplicated, were found by searching the BVS/VHL database, using only descriptors in Portuguese.After reading the titles, two articles were excluded because they had already been included through the search in the SCIELO database, and the other 11 articles did not meet the proposed inclusion criteria.Therefore, no article in this database was selected for the study.
In the PUBMED database, 235 articles were found, using two combinations of descriptors.After the title analysis, the following exclusions were made: 22 of them did not meet the language criteria, one referred to the thesis, 12 articles were not found in full and 25 were duplicated or repeated searches carried out in other databases.With that, 175 articles were analyzed and of these, 117 were excluded due to non-adherence to the theme proposed for this study.Afterward, 58 articles were selected for reading the abstracts, 29 of them were excluded, as they did not present information regarding training for the classification of hypernasality.Finally, 29 articles from the PUBMED database were read in full, and in which 8 of them were included in this study.
A single article, which had not been found in the previous databases, but which was identified in Google Scholar (gray literature) was included due to its importance regarding the proposed theme.Therefore, ten articles are the total sample for this review (Scielo N=1, PUBMED N=8; Google Scholar N=1). Figure 1 shows the article selection process, considering the three databases consulted together.

Data Extraction and Analysis
The search for material in the databases was carried out manually and independently by three evaluators.The selection of material was carried out in three stages: 1) reading the titles of the articles found, 2) reading the abstracts, and 3) reading the articles in full.Initially, the evaluators individually read the titles and abstracts to analyze adherence to the theme.Studies that did not meet the eligibility criteria defined in this study (non-adherence to the theme, duplicate studies, language other than Portuguese or English) were excluded.The three evaluators had a consensus regarding these exclusions.Afterward, the evaluators read the texts in full, individually.The results of each of the evaluators were discussed and a consensus was reached regarding the characteristics of the research.
After the selection of the articles, the following data were extracted from the studies: author and year of publication, study objectives, training methods (type, duration, and modality), speech stimuli used in judgments, scales used to assess hypernasality, characterization of trained listeners and results obtained.Data extraction and organization were performed in an Excel spreadsheet.For extraction, the instrument was adapted, based on previous studies.The items considered for analysis were study identification and training characteristics (types, duration, training modality, speech stimuli and scales used, and trained listeners).Also, a summary of the findings was presented, however, it was not an object of analysis to identify the best perceptual training proposal that would lead to more reliable results.

LITERATURE REVIEW
The search strategy found a total of 259 studies.Of this total, nine articles were found in the SCIELO  A significant difference in interrater agreement between the groups with practice (training with and without feedback) and the group without practice (exhibition of samples).Between the two groups that performed the practice, there was no significant difference.
Both groups with practice showed an increase in the reliability of intra-rater and interrater analyses.Therefore, training (practice with and without feedback) helped improve the reliability of hypernasality ratings.

(To describe the reliability indices of the speech assessment of two perceptual assessment protocols CAPS-A and CAPS-A-AM) *CAPS-A-AM: modifications made to the CAPS-A protocol
Training offered using the CAPS-A (Study 1) Training included a description of the steps and details of the procedures adopted; phonetic transcription and classification of speech parameters using scales, and practices with consensus.Study 2 CAPS-A-AM: There were no significant differences in results between raters, including for hypernasality.

Duration
Inter-rater reliability indices of Study 2 were lower than those of Study 1.After adjusting the data analysis procedures, the findings were similar.
In both studies, there was good intra-rater reliability.In general, the training positively affected the reliability results of the analyses and the students' confidence in making the judgments (another aspect of interest in the study); however, these findings were dependent on the analyzed speech variable and at the time of the measurement.
Little or no training effect (short and long-term) was observed for hypernasality, air emission, and nasal turbulence variables.

Captions: N = Number of participants; ENT = Otorhinolaryngology
Regarding the identification of studies, five of the ten reviewed studies had the main objective of describing perceptual training and verifying the effect of training on the reliability of listeners' analyses of the speech parameters, including hypernasality 13,14,16,18,20 .These studies were named "training" in this review.One of these five articles was national 14 and four were international 13,16,18,20 .The other five studies aimed to develop and validate protocols for perceptive assessment of the speech of individuals with cleft lip and palate and proposed training to verify the agreement between different listeners when using the developed protocols 17,21-24.These studies were named "perceptual assessment protocol with training for protocol use" in this review.All of these five articles were international.
The study developed in 2009 16 was the forerunner among the studies whose focus was to describe auditory-perceptual training for hypernasality and to verify its outcomes (training) 13,14,16,18,20 .This study investigated the effect of training and feedback on the intra-and inter-rater reliability of the hypernasality judgment in non-experienced listeners.Two other studies carried out in 2016 aimed to assess the ability of Otorhinolaryngology residents to classify hypernasality in patients with velopharyngeal dysfunction 18 and to investigate the influence of experienced evaluators' training on the agreement in the perceptive judgment of hypernasality before and after the prior training 14 .
Two other more recent investigations were carried out, one in 2021 13 and the other in 2022 20 .The study conducted in 2021 13 aimed to evaluate the results of training (short and long-term) through an e-learning tool, developed to evaluate speech characteristics related to cleft lip and palate and the students' perception of the training proposal presented.The study conducted in 2022 20 aimed to evaluate the immediate and long-term effect of perceptual training on the reliability of intra-and inter-rater analyses, in addition to the student's experience in auditoryperceptual assessments of speech in patients with cleft palate, using an assessment protocol developed in a study previous 24 .
Based on the objectives of these five studies, it was verified that there is a constant and even recent concern of scholars in proposing auditory-perceptual training that favors auditory-perceptual analysis of speech hypernasality.This concern has even led scholars to propose auditory-perceptual training for the assessment of speech hypernasality directed at speech therapists already experienced in assessing the speech of individuals with cleft lip and palate and/ or velopharyngeal dysfunction (VPD) 14 .Scholars are also concerned about verifying the effect of long-term training, particularly when it is offered to untrained listeners 13,20 .In addition, there is a tendency to seek information about the students' perception of the presented proposal 20 .
Regarding the studies that used auditory-perceptual training of aspects of speech in cleft lip and palate, when developing and validating speech assessment protocols for individuals with cleft lip and palate, the study developed in 2006 21 was the forerunner.This study aimed to develop an assessment instrument -Cleft audit protocol for speech (CAPS-A) to use in intercenter audit studies in cleft lip and palate and to test the reliability and validity of this assessment instrument.Later, in 2009, scholars 22 aimed to design, execute, and evaluate a training program for speech therapists with systematization and reliability in the use of the CAPS-A assessment protocol, directing the problems of sample standardization, data acquisition, recording, reproduction, and listening guidelines.
In 2016, researchers 17 sought to describe the speech assessment reliability indices of two perceptual assessment protocols (Cleft Audit Protocol for Speech-Augmented -CAPS-A and Cleft Audit Protocol for Speech-Augmented-Americleft Modification -CAPS-A-AM) developed to assess speech outcomes in inter-center collaborative studies and investigated the effect of training on agreement between different listeners.For this, two studies were conducted, one using CAPS-A and the other with modifications in this assessment protocol (CAPS-A-AM).
A study developed in 2018 23 aimed to measure interand intra-rater reliability using the Dutch Cleft Speech Evaluation Test (DCSET) and convert the DCSET into universal scales.In 2020, a study aimed to develop and validate an instrument in the Belgian language for perceptual assessment in patients with cleft palate 24 , based on a previous assessment protocol (CAPS-A)21.Intra-and inter-rater reliability indices were reported (including for hypernasality) after a 4-hour training that included protocol presentation and consensus practices.
Based on the objectives proposed in these five studies, it appears that auditory-perceptual training is considered an important strategy to obtain the reliability of the auditory-perceptual analysis of hypernasality, when developing and validating protocols for evaluating the speech of individuals with cleft lip and palate 17,[21][22][23][24] .

Types of training
The types of auditory-perceptual training to assess speech hypernasality varied among the five studies that aimed to describe perceptual training and verify its outcomes in the hypernasality speech parameter (training).A single study 16 used practice (with and without feedback) carried out in stages, with a gradation of levels of difficulty (four levels, from easiest to hardest) to prepare two groups of untrained listeners to classify degrees of hypernasality 16 .The other group included in the study did not receive training and was only exposed to speech samples (passive listening).
Two studies 14,20 used consensus analyses.One of these studies 14 used the definition of criteria, followed by the establishment of reference samples, consensually established by speech-language therapists with experience in evaluating the speech of individuals with cleft lip and palate for later use of these references in the judgment of hypernasality.In this study, therefore, consensual analyses were essential for establishing references that were used by speech-language therapists in their analyses of speech samples with different degrees of hypernasality.The other study involving consensus analyses 20 used criteria definition, followed by the presentation of audio and audiovisual speech samples for practice in the judgment of speech parameters (including hypernasality), with additional samples for training in the judgment of these parameters, first individually and afterward, consensually.The discussion of analyses by listeners (untrained) was pointed out in this study as an important strategy that can favor the learning of these listeners.
One study 13 used reference samples (anchors) to favor the perceptive analyses of non-experienced listeners.In this study, an e-learning platform was used for perceptual assessment focusing on speech disorders in cleft lip and palate (including hypernasality).On this platform, basic information about the condition or subcategories of interest for analysis is offered, followed by information and exercises (including videos and audio) referring to the different domains to be evaluated.In the study, the perceptual training was carried out through the platform with access to video (for phonetic transcription) and audio, contemplating examples of types and degrees of speech disorders (including hypernasality), with the possibility of comparison with samples established consensually by experienced speech therapists (reference samples), aiming to familiarize listeners (students from two universities) with the types of errors, in addition to calibrating them and, in one instance, enabling them to perform reliable analyses.Another study 14 also used reference samples, but these references were pre-established by consensus analyses, as already mentioned.In general, auditory-perceptual training, using fixed external references (pre-established) is considered essential to reduce the impact of internal factors related to the evaluator (for example, experience) and, also, factors related to the task imposed in the evaluation 16,17,22 .
On study 18 used the exposure of listeners to speech samples (passive listening).This study proposed access to an educational module contemplating speech sample, in addition to explanations about aspects of velopharyngeal dysfunction and clinical correlation.However, no further details were reported on the training used to classify hypernasality.
The types of auditory-perceptual training to assess hypernasality used in the five studies focused on the development, measurement, and validation of speech assessment protocols were well-defined and not very variable.In general, these studies 17,21-24 described carrying out practical exercises to obtain consensual analyses, as part of the training carried out.
More specifically, a study 17 proposed a workshop to carry out the training, with a presentation of the materials and methodology of the CAPS-A protocol.All parameters and definitions were described to the participants, including categories and subcategories, use and standardization of scales.As part of the training, participants performed tasks that included the classification of speech and transcription parameters, as well as practical exercises to obtain consensus.After the training, changes were proposed in some speech samples, a presentation of videos instead of audio, in addition to subtle adjustments in speech parameters and definitions of these parameters (CAPS-A-AM).In another study 21 the training was established by consensus and involved the presentation of six cases, and included the definition of criteria, being offered to the speech-language therapists who participated in the training, qualitative descriptions regarding the use of the instrument before performing the task.
Another study 22 proposed training led by experienced professionals familiar with the CAPS-A assessment protocol.Initially, information on the development process of the CAPS-A assessment protocol was presented for auditing purposes, with a review of the definitions adopted by the protocol (use and standardization of numerical scales of speech parameters, including hypernasality) and a review of the types of errors in the production of consonants in speech of individuals with cleft lip and palate.For this, videos and audio were presented to illustrate the aspects of speech to be evaluated and cases for consensus analysis.After, it was requested that each speech-language therapist analyze auditory or visually the material offered, followed by a discussion of the findings jointly to obtain consensus.
One study 23 used the DCSET protocol to evaluate the speech parameters of children with cleft lip and palate, using audio recordings.After that, modifications in this assessment protocol were proposed from consensual discussions (training) in the scales used to evaluate the speech parameters.These discussions were conducted using video recordings of children with cleft lip and palate.
One study 24 developed and validated a Belgian protocol based on CAPS-A.In phase 1 (preliminary study), adaptations of some speech parameters and protocol structure were made for perceptual analysis.Reference samples were also presented pre-classified for each speech variable and each degree of the scale (training).Consensus practice was performed for 1 hour and a half.After an experimental session at this stage, the protocol was optimized.Phase 2 (protocol validation) included a description and explanation of definitions, scales for the classification of speech parameters, and a presentation of the structured assessment protocol.Edited speech samples were used to illustrate the corresponding speech variables and scale degrees (training).
The analysis of the ten studies included in the review shows that consensual analyses are the most used trainings 14,17,20,[21][22][23][24] , followed using reference samples 13 .A study 14 established reference samples based on consensual analyses, for later use of these references in individual analyses, suggesting the use of consensual analyses and reference samples to favor the analyses of speech-language therapists (listeners with experience).
Consensus analysis requires a group of listeners to actively listen to audio and/or video speech samples.Discussions among evaluators are expected to reach a consensus analysis of the evaluated aspect 19 .According to scholars 19 , when establishing consensus, it is sought to offer multiple opportunities for listeners to analyze the parameter of speech of interest (hypernasality), aiming at the development of the listener's ability to accurately quantify the presented speech parameter.
The reference samples are samples pre-classified by experienced speech-language therapists for the listeners to use during their training, and these should compare the new sample to that pre-classified, judging this new sample as more or less hypernasal concerning the external reference 19 .The references are considered effective strategies to establish internal standards for the evaluator because they enable them to experience and become familiar with the references used in the training.Therefore, listeners tend to store these models in their memory as internal patterns, that is, the representations are stored in memory as examples 25 .Some authors 14 argue that the use of anchor samples (reference) during the classification task may result in a significant improvement in the accuracy of speech sample severity classifications of gravity of speech samples, even for experienced listeners.
The practice (with and without feedback), although little explored among the studies that propose perceptual-auditory training for the evaluation of hypernasality of speech, also offers multiple opportunities for listeners to analyze the parameter of speech of interest (hypernasality), for the development of the listener's ability to accurately quantify the speech parameter presented 19 .

Duration of training
The duration of training described in the reviewed articles varied and was dependent on the purpose of the study and the type of training offered.Studies that aimed to describe perceptual training and verify the effect of training on the reliability of the listeners' analyses about hypernasality (training) indicated a duration ranging from 40 minutes (minimum) 18 to two hours (maximum) 20 .In one study 13 , the duration of individual training was not measured since listeners (students in training) performed the training according to their own time.One study 14 did not show the duration of training performed.Studies that focused on the development, measurement, and validation of speech assessment protocols for individuals with cleft lip and palate pointed to a duration of 2 hours 24 , 6 hours 21 , 3 days 17 , or 4 days 22 .One study 23 did not inform the duration of the training performed.The analysis of the duration of auditory-perceptual training to assess hypernasality was longer in studies involving the development and validation of speech protocols and shorter in studies involving training, particularly aimed at non-experienced listeners.

Modality of training
Eight of the ten articles reviewed conducted auditory-perceptual training in person 14,16,17,[20][21][22][23][24] .The two remaining studies reported the use of remote platforms to perform perceptual training 13,18 .These two studies point to the possibility of auditory-perceptual training of hypernasality and other speech parameters of individuals with cleft lip and palate in the online modality.According to scholars 20 , this type of training can be advantageous since it enables the participation of students in distance training and also allows listeners to carry out practices in their time.On the other hand, technical problems were pointed out by students who participated in interactive activities (online) using an e-learning platform 13 .By using online learning platforms, it becomes possible to propose continuous perceptual training, in addition to calibrating listeners for studies focused on the outcome of treatment results for the management of speech disorders in cleft lip and palate, which makes this resource very attractive 13 .
Six of the ten articles analyzed 14,17,[20][21][22]24 used combinations of speech samples that included phrases and other stimuli. Thefollowing combinations were reported: sentences, counting and spontaneous speak 21 , sentences, counting, spontaneous speak 22 ; sentences, counting and rhyming 17 ; sentences and counting 14 ; sentences, counting and connected speak 22 and phrases and spontaneous speak 22 .One study used only isolated words as a speech stimulus 13 and three other studies used only phrases 16,18,23 .In one study 24 the speech stimuli varied according to the age group of the participants.In another study 24 , children counted from 1 to 10, while adults did from 1 to 20 and 60 to 70.A single article did not provide information about the speech stimuli contained in the recordings used in its study 18 .
In general, it was observed that the training offered included combinations of speech stimuli, and phrases were the stimulus most present in training.Sets of oral phrases consisting of controlled stimuli may favor auditory-perceptual analysis of hypernasality 9 and are commonly used in research and clinical practice.

Scales used in training
Nine of the studies presented in the review used the equal intervals scale, using a score of 3 to 5 points, showing that this type of scale is still the most used in training studies for perceptual analysis of hypernasality.In this type of scale, the evaluator assigns a score to the evaluated aspect, indicating its level of severity, in which the lowest value of the scale refers to the absence of change while the highest value points to the maximum degree of disorder 3 .Only one of the studies that used the equal interval scale reported using a threepoint scale (absent, low hypernasality, and high hypernasality) 23 .The others used a scale of 4 or 5 points.A single study of this review used the direct magnitude scale for the perceptual analysis of hypernasality 16 .
The use of scales based on proportion (relation), including direct estimation, is advocated by scholars who argue that equal interval scales are inconsistent with the perceptual nature of speech nasality 26 .According to literature, nasality is a sensation mentally processed as a prosthetic dimension, that is, it differs in terms of changes in quantity or magnitude.Thus, when judging prosthetic stimuli, the listeners do not perceive the intervals between the categories as equal at different points on the scale 27 .Although scales based on proportion (relation) are pointed out as appropriate for analysis of speech nasality, scales with equal intervals are still the most used clinically and in research, including those that present perceptual training of hypernasality, by favoring comparisons between scales and between evaluators 3 .

Listeners included in the training
The evaluators in the review studies included listeners with and without experience in assessing speech hypernasality.Most of the reviewed studies that described perceptual training and verified its outcome in the speech parameter hypernasality (training) included listeners without experience (students in Speech-Language Therapy (N=3) 13,16,20 or residents of Otorhinolaryngology (N=1) 18 .A single study 14 proposed auditory-perceptual training for speechlanguage therapists with experience in the evaluation of speech changes in cleft lip and palate.
Auditory-perceptual training is necessary to reach agreement in auditory-perceptual analyses 13 and, therefore, represents an important strategy that can be used to prepare listeners for their clinical practices 18 .As for inexperienced listeners, Speech-Language Therapy students were selected in three studies 13,16,20 , as they represented individuals who should be prepared to conduct clinical assessments of speech aspects, including hypernasality, for initial diagnosis of velopharyngeal dysfunction.Otorhinolaryngology residents were included in a study 18 .According to the authors, the assessment of many aspects of speech requires collaboration between speech-language therapists and otorhinolaryngologists; however, the otorhinolaryngologist may be the professional who has the first contact with a symptomatic patient, having the responsibility of carrying out preliminary assessments of speech disorders, including the hypernasality.In this sense, these professionals must be trained to initially identify hypernasality.
A study 14 included experienced listeners (speechlanguage therapists who work at a craniofacial center).As summarized by some authors 13 , the evaluator's internal standards can be considered unstable, based on previously heard speech samples and, therefore, the evaluator's experience alone does not guarantee intra-and inter-evaluator agreement.Thus, the more exposure to deviant speech, the greater differentiation in internal patterns is achieved, which can result in poor concordance.On the other hand, listeners may begin to converge, in their analyses, when they act together in the same craniofacial center.Furthermore, the coexistence of speech disorders can impact perceptual analyses, including the aspect of hypernasality.In this sense, auditory-perceptual training for experienced speech-language therapists may represent an important strategy to increase reliability levels of hypernasality perceptual analysis.
As expected, all studies focusing on the development, measurement, and validation of speech assessment protocols for individuals with cleft lip and palate included speech-language therapists.These professionals received training to verify whether training would result in increased reliability of responses (intra and between evaluators) for the speech parameters analyzed, including hypernasality.In addition, professionals had the opportunity to maintain the skills developed after training, as part of continuing education 22 .

Results achieved in studies
In general, the reviewed studies reported favorable results for the use of auditory-perceptual training for the assessment of speech hypernasality.However, the results reported in the studies that described perceptual training and its outcomes (training) 13,14,16.18,20were less expressive than those obtained in the studies that used training, to develop and validate speech assessment protocols for individuals with cleft lip and palate 17, .
One of the five studies that aimed to describe perceptual training and to verify the effect of training on the reliability of the listeners' analysis on hypernasality (training) 13,14,16,18,20 demonstrated that the training of listeners without experiences (with and without feedback) proposed through a hierarchy of tasks, from the simplest to the most difficult, increased reliability (inter and intra) in the judgment of hypernasality, a fact that did not occur for the group that was only exposed to the samples of speech 16 .Similarly, another study 14 showed that, after training, there was an increase in the index of intra and inter-evaluator agreement of the analyses performed, even by listeners with previous experience in the assessment of speech hypernasality.However, three other studies involving non-experienced listeners showed less favorable results 13,18,20 .
Particularly, in the study that investigated the ability of Otorhinolaryngology residents to perform the classification of speech hypernasality 18 , an improvement in the concordance indices was observed after training, but with no significant difference between the groups (control and experimental, with training).Therefore, this is different from results found in other studies 14,16 .In another study 13 , a significant increase in the total performance of Speech-Language Therapy students in phonetic transcription and the classification of some aspects of speech (hyponasality, weak intraoral pressure) was observed with the use of an e-learning instrument for perceptive evaluation of aspects of speech.However, there was only a trend towards improvement in responses for hypernasality and this trend was only observed for one of the two groups of students included in the study.Again, these results differ from those found in other studies 14,16 .
Finally, in the study 20 that investigated the immediate and long-term effect of training on intra-and interrater reliability, the results suggested a positive effect of training on inter-rater reliability, but dependent on the analyzed variable and the time in which the measurement was taken.In general, little or no training effect was observed for the variables hypernasality, air emission, and nasal turbulence, which differed from previous results 14,16 .
The analysis of the results obtained in these studies suggests that the variability in the findings involving non-experienced listeners seems to be related to the differences in the training strategies employed, the scales used, and the training modalities offered (online or face-to-face).Some scholars 20 argue that training including practices with levels of difficulty 16 may favor the development of the ability to classify speech hypernasality, particularly if this training is offered online 13 , making it possible to include curricular hours for interaction between students and teachers with a transfer of fundamental information.According to these scholars 20 , training could be optimized by including a comparison of analyses with samples pre-classified by experienced listeners and by exercises to establish consensus analyses among students.
The five studies that used auditory-perceptual training of speech aspects in cleft lip and palate, but with a focus on the development, measurement, and validation of protocols for evaluating the speech of individuals with cleft lip and palate 17,[21][22][23][24] , showed positive results.That is, there were better reliability indices in the hypernasality analyses after speechlanguage therapists' auditory-perceptual training.Particularly, the precursor study (development of CAPS-A) 21 suggested that, after a 6-hour training, reliability indexes in perceptual analyses can be achieved.In a later study 22 , in which a training package was developed for the use of the CAPS-A protocol, the results showed that, in general, there was a significant increase in the reliability indexes for intra-and interevaluator analyses for different aspects of speech (including hypernasality).This finding was obtained based on analyses performed immediately after training and repeated after one month of training.
Similar results were pointed out in a later investigation 17 , using a training package developed for the use of the CAPS-A protocol 17 .Another study 23 observed positive effects arising from training and the standardization of an assessment protocol (Dutch Cleft Speech Evaluation -DCSET).Intra-and inter-rater reliability rates of speech parameters increased after training.The lowest indexes achieved were speech hypernasality.Finally, in the validation process of a study that developed a speech assessment protocol in the Belgian language, the intra-and inter-rater reliability indexes were reported (including for hypernasality) after a 4-hour training that included the presentation of the protocol and consensus practices.The analysis of the results obtained in these studies suggests a positive effect of training on the reliability of the analysis of listeners with experience in the assessment of hypernasality when using training as part of the development or validation of protocols for evaluating the speech of individuals with cleft lip and palate.
This literature review shows that auditory-perceptual training for the assessment of speech hypernasality in individuals with cleft lip and palate is poorly described in the literature, especially for non-experienced listeners.However, there is a tendency to propose perceptualauditory training for listeners who are students of Speech-language Audiology courses 13,20 .This fact reflects the continuous search for structured training aimed at students of Speech-Language Therapy courses to enable them to perform reliable analysis of speech parameters of individuals with cleft lip and palate, including hypernasality 20 .
In the reviewed studies, some limitations on the training offered were pointed out and may be useful to guide future studies.These limitations include limited time of the auditory-perceptual training offered 20,24 , the failure to carry out the pre-training analysis for comparison purposes 17, and the difficulty in achieving a positive effect on the auditory-perceptual classification over time, after auditory-perceptual training of the hypernasality 20 .Other factors that may affect auditoryperceptual training include restricted speech samples 18 and of evaluators in the trainings 1,16,18 , lack of standardization of equipment used in the classifications by the evaluators 21 , the audio quality of the speech samples used 22,23 , the use of ordinal scales because they are inconsistent with the perceptive nature (prosthetic dimension) of hypernasality 14,24 and coexistence of other speech disorders when evaluating hypernasality 20 .
Aiming to meet some of these limitations, scholars reinforce the need to propose, in future studies, structured auditory-perceptual training that allows the maintenance of the skills learned in the long-term 20 .Recommendations on the need for investigations aimed at understanding the training and the type of feedback that favors the maintenance of the skills developed by listeners in the long-term were made long-term 16 and therefore deserve attention.
Samples of external references established based on consensual analyses (group consensus) or determination of anchor samples by the listeners who will analyze the speech aspects or, even, the combination of them are pointed out as strategies that can favor perceptive analyses of non-experienced speech therapists 24 and, therefore, should be considered in future studies.The listeners' skills can also be favored by offering feedback on correct answers in practices involving consensual analysis and by offering additional information such as the presentation of specific cases with precise definitions for the use of the ordinal scale 24 .The use of proportion scales (direct magnitude estimation) is still suggested in future studies involving training to favor perceptive analyses 24 .
Perceptual training with an emphasis on speech samples with mild and moderate hypernasality is also recommended to enable more accurate discrimination of these degrees of hypernasality of speech by the listeners 18 .The use of simpler perceptual scales is also suggested since they can increase the reliability of classifications and consensus among evaluators 23 .In future studies, these aspects can be considered.
It is known that perceptual training, when performed gradually, that is, in stages, with difficulty levels (4 levels, easier to more difficult) 16 favor the analysis of non-experienced listeners and, therefore, should be further explored in future studies.These trainings can also be offered in the online modality 13 .The online modality of auditory-perceptual training is very attractive since it allows the listener to perform perceptual analysis of the evaluated parameter in their own time and an interactive.Therefore, it should be explored in future studies aimed at the auditory-perceptual training of hypernasality.
In future studies, questionnaires should be sent to participants inserted in the online perceptual-auditory training since they can offer valuable suggestions.In a previous study, for example, participants suggested the insertion of technical information in the proposed training sessions to avoid possible problems during the training performance 13 .
This review provides important information on the characteristics of auditory-perceptual training of hypernasality.This information can guide future studies that aim to optimize the perceptual analysis of speech nasality to favor appropriate diagnoses and direct therapeutic processes.

CONCLUSION
Studies involving auditory-perceptual training to identify hypernasality are still scarce.Of those existing, half refer to investigations that aimed to describe and analyze the outcomes of auditory-perceptual training to identify hypernasality.The other half aimed to develop, measure, and validate speech assessment protocols and used auditory-perceptual training for this purpose.The characteristics of speech perceptual training for the assessment of hypernasality varied widely, especially among studies that proposed auditoryperceptual training for inexperienced listeners.The most used types of training were consensual analyses and reference samples.The duration of the training was dependent on the purpose of the study and the type of training used (face-to-face or online).The most recurrent speech stimulus in perceptual training was the set of sentences.Equal interval scales were the most used in the studies.Trained listeners were speech-language therapists, speech-language therapy students, and residents in Otorhinolaryngology.The findings of auditory-perceptual training for the assessment of hypernasality derived from this review point to the need for new training that favors the perceptive analysis of non-experienced listeners and to identify training that can maintain the skills achieved by listeners in the long term.

Figure 1 .
Figure 1.Flowchart of the article selection process

1 Counting
: 3 days Modality: face-to-face Individual analyses Study 1 (CAPS-A): before, immediately, one month after training Study 2 (CAPS-A-AM): analyses on two moments, 5 weeks after study -A: Significant improvements in inter-rater reliability indices after training, including hypernasality parameter.
The combinations used between the descriptors were fissura palatina AND distúrbios da fala AND fala AND julgamento; education AND speech disorders AND (velopharyngeal insufficiency OR cleft palate) AND listener training; cleft palate AND speech disorders AND judgment; education AND speech disorders AND velopharyngeal insufficiency AND cleft palate AND training; and speech disorders AND cleft palate OR velopharyngeal insufficiency AND training, as shown in Chart 1.

). Chart 2. Studies
contemplating characteristics of auditory-perceptual training for the assessment of speech hypernasality