Two-dimensional perceptual videofluoroscopic swallowing analysis of the pharyngeal phase in patients older than 50 years

Chaves, Rosane de Deus; Mangilli, Laura Davison; Sassi, Fernanda Chiarion; Jayanthi, Shri Krishna; Zilberstein, Bruno; Andrade, Claudia Regina Furquim de

doi:10.1590/S0102-67202013000400005

Abstracts

BACKGROUND: Videofluoroscopy is considered the "gold standard" procedure for the evaluation of swallowing by most units that treat patients with dysphagia, having a great impact in decision making, not only in therapeutic terms, but also in determining the prognosis. AIM: To propose and to verify the reproducibility of the results of a perceptual two-dimensional videofluoroscopic protocol for the analysis of the pharyngeal phase of swallowing in a population of healthy adults. METHODS: Participants were 20 healthy adults, of both genders, with ages between 50 and 65 years. Videofluoroscopy was performed during the swallow of the following consistencies: 10 ml of liquid; 7 ml of paste; and half a "salt and water" biscuit. The protocol was composed by four parts: assessment of the pharyngeal transit time; assessment of the duration of the tongue base movement to the posterior pharyngeal wall; valleculae residue ratio; assessment of penetration/aspiration. Statistical analysis involved the assessment of data reproducibility between raters and analysis of the quantitative data regarding gender. RESULTS: Comparison among raters indicated that data was highly reproducible. No significant differences were found between genders for pharyngeal transit time; for the duration of the tongue base movement to the posterior pharyngeal wall; and for the valleculae residue ratio. CONCLUSION: The perceptual two-dimensional videofluoroscopy analysis demonstrated to be a reproducible method. Valleculae residue was present in 40% of the study sample, suggesting that this parameter alone does not indicate alterations of the pharyngeal phase of swallowing.

Swallowing; Swallowing disorders; Fluoroscopy

RACIONAL: A videofluoroscopia é considerada o procedimento "padrão-ouro" para avaliação da deglutição pela maioria das unidades que atendem pacientes com disfagia, tendo grande impacto na tomada de decisão, não só em termos terapêuticos, como também na determinação do prognóstico. OBJETIVO: Propor e verificar a reprodutibilidade de protocolo de análise videofluoroscopica bidimensional perceptual da fase faríngea da deglutição em uma população de adultos saudáveis. MÉTODOS: Participaram desta pesquisa 20 indivíduos saudáveis, de ambos os gêneros, com idades entre 50 e 65 anos. O exame foi realizado durante a ingestão de "alimentos" com as seguintes consistências: 10 ml de líquido; 7 ml de pastoso; e meio biscoito do tipo "água e sal". O protocolo proposto foi composto de quatro partes: avaliação do tempo de trânsito faríngeo; avaliação da duração do contato da base de língua na parede posterior da faringe; avaliação da porcentagem de resíduo alimentar em valécula; avaliação de penetração e aspiração na árvore respiratória. A análise estatística envolveu a avaliação da reprodutibilidade do método de análise entre avaliadores e a análise dos dados quantitativos, levando-se em consideração os gêneros. RESULTADOS: Pela análise comparativa entre os avaliadores houve alta reprodutibilidade. Não foram encontradas diferenças estatisticamente significantes para o tempo de trânsito faríngeo; para a duração do contato da base de língua na parede posterior da faringe e para a porcentagem de resíduo na valécula. Os resultados sugerem que os parâmetros de deglutição avaliados não tem diferença entre os gêneros. CONCLUSÃO: As análises asseguram a reprodutibilidade do protocolo proposto para análise bidimensional perceptual da videofluoroscopia. O resíduo na valécula mostrou-se presente em 40% da amostra, sugerindo que este parâmetro, isoladamente, não é indicativo de alteração na fase faríngea da deglutição.

Deglutição Transtornos da deglutição; Fluoroscopia

ORIGINAL ARTICLE

Unit of the Central Institute of Speech and the Department of Radiology, Institute of Radiology, Hospital das Clinicas, Faculty of Medicine, University of São Paulo, São Paulo, SP, Brazil

Correspondence

ABSTRACT

BACKGROUND: Videofluoroscopy is considered the "gold standard" procedure for the evaluation of swallowing by most units that treat patients with dysphagia, having a great impact in decision making, not only in therapeutic terms, but also in determining the prognosis.

AIM: To propose and to verify the reproducibility of the results of a perceptual two-dimensional videofluoroscopic protocol for the analysis of the pharyngeal phase of swallowing in a population of healthy adults.

METHODS: Participants were 20 healthy adults, of both genders, with ages between 50 and 65 years. Videofluoroscopy was performed during the swallow of the following consistencies: 10 ml of liquid; 7 ml of paste; and half a "salt and water" biscuit. The protocol was composed by four parts: assessment of the pharyngeal transit time; assessment of the duration of the tongue base movement to the posterior pharyngeal wall; valleculae residue ratio; assessment of penetration/aspiration. Statistical analysis involved the assessment of data reproducibility between raters and analysis of the quantitative data regarding gender.

RESULTS: Comparison among raters indicated that data was highly reproducible. No significant differences were found between genders for pharyngeal transit time; for the duration of the tongue base movement to the posterior pharyngeal wall; and for the valleculae residue ratio.

CONCLUSION: The perceptual two-dimensional videofluoroscopy analysis demonstrated to be a reproducible method. Valleculae residue was present in 40% of the study sample, suggesting that this parameter alone does not indicate alterations of the pharyngeal phase of swallowing.

Headings: Swallowing. Swallowing disorders. Fluoroscopy.

INTRODUCTION

Over the last two decades, studies have greatly increased our knowledge of the swallowing process. It is known that difficulties in swallowing, also known as dysphagia, can occur at any age and have different causal factors¹⁴. Problems related to swallowing usually have a negative impact on nutrition and diet maintenance in people⁸. Both clinical and videofluoroscopic evaluations are indicated for managing swallowing disorders^7,24.

Most units treating patients with dysphagia consider videofluoroscopy (VF) the "gold standard" procedure for swallowing evaluation because of its great impact on decision making in regards to both treatment and prognosis^24,25. VF is an objective examination that assesses the anatomy and physiology of the swallowing process, and it enables the clinician to observe events beginning in the oral cavity up to events occurring in the stomach²⁴.

The techniques used in VF vary considerably regarding the food provided to the patient during the exam and the methodology used for its implementation and interpretation of results. The methodological inconsistencies hinder the comparison of results from different evaluators and different studies and the ability to determine a standard for normal swallowing^2,18,30.

Although the standardization of VF procedures has been suggested¹⁴, the quality and interpretation of findings have not been sufficiently researched^2,10,29. The presence or absence of aspiration has been the most relevant VF sign and is usually the only factor considered for decision making in oral feeding management and determining most suitable consistency for each case¹⁹. Because of its objectivity and specificity, the examination enables the analysis of other aspects of swallowing such as motility disorders of the oral and pharyngeal cavities, the presence of residue in the oral and pharyngeal cavities, the percentage of food residue accumulated along the digestive tract, and other esophageal aspects¹⁵. To assist in standardizing VF, some variables, including a definition of terms, criteria of event observations, analysis method, reproducibility of measurements, and evaluator expertise, deserve particular attention so that VF can be validated^10,12,16,29.

The aim of the current study is to propose and assess the reproducibility of a protocol for the two-dimensional perceptual videofluoroscopic analysis of swallowing in a population of healthy adults.

METHODS

Participants

This research was approved by the Ethics Committee of the Institution (CAPPesq 0074/08). Participants only began the research procedures after signing a free and informed consent form.

Twenty healthy subjects of both genders (10 male and 10 female) aged between 50 and 65 years (59±4.39) with no history of pulmonary, cardiac, or neurological diseases, neoplasias, oropharyngeal and/or laryngotracheal surgeries, or a history of hospitalization in the last 12 months and with self-reported normal swallowing and no complaints regarding swallowing participated in this study.

Videofluoroscopy of swallowing

VF procedures were performed by an experienced radiologist and monitored by an experienced speech therapist at the Institute of Radiology (Instituto de Radiologia - InRad) of Hospital das Clínicas, University of São Paulo, São Paulo, SP, Brazil. The participants remained seated throughout the examination at an angle of 90º, and the swallowing assessment was performed in a lateral view. The method used to perform the exam is the standard procedure used at InRad and consists of: 1) offering of liquid (50% filtered water, 50% barium contrast) at a volume of 10 ml, measured on a disposable syringe and subsequently placed in a plastic cup; 2) offering a paste consistency (20-ml industrial fruit puree, 5-ml barium contrast), the volume was measured using disposable syringe and subsequently placed on a spoon with a standardized volume of 7 ml; 3) offering a solid consistency (half saltine cracker dipped in 2.5 ml of the pasty preparation).

For each one of the tested consistencies participants were asked to eat in their usual manner to reduce possible interferences in their swallowings. The images were recorded on VHS tapes and subsequently digitized and stored on DVDs. TOO DVD Ripper Platinum v4.0.64 software was used for film digitization. VirtualDub v1.8.6 software was used for analyzing the digitized version of examinations, which enabled frame-by-frame analysis.

Protocol for the two-dimensional perceptual videofluoroscopic analysis of swallowing

The protocol consists of four parts and is based on the specific literature^7,17,21: It involved the following: assessment of pharyngeal transit time, assessment of the duration of the tongue base contact with the posterior pharyngeal wall, assessment of the percentage of valleculae residue, and assessment of penetration and aspiration. In order to guarantee reliability of the performed measurements, all images were analyzed on the same computer with the zoom tool at 100%.

Pharyngeal transit time - corresponds to the time interval (in seconds) from the bolus head passing through the ramus of the mandible (event 1) until the bolus tail passes through the cricopharyngeal sphincter (event 2). In order to determine this measure, each one of the two events was identified on the digital images. The conversion to seconds was performed, considering that for the used software each frame corresponds to 0.033 seconds(∆t Event 1/Event 2).

Duration of the tongue base contact with the posterior pharyngeal wall - corresponds to the time interval (in seconds) from the first (event 1) until the last (event 2) contact of the tongue base to the posterior pharyngeal wall during swallowing. In order to determine this measure, each one of the two events was identified on the digital images. The conversion to seconds was performed, considering that for the used software each frame corresponds to 0.033 seconds (∆t Event 1/Event 2).

Percentage of valleculae residue - corresponds to the quantification of bolus residue after the first swallow. In order to obtain this percentage, two measurements are considered: the area of the valleculae and the area of the residue inside the valleculae.

Measurement of the area of the valleculae: this measure is obtained by using an area function (height by width). The image of the valleculae prior to swallowing is obtained by freezing the digitalized image on the computer screen. Measurements (height and width), in millimeters, are performed on the actual computer screen using a digital caliper. The vallecuale height is considered the distance from the tip of the epiglottis perpendicular to the base of the valleculae and the width is considered the widest horizontal portion of the valleculae.

Measurement of the area of the residue inside the valleculae: this measure is obtained by using an area function (height by width) of the residue inside the valleculae. In order to perform this measurement, the image of the valleculae after the first swallowing of each one of the tested consistencies is frozen on the computer screen. Measurements (height and width), in millimeters, are performed on the actual computer screen using a digital caliper. The residue height is considered the distance from the base of the valleculae residue perpendicular to the residue top and the residue width is considered the widest horizontal portion of the valleculae residue. The percentage of valleculae residue is determined by expressing the area of the residue in the valleculae relative to the area of the available space in the valleculae as shown below:

Severity scale for valleculae residue - After determining the percentage of valleculae residue, severity is determined as follows: normal - <3%; mild - >3 to <25%; moderate - >25 to <55%; severe >55%.

Penetration/Aspiration scale: Was used the methodology already validated in the literature¹⁵. Penetration/aspiration was determined by using an 8-point multidimensional perceptual scale. Depth of bolus invasion into the airway is a major dimension. Scores were attributed as follows: material does not enter the airway (1); material enters the larynx but stays above the vocal folds and is expelled from the airway (2); material enters the airway, stays above the vocal folds and is not expelled from the airway (3); material enters the airway, touches the vocal folds and is expelled from the airway (4); material enters the airway, touches the vocal folds and is not expelled from the airway (5); material enters the airway, passes below the vocal folds and is expelled from the airway (6); material enters the airway, passes below the vocal folds and is not expelled from the airway (7); material enters the airway, passes below the vocal folds and no effort is made to expel it (8).

Data analysis

SPSS software Version 13.0 was used for the statistical analysis of results. A significance level of p<0.05 was used in all of the analyses. The results regarding the swallowing of 10 ml of liquid consistency were used as standards.

The statistical analysis involved two steps. The first, assessed the inter-rater reproducibility. For this step, 20% of the samples were randomly selected. The samples were evaluated according to the protocol described by six raters, who were speech therapists with experience in the field. The analyses were independently performed on the same computer with three previous meetings for training on the applied methodology. All of the raters could watch the videos as many times as necessary until they were satisfied with their interpretations. The intraclass correlation coefficient (ICC) for quantitative variables by analysis of variance (paired Student's t-test) and the Bland-Altman plot analysis were used to assess reproducibility. The second step involved the analysis of quantitative data from the sample considering the genders. Descriptive analyses were performed for normally distributed data (using the Shapiro-Wilk test) presenting the means and standard deviations. Data without a normal distribution (according to Levene's test) were presented using the medians and interquartile ranges IQRs (25-75%). Student's t-test was used to compare quantitative variables with normal distributed data, and the nonparametric Mann-Whitney's test was used to compare variables without normal distribution.

RESULTS

The parameters of pharyngeal transit time, duration of the tongue base contact with the posterior pharyngeal wall, and the percentage of valleculae residue were considered in both steps of the analyses. All of the participants had a score of 1 on the penetration-aspiration scale¹⁵, validating that all presented a healthy swallow.

Step 1 - Assessment of interrater reproducibility

Data were analyzed using two different statistical methods to extensively ascertain the reproducibility of the VF assessment methodology proposed in the present study. The interrater reproducibility of responses was initially analyzed using the ICC. The ICC was based on results of the analysis of variance of repeated measurements and was classified as follows¹⁷: 0.90-0.99=high reproducibility; 0.80-0.89=satisfactory reproducibility; 0.70-0.79=weak reproducibility; and <0.69=poor reproducibility. Bland and Altman¹⁸analysis was performed to assess the interrater reproducibility of measurements performed by the six raters.

The results are described in Tables 1 to 3.

Thumbnail

The t-Student test did not identify significant differences among the measurements obtained by each one of the raters for any of the tested parameters. Regarding the ICC, all of the raters reached statistical significance with high reproducibility for most comparisons. The ICC represented by a value of 1.000 indicates the situations in which the raters showed no differences for the performed analyses. For these situations, the remaining statistical analyses were not applicable.

Step 2 - Analysis of quantitative data

Tables 4 to 6 show the analysis of quantitative results related to gender. No significant differences were found between gender for the pharyngeal transit time (Table 4), for the duration of the tongue base contact with the posterior pharyngeal wall (Table 5), nor for the percentage of valleculae residue (Table 6). The results suggest that the assessed swallowing parameters do not differentiate.

Thumbnail

Although the evaluated subjects were healthy individuals and had no swallowing complaints and/or penetration/aspiration, 40% of the participants presented residue with a severity ranging from mild to moderate (12 participants classified as normal; six classified as mild; two classified as moderate).

DISCUSSION

A low interrater reliability is unacceptable for any instrument of evaluation. The rater should be able to reproduce consistent results over time for the same patients and should be able to maintain identical parameters when evaluating different patients. A satisfactory evaluation protocol should enable different raters to reproduce the results. In dysphagia, the intra- or interrater discrepancy may impact the decision to prescribe treatments or the choice between performing exercises and using compensatory maneuvers⁷.

The choice of an evaluation method is a critical issue in data analysis for instrument validation because it should represent the greatest possible accuracy²⁷. Two statistical methods were used to assess the reproducibility of the VF analysis protocol and showed high interrater reliability. Thus, the results suggest that the measures proposed in the protocol are reproducible in clinical practice and may be used in a larger scale.

Videofluoroscopy is the procedure of choice to evaluate the relationship between pharyngeal physiology and the patient's symptoms¹⁴. Most VF protocols published identify the vallecular residue as a diagnosis parameter¹⁴. A reliable interpretation of images must be performed regardless of whether the images are from the same subject at different times or from a group of subjects at the same time for the VF to be regarded as a valid and reliable instrument of evaluation. VF studies show low levels of agreement for several parameters^16,25,29. The parameter with the highest agreement regards the presence or absence of aspiration¹⁰, especially when compared with the assessment of severity of the vallecular residue^16,19,21,25. In the present study, the methodology used for assessing the percentage of vallecular residue was effective and demonstrated high/satisfactory reproducibility.

The mean age of the participants in this study was 59 years (±4.39). According to Estatuto do Idoso [Elderly Statute]⁴, subjects are considered to be elderly from the age of 65. Although the participants of the present study were classified as healthy and had no swallowing-related complaints, the percentage of valleculae residue occurred at mild and moderate levels. This may be related to age because the group of participants exhibited age limits between the age to be considered as adult or as elderly. Swallowing disorders are common in elderly people and will become a relevant issue in Health Care management because the elderly population has been growing²³. The physiology of swallowing is known to change with age. The physiological decrease of muscle mass and connective tissue elasticity results in losses of strength and mobility¹⁵. Those changes may negatively impact the effectiveness and efficiency of swallowing and airway protection. In general, according to the literature, there is a subtle slowdown of the swallowing process with age along with other alterations related to the oral preparatory phase, the number of swallows, and the presence of food residue along the digestive tract²⁶. Future studies with groups of healthy subjects should consider other age groups to determine parameters of normalcy and contribute to the diagnosis of swallowing disorders.

The present study observed no significant differences between genders. The literature describes the existence of anatomical and functional differences between men and women throughout the digestive tract (oral cavity, pharynx, upper esophageal sphincter, and esophagus)^1,6. According to the literature, women demonstrate higher oropharyngeal transit times^5,6, longer durations of the upper oesophageal sphincter opening²⁶, shorter time periods between swallows and lower food bolus volume for each swallowing¹³, longer durations of laryngeal closure during swallowing¹, and early laryngeal closure relative to the upper esophageal sphincter opening¹¹ compared with men. Hiss et al.⁹ also reported that women present longer swallowing apnea. In contrast, studies show that men have higher pressure and muscle endurance relative to the tongue pressure in the oral cavity²⁸. However, our findings suggest that those differences had no impact on the analytical variables described in the proposed protocol. One possible explanation is that many previous studies used manometry to assess the differences between genders, and many of the swallowing parameters presented were not addressed in the protocol. A proposal for future studies is the application of the VF analysis protocol on a large scale to confirm the lack of a difference between genders.

Finally, this study had a few limitations. Although the presented protocol explores the measurement of valleculae residue, the analyses is perceptual and is related to the method used for calculating area (linear height by linear width). This approach calculates the area of a rectangle (two-dimensional) and not the circumscribed area of the actual residue or space (i.e. elliptical or oval triangular shape)²⁰. Also, it does not consider possible individual anatomical differences. As described in a recent study²⁰, this limitation implies that two different residue volumes could generate identical proportional values. Besides that, it is well known that the space inside the valleculae determines the amount of residue that can be stored until it reaches the limit of pooling and possible aspiration. It is hypothetically possible for two individuals with similar heights to present valleculaes of different sizes or, on the contrary, two individuals with different heights to present valleculaes of similar sizes and capacities. A better understanding of the relationship between the size of the valleculae and body structures is still needed in order to determine the real risk of aspiration.

CONCLUSION

The analyses performed in the present study ensure the reproducibility of the proposed protocol for the two-dimensional perceptual videofluoroscopic swallowing analysis. Valleculae residue was found in 40% of the studied sample, suggesting that this parameter alone is not indicative of alterations in the pharyngeal phase of swallowing for this age group.

REFERENCES

1. Alves LM, Cassiana RA, Santos CM, Dantas RO. Gender effect on the clinical measurement of swallowing. Arq Gastroenterol. 2007;44:227-229.
2. Baijens LW, Speyer R, Passos VL, Pilz W, Roodenburg NR, Clave P. Swallowing in Parkinson patients versus healthy controls: reliability of measurements in videofluoroscopy. Gastroenterology Research and Practice. 2011:380682. epub 2011 october 3.
3. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet. 1986;8:307-310.
4. Câmara dos Deputados. Estatuto do Idoso (2009) [Internet]. Biblioteca Digital da Câmara dos Deputados, 4Ş edição. - Centro de Documentação e Informação. Disponível em: http://bd.camara.gov.br
5. Dantas RO, Alves LMT, Santos CM, Cassiani RA. Possible interaction of gender and age on human swallowing behavior. Arq Gastroenterol. 2011;48:195-198.
6. Dantas RO, de Aguiar Cassiani RA, dos Santos CM, Gonzaga GC, Alves LMT, Mazin SC. Effect of gender on swallow event duration assessed by videofluoroscopy. Dysphagia. 2009;24:280-284.
7. Dyer JC, Leslie P, Drinnan MJ. Objective computer-based assessment of valleculae residue - is it useful? Dysphagia. 2008;23:7-15.
8. Heffner JE. Swallowing complications after endotracheal extubation: moving from "whether" to "how". Chest. 2010;137:509-510.
9. Hiss SG, Treole K, Stuart A. Effects of age, gender, bolus volume, and trial on swallowing apnea duration and swallowing/respiratory phase relationships of normal adults. Dysphagia. 2001;16:128-135.
10. Kuhlemeir KV, Yates P, Palmer JB. Intra- and interrater variation in the evaluation of videofluorographic swallowing studies. Dysphagia. 1998;13:142-147.
11. Kurosu A, Logemann JA. Gender effects on airway closure in normal subjects. Dysphagia. 2010;25:284-290.
12. Leslie P, Drinnan MJ, Finn P, Ford GA, Wilson JA. Reliability and validity of cervical auscultation: a controlled comparison using videofluoroscopy. Dysphagia. 2004;19:231-240.
13. Logemann JA, Pauloski BR, Rademaker AW, Kahrilas PJ. Oropharyngeal swallow in younger and older women: videofluoroscopic analysis. Journal of Speech, Language and Hearing Research. 2002;45:434-445.
14. Logemann JA. Evaluation and treatment of swallowing disorders. 2nd ed. Austin, TX: Pro-Ed, 1998.
15. Marik PE, Kaplan D. Aspiration pneumonia and dysphagia in the elderly. Chest. 2003;124:328-336.
16. McCullough GH, Wertz RT, Rosenbek JC, Millis RH, Ebb WG, Ross KB. Inter- and intrajudge reliability for videofluoroscopic swallowing evaluation measures. Dysphagia. 2001;16:110-111.
17. Mokhlesi B, Logemann JA, Rademaker AW, Stangl CA, Corbridge TC. Oropharyngeal deglutition in stable COPD. Chest. 2002;121:361-369.
18. Monte FS, Da Silva-Junior, Braga-Neto P, Nobre-e-Souza M, Sales de Bruin VM. Swallowing abnormalities and dyskinesia in Parkinson's disease. Movement Disorders. 2005;20(4):457-462.
19. Ott DJ. Observer variation in evaluation of videofluoroscopic swallowing studies: a continuing problem. Dysphagia. 1998;13:148-150.
20. Pearson Jr. WG, Molfenter SM, Smith ZM, Steele CM. Image-based measurement of post-swallow residue: the normalized residue ratio scale. Dysphgia. 2013;28:167-177.
21. Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A penetration-aspiration scale. Dysphagia. 1996;11:93-98.
22. Ross MD, Langford B, Whelan PJ. Test-retest realiability of 4 single-leg horizontal hop tests. Journal of Strenght and Conditioning Research. 2002;16:617-622.
23. Schindler JS, Kelly JH. Swallowing disorders in the elderly. The Laryngoscope. 2002;112:589-602.
24. Shem KL, Castillo K, Wong SL, Chang J, Kao M, Kolakowsky-Hayner SA. Diagnostic Accuracy of Bedside Swallow Evaluation Versus Videofluoroscopy to Assess Dysphagia in Individuals With Tetraplegia. PM&R. 2012;4:283-289.
25. Stoeckli SJ, Thierry AGM, Seifert B, Martin-Harris BJW. Interrater reliability of videofluoroscopic swallow evaluation. Dysphagia. 2003;18:52-57.
26. Sura L, Madhavan A, Carnaby G, Crary MA. Dysphagia in the elderly: management and nutritional considerations. Clinical interventions in aging. 2012;7:287-298.
27. Takito MY, Neri LCL, Benício MHDA. Avaliação da reprodutibilidade e validade de questionário de atividade física para gestantes. Revista Brasileira de Medicina Esportiva. 2008;14:132-138.
28. Vanderwegen J, Guns C, Nuffelen GV, Elen R, Bodt MD. The influence of Age, Sex, Bulb Position, Visual Feedback, and the Order of Testing on Maximum Anterior and Posterior Tongue Strength and Endurance in Healthy Belgian Adults. Dysphagia. 2012. doi: 10.1007/s00455-012-9425-x.
29. Wilcox F, Liss JM, Siegel GM. Interjudge agreement in videofluoroscopic studies of swallowing. Journal of Speech and Hearing Research. 1996;39:144-152.
30. Wooi M, Scott A, Perry A. Teaching speech pathology students the interpretation of videofluoroscopic swallowing studies. Dysphagia. 2001;16:32-39.