Clinical intervention studies of orofacial motricity: an analysis of the methodological quality of brazilian studies

Mailing address: Rebecca Rhuanny Tolentino Limeira Universidade Federal da Paraíba, Campus I CEP: 58.051-900 João Pessoa, Paraíba, Brasil E-mail: rebecca.rhuanny@hotmail.com ABSTRACT Purpose: to evaluate the methodological quality of clinical trials published in Brazilian journals. Methods: four trained independent researchers conducted a systematic literature search of all Brazilian speech therapy-related journals over the last ten years, whether the journals were active or inactive. All journal volumes published during that period were selected, and each researcher conducted an individual analysis to identify articles that focused primarily on orofacial motricity. The tools used were the Downs and Black Quality Checklist and the Jadad scale. Results: after the studies were selected and categorized, the final sample comprised six articles, all of which were classified as clinical trials. The observed methodological limitations included a lack of sample planning, randomization and blinding. Mean scores of 16.3 points on the Downs and Black Quality Checklist and 2.3 on the Jadad scale were obtained. Conclusion: the randomized controlled trials in the area of orofacial motricity are scarce in Brazilian literature, suggesting that studies in this area adopting this research design should be expanded and their quality should be improved to promote clinical practice based on scientific evidence.


INTRODUCTION
Speech therapy is a science that deals directly with the treatment of diseases in several areas.Orofacial motricity (OM) is the area of speech therapy dedicated to the study, research, prevention, evaluation, diagnosis, development, qualification, improvement and rehabilitation of structural and functional aspects of the orofacial and cervical regions¹.
Given this definition, it follows that the clinical practice of speech therapy requires a foundation upon which it can operate.Scientific knowledge needs to be connected with practice in ways that allow the subject of study to benefit from research ².Evidence-based practice is therefore essential to treatment.Clinicians receive information from various sources to define their practice, and it is necessary to separate what is appropriate for use in their practice from what is not³.
Research in speech therapy has grown over recent years, and the number of related publications has increased considerably.As a result, fields that had not been previously outlined were incorporated into this science's objects of study, leading to the expanded knowledge and practice of speech therapy.However, studies with a high level of scientific evidence, such as controlled clinical trials, are underrepresented, which limits the availability of scientific knowledge that can contribute to effective new treatments and minimizes the ability to refute or confirm the effectiveness of treatments already in use 4 .
Controlled clinical trials are considered the best method for determining the effectiveness of an intervention; they are the methodological basis for evidencebased health.They can produce scientific evidence that is less error-prone when clarifying cause-effect relationships between two events 5.A literature survey conducted for the years between 1970 and 2000 identified the need for more publications and research focusing on OM treatments 6 .A descriptive analysis of Brazilian scientific production from 2005 to 2015 revealed an increase in the number of published scientific articles, especially those with a cross-sectional design, which are important for identifying the occurrence and distribution of and the factors associated with the emergence of health problems related to the area under study 2 .
In this regard, it is necessary to know the current profile of Brazilian scientific production related to the effectiveness of treatment in the OM field to stimulate reflection and encourage the performance of scientific studies that consider the methodological aspects currently proposed for this type of research.On this basis, the aim of this study was to identify and evaluate, from a methodological point of view, controlled clinical trials in the OM field published in Brazilian journals.

METHODS
This was a literature search based on an extensive systematic method.Four independent researchers trained in the appropriate scientific methodology conducted a survey of all national journals related to speech therapy over the last ten years, regardless of whether they were currently active or inactive.All volumes published during that period were selected, and each researcher conducted an individual analysis to identify the articles in which the main object of study was within the field of OM.
The research involved two phases: (1) The categorization and selection of intervention studies, and (2) The evaluation of methodological aspects of clinical trials in OM.

Phase 1 (Categorization and selection of intervention studies)
All publications in the speech therapy field dated between January 2005 and March 2015 were evaluated by four independent researchers.All volumes of scientific journals in the field that were registered with an International Standard Serial Number (ISSN) were included, whether the journals were active or discontinued and without language restriction.In this analysis, all publications with a main theme within the OM field were included.The Brazilian speech therapy journals are summarized in Figure 1.
At the beginning of the last decade, a resolution of the Brazilian Federal Council of Speech Therapy (Conselho Federal de Fonoaudiologia) established five specialties within this science: audiology, language, OM, voice and public health 1 .Within the scope of Brazilian OM are the research, evaluation, diagnosis and rehabilitation of structural and functional aspects of the orofacial and cervical regions relating to the stomatognathic functions of sucking, breathing, chewing, swallowing and speaking.In 2010, the same entity regulated the dysphagia field and shifted more complex swallowing abnormalities and hospital practices away from OM.For this reason, publications dedicated to dysphagia were not considered in this systematic review.
The intervention studies were identified and categorized based on (I) type of article, (ii) study design, (iii) objective and method directly involving humans, (iv) quantitative data analysis and (v) objective involving clinical OM intervention.Because of the observed discrepancies between methodological descriptions and their actual classification in some studies, the studies were categorized by researchers who underwent calibration training in the study methodology using the most common definitions in the literature 7 .
The last step in this phase was the selection of clinical trials.Studies had to meet the minimum requirements for a trial, such as the presence of more than one intervention group (or placebo) and of groups comprising individuals recruited from the same study population.The data had to have been collected directly from patients by the research team; studies that used secondary data as intervention parameters were therefore excluded.Thus, only studies with the essence and classification of a clinical trial remained, regardless of whether the authors described using this method.

Phase 2 (The evaluation of methodological aspects of clinical OM trials)
The methodological evaluation of the clinical OM trials occurred in three stages: (1) Methodological analysis of the clinical trials and (2) Evaluation of methodological quality using the Downs and Black Quality Checklist (DBQC) 8 and the Jadad scale 9 .
The methodological analysis involved identifying and describing the intervention study's objectives and method, the presence of a control group, sample planning, the number of intervention groups, randomization, blinding, the number of statistical tests and the evaluation and diagnostic tools used.Methodological quality was then measured using the two tools.
The DBQC was developed to meet the growing demand for evidence evaluation in systematic reviews and meta-analyses.It was chosen for this review because of its psychometric properties and its ability to assess the methodological quality of both randomized and non-randomized studies 10 .The checklist has a numerical score of 25-30 points based on the evaluation of five subscales.These subscales include study quality (overall quality, including writing), external validity (generalizability of the findings), bias (in intervention and measurement), confounders and selection bias (in the sample) and power (identifying the possibility that the conclusions were haphazard).The DBQC is a valid, reliable and methodologically strong instrument that measures what it proposes to measure 8 .

RESULTS
Each publication was analyzed considering the field of OM its title, resume and full text.After that, articles which study designs not involved the analysis of clinical intervention on orofacial myology were excluded with no exception.Figure 2 outlines the research stages and describes the exclusions in each step.The final sample comprised six articles, all classified as clinical trials.
Figure 3 shows the distribution of the scientific articles with a clinical trial design published in Brazilian journals between 2005 and 2015.
We observed that outcomes related to the subject area of neonatology were more frequent (Figure 4), especially for the child and infant age groups (Figure 5).
Figure 6 describes the general aspects (objective and methodological aspects) used in the clinical trials evaluated in this study.
Figure 7 shows the methodological design aspects adopted in the evaluated clinical trials.
Table 1 describes the methodological parameters and quality scores for the Quality Check List for RCTs and Observational Studies and the Jadad scale.
The articles were scored in three possible ways: A score of "1" was granted when the element was identified in the article or when (as a precaution) we considered that the author forgot to mention it; a score of "0" was assigned when the element was not present in the article or was not considered; and a score of "0" was awarded when the item could not be determined.Studies with scores greater than or equal to 20 are considered good; those with scores between 15 and 19 are considered fair; and those with scores of 14 or below are considered poor 7 .
The Jadad scale evaluates three aspects: randomization (randomness in the selection of the study sample and the reliability of this aspect), blinding (whether it is appropriate) and the losses to follow-up in the study.This scale is well accepted in the international literature and aims to measure the quality of clinical trials [11][12][13] .It contains five questions and evaluates the quality of the methodology used.Scores can be as high as five ( 5) points, and negative values are possible.Studies with scores greater than or equal to three (3) are considered good.

Phase 1
Selection and categorization of studies

Sample:
Two groups of 10 children between 4 years and 4 years and 8 months with suction habits involving the pacifier and bottle.

Treatment groups:
RSH group: Removal of habits using the modified clarification method.MFT group: Removal of habits using the modified clarification method associated with myofunctional therapy.

Procedures:
Tested pre-treatment and 60 and 180 days post-intervention.

DEGAN & PUPPIN-RONTANI, 2007
To evaluate the effects of the association between RSH and orofacial myofunctional therapy on increased nasal breathing

Sample:
Two groups of 10 children aged 4 years to 4 years and 8 months with pacifier and bottle sucking habits.

Treatment groups:
RSH group: Subjected to RSH via the modified clarification method.
MFT group: RSH associated with MFT.The airflow expired through the nostrils was recorded using Altmann's millimeter nasal mirror and measured on millimeter paper known as Altmann's mirror reference block.

Procedures:
The tests were conducted on three occasions (pre-treatment and 60 and 80 days post-treatment).

Statistical analysis:
The results were subjected to statistical analysis using the t-test for two independent samples and for paired data, considering a significance level of p<0.05.

IDERIHA & LIMONGI 2007
To examine the use of surface electromyography to detect the efficacy of oral MFT for improving lip movement during suction in infants with Down syndrome.

Sample:
Two groups (intervention and control), eight infants in each group.

Procedures:
Three phases: Initial evaluation, therapy and re-evaluation.

Statistical analysis:
ANOVA statistical test with p<0.05.

Sample:
Twenty subjects with grade IV peripheral facial paralysis in the study group and nine subjects with grade IV peripheral facial paralysis in the control group.

Treatment groups:
Study group: Twenty subjects with grade IV peripheral facial paralysis, irrespective of etiology, aged 20-60 years old, with an anatomically intact facial nerve and onset of paralysis of 15 days.Control group: Nine subjects with grade IV peripheral facial paralysis only, irrespective of cause, who had not undergone prior orofacial rehabilitation.

Procedures:
Resting facial evaluation with photographic documentation for the two groups, video documentation for the evaluation of facial movements, measurements of the labial commissure angle (LCA), functional evaluation, implementation of the rehabilitation protocol and reevaluation after 1 year of treatment.

Statistical analysis:
The data were analyzed using the Kolmogorov-Smirnov test and Student's t-test for paired samples and independent samples.The significance level adopted was 5%.

Sample:
Twenty newborns hospitalized in the neonatal unit, with a gestational age between 26 and 33 weeks, adequate or small for gestational age and who had not been fed orally.

Treatment groups:
The newborns were randomly distributed between the stimulated group (SG) and control group (CG).

Procedures:
The groups underwent speech therapy evaluations on two different occasions: upon clearance for oral feeding and after oral enteral feeding for a 24-hour period.The evaluations were filmed, and the results analyzed by the researcher.

Statistical analysis:
Fisher's exact test and Student's t-test of the STATA 10.0 software package were used.p<0.05 was adopted.

ROSA, MOREIRA, ARAÚJO, MOREIRA JUNIOR & MOTTA, 2010
To verify acupuncture's contribution as a form of complementary treatment for speech therapy in patients with Bell's peripheral facial paralysis.

Sample:
Fifteen patients with Bell's peripheral facial paralysis, aged between 18 and 70 years old.

Treatment groups:
The subjects were selected at random to join the experimental group (speech therapy with concomitant acupuncture) or control group (speech therapy only).

Procedures:
The two groups were evaluated and then treatment sessions ensued (speech therapy and acupuncture).The groups were re-evaluated soon after by an external examiner.

Statistical analysis:
The data were analyzed using the exact test for single proportions, the t-test for independent samples, the paired t-test, the Mann-Whitney test and Fisher's exact test, with 5% significance levels.evaluate treatment effectiveness.This process provides sufficient information for the experiment to be replicated and permits the comparative analysis of treatment groups 15 .The magnitude of the experimental treatment effect compared with the control group enables the planning of later clinical trial stages, for example, stages III and IV.
The methodological limitations identified in the Brazilian studies include issues related to sample planning (sample size definition and allocation of research subjects), which directly affects the possibility of generalizing the results to the wider population.In this sense, the statistical analysis of data obtained from a non-representative portion of the population can lead to erroneous interpretation.To avoid this bias, it is necessary to consider statistical parameters related to the type of variable (discrete or continuous), the standard deviation of the variable in the population, the significance level of the estimate and the maximum sampling error tolerated.Statistical formulae for calculating sample sizes help to establish the appropriate number of individuals required for the study 18 .
Sample planning also incorporates procedures related to sampling techniques (e.g., the allocation of research subjects).Biostatistical parameters consider that the probabilistic method leads to the formation of a sample with greater representative power.Conversely, convenience allocation of specific outpatient subjects may not adequately represent the population 18 .Moreover, the decision to include or exclude a subject based on the defined inclusion and exclusion criteria should precede the randomization process 19 .
Randomization was not prioritized in all of the reviewed studies.The absence of this procedure may mean that the composition of treatment groups is biased, thus indicating an important selection bias.Randomization offers an individual the same change of participating in the control group as in the experimental group, making the groups more homogeneous and providing a balance of predictors of good and poor prognosis 15,20,21 .
Blinding was not considered in all of the evaluated studies, and when it was adopted, it did not apply to all of those involved in the research (researchersexaminers, research participants and statisticians).Blinding aims to minimize the conscious or unconscious influence of the involved parties on the study results, whether during the evaluation process or during the classification of identified effects; therefore, masking of the adopted procedures should be considered 15,21,22 .

DISCUSSION
The analysis of studies with a clinical trial design and outcomes related to OM that were published in Brazilian journals of interest to speech therapy is pioneering; it represents a strategy of reflecting on knowledge production that, in response to current scientific conceptions, proposes incorporating procedures and methodological techniques that can adequately respond to the research questions.In this regard, important aspects related to this study's findings should be highlighted: (1) in ten years of Brazilian scientific production in OM, only eleven experimental studies involving human subjects have been published; and (2) of these, only six were clinical trials.These data immediately demonstrate the need to increase the number of studies that adopt an appropriate methodological design that can evaluate the effectiveness of treatments used in speech therapy within the OM field.
Clinical trials were evaluated because they represent the gold standard study design for testing hypotheses related to the efficacy of proposed treatments 14 .In this regard, it is necessary to recognize the importance of sample size definition, the subject allocation process, the use of control groups, randomization, blinding and adequate statistical analyses, among other aspects 15,16 .
In terms of the expansion of areas of practice, it is important to consider that clinical trials provide the basis for the ethical use of interventions among the population.Analyzing and reflecting on existing scientific production therefore contributes to the dissemination of information to the academic community, clinical professionals and the public and contributes to cost reduction (for both public and private funds) because actions can be rationally planned.The absence of trials demonstrating the efficacy of a treatment in target OM treatment populations continues to delay the availability of treatments to specific market niches and means that claims of treatment effectiveness are not thoroughly justified.
In 2006, 13 systematic reviews in the Cochrane Library that aimed to evaluate the efficacy of speech therapies were identified.Methodological weaknesses related to the small number of clinical trials, the use of appropriate methodology and sample size definition criteria led to a low level of scientific evidence regarding the effectiveness of treatments in various clinical settings 17 .
The designs of the clinical trials evaluated here considered the inclusion of a control group and are characterized as stage II clinical studies because they The evaluation of the clinical trials' quality based on the Quality Checklist for RCTs and Observational Studies 7 indicates that the trials fall into the category of regular studies, with an emphasis on the categories of external validity and power, which generally received low scores.This reflects the low generalizability of results caused by the failure to adopt detailed methodological procedures related to the issues described above.
The lack of instruments with acceptable sensitivity and specificity and well-established diagnostic criteria means that the studies' conclusions cannot be compared with concrete measures.Differences in methods for evaluating and classifying dysfunctions or disorders limit the external validity of the results, meaning that they are only representative of the investigated population.
Although it was not the object of analysis in this study, the importance of considering indicators that have been validated for the clinical evaluation of injuries related to outcomes of interest in the OM field must be emphasized.Such indicators should ideally meet the requirements of observational or experimental epidemiological research.That is, they must be validated for reproducibility and broad representativeness.This is related to the indicators' ability to be applied to a large number of subjects and to meet ethical requirements in terms of not causing harm or damage to the investigated individuals 23 .
The establishment of standardized measures with acceptable reproducibility and sensitivity is the first step toward the significant advancement of research and clinical practice in the area of OM.With such definitions, similar studies can be analyzed in a comparative way, thus facilitating the application of evidence from scientific research in the field.

CONCLUSIONS
Despite advances in recent years, experimental research (clinical trials) in the OM field still have characteristics of incipient development.The use of research techniques with a good quality of evidence and the standardization and validation of methodological tools are essential steps toward evidence-based clinical practice.Multicenter studies may represent an important strategy for understanding the health problems related to the field at a population level, and the implementation of treatment approaches supported by exemplary scientific evidence will provide more

Figure 1 .
Figure 1.Brazilian speech therapy scientific journals

Figure
Figure 6.Description of clinical trials published in Brazilian journals from 2005 to 2015 focusing on outcomes of interest in the orofacial motricity field

Table 1 .
6. Description of clinical trials published in Brazilian journals from 2005 to 2015 focusing on outcomes of interest in the orofacial motricity field Methodological designs adopted by the clinical trials published in Brazilian journals from 2005 to 2015 Evaluation of clinical trial quality according to the Quality Checklist for Randomized Controlled Trials and Observational Studies (KENNELLY, 2011) and the Jadad Scale (JADAD et al., 1996)