SciELO - Scientific Electronic Library Online

 
vol.20 número3Influência do modelo da seringa nos resultados do teste de fluxo para líquidos proposto pela International Dysphagia Diet Standardisation InitiativeHiperplasia do processo coronóide da mandíbula: relato de caso índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

Compartilhar


Revista CEFAC

versão impressa ISSN 1516-1846versão On-line ISSN 1982-0216

Rev. CEFAC vol.20 no.3 São Paulo maio/jun. 2018

http://dx.doi.org/10.1590/1982-021620182032318 

REVIEW ARTICLES

Clinical intervention studies of orofacial motricity: an analysis of the methodological quality of brazilian studies

James Tomaz-Morais1 

Jully Anne Soares de Lima2 

Brunna Thaís Luckwu-Lucena2 

Rebecca Rhuanny Tolentino Limeira2 

Sâmara Munique Silva2 

Giorvan Ânderson dos Santos Alves2 

André Ulisses Dantas Batista2 

Ricardo Dias de Castro2 

1Centro Universitário de João Pessoa - UNIPÊ, João Pessoa, Paraíba, Brasil.

2Universidade Federal da Paraíba - UFPB, João Pessoa, Paraíba, Brasil.

ABSTRACT

Purpose:

to evaluate the methodological quality of clinical trials published in Brazilian journals.

Methods:

four trained independent researchers conducted a systematic literature search of all Brazilian speech therapy-related journals over the last ten years, whether the journals were active or inactive. All journal volumes published during that period were selected, and each researcher conducted an individual analysis to identify articles that focused primarily on orofacial motricity. The tools used were the Downs and Black Quality Checklist and the Jadad scale.

Results:

after the studies were selected and categorized, the final sample comprised six articles, all of which were classified as clinical trials. The observed methodological limitations included a lack of sample planning, randomization and blinding. Mean scores of 16.3 points on the Downs and Black Quality Checklist and 2.3 on the Jadad scale were obtained.

Conclusion:

the randomized controlled trials in the area of orofacial motricity are scarce in Brazilian literature, suggesting that studies in this area adopting this research design should be expanded and their quality should be improved to promote clinical practice based on scientific evidence.

Keywords: Evidence-Based Clinical Practice; Review Literature as Topic; Myofunctional Therapy; Methodology

Introduction

Speech therapy is a science that deals directly with the treatment of diseases in several areas. Orofacial motricity (OM) is the area of speech therapy dedicated to the study, research, prevention, evaluation, diagnosis, development, qualification, improvement and rehabilitation of structural and functional aspects of the orofacial and cervical regions1.

Given this definition, it follows that the clinical practice of speech therapy requires a foundation upon which it can operate. Scientific knowledge needs to be connected with practice in ways that allow the subject of study to benefit from research 2. Evidence-based practice is therefore essential to treatment. Clinicians receive information from various sources to define their practice, and it is necessary to separate what is appropriate for use in their practice from what is not3.

Research in speech therapy has grown over recent years, and the number of related publications has increased considerably. As a result, fields that had not been previously outlined were incorporated into this science's objects of study, leading to the expanded knowledge and practice of speech therapy. However, studies with a high level of scientific evidence, such as controlled clinical trials, are underrepresented, which limits the availability of scientific knowledge that can contribute to effective new treatments and minimizes the ability to refute or confirm the effectiveness of treatments already in use4.

Controlled clinical trials are considered the best method for determining the effectiveness of an intervention; they are the methodological basis for evidence-based health. They can produce scientific evidence that is less error-prone when clarifying cause-effect relationships between two events5).

A literature survey conducted for the years between 1970 and 2000 identified the need for more publications and research focusing on OM treatments6. A descriptive analysis of Brazilian scientific production from 2005 to 2015 revealed an increase in the number of published scientific articles, especially those with a cross-sectional design, which are important for identifying the occurrence and distribution of and the factors associated with the emergence of health problems related to the area under study2.

In this regard, it is necessary to know the current profile of Brazilian scientific production related to the effectiveness of treatment in the OM field to stimulate reflection and encourage the performance of scientific studies that consider the methodological aspects currently proposed for this type of research. On this basis, the aim of this study was to identify and evaluate, from a methodological point of view, controlled clinical trials in the OM field published in Brazilian journals.

Methods

This was a literature search based on an extensive systematic method. Four independent researchers trained in the appropriate scientific methodology conducted a survey of all national journals related to speech therapy over the last ten years, regardless of whether they were currently active or inactive. All volumes published during that period were selected, and each researcher conducted an individual analysis to identify the articles in which the main object of study was within the field of OM.

The research involved two phases: (1) The categorization and selection of intervention studies, and (2) The evaluation of methodological aspects of clinical trials in OM.

Phase 1 (Categorization and selection of intervention studies)

All publications in the speech therapy field dated between January 2005 and March 2015 were evaluated by four independent researchers. All volumes of scientific journals in the field that were registered with an International Standard Serial Number (ISSN) were included, whether the journals were active or discontinued and without language restriction. In this analysis, all publications with a main theme within the OM field were included. The Brazilian speech therapy journals are summarized in Figure 1.

* Active journal; † Discontinued journal.

Timeline of discontinued journals: PF>JSBFa>CoDAS and RSBFa>ACR.

Figure 1: Brazilian speech therapy scientific journals 

At the beginning of the last decade, a resolution of the Brazilian Federal Council of Speech Therapy (Conselho Federal de Fonoaudiologia) established five specialties within this science: audiology, language, OM, voice and public health1. Within the scope of Brazilian OM are the research, evaluation, diagnosis and rehabilitation of structural and functional aspects of the orofacial and cervical regions relating to the stomatognathic functions of sucking, breathing, chewing, swallowing and speaking. In 2010, the same entity regulated the dysphagia field and shifted more complex swallowing abnormalities and hospital practices away from OM. For this reason, publications dedicated to dysphagia were not considered in this systematic review.

The intervention studies were identified and categorized based on (I) type of article, (ii) study design, (iii) objective and method directly involving humans, (iv) quantitative data analysis and (v) objective involving clinical OM intervention. Because of the observed discrepancies between methodological descriptions and their actual classification in some studies, the studies were categorized by researchers who underwent calibration training in the study methodology using the most common definitions in the literature7.

The last step in this phase was the selection of clinical trials. Studies had to meet the minimum requirements for a trial, such as the presence of more than one intervention group (or placebo) and of groups comprising individuals recruited from the same study population. The data had to have been collected directly from patients by the research team; studies that used secondary data as intervention parameters were therefore excluded. Thus, only studies with the essence and classification of a clinical trial remained, regardless of whether the authors described using this method.

Phase 2 (The evaluation of methodological aspects of clinical OM trials)

The methodological evaluation of the clinical OM trials occurred in three stages: (1) Methodological analysis of the clinical trials and (2) Evaluation of methodological quality using the Downs and Black Quality Checklist (DBQC)8 and the Jadad scale9.

The methodological analysis involved identifying and describing the intervention study's objectives and method, the presence of a control group, sample planning, the number of intervention groups, randomization, blinding, the number of statistical tests and the evaluation and diagnostic tools used. Methodological quality was then measured using the two tools.

The DBQC was developed to meet the growing demand for evidence evaluation in systematic reviews and meta-analyses. It was chosen for this review because of its psychometric properties and its ability to assess the methodological quality of both randomized and non-randomized studies10. The checklist has a numerical score of 25-30 points based on the evaluation of five subscales. These subscales include study quality (overall quality, including writing), external validity (generalizability of the findings), bias (in intervention and measurement), confounders and selection bias (in the sample) and power (identifying the possibility that the conclusions were haphazard). The DBQC is a valid, reliable and methodologically strong instrument that measures what it proposes to measure8.

The articles were scored in three possible ways: A score of "1" was granted when the element was identified in the article or when (as a precaution) we considered that the author forgot to mention it; a score of "0" was assigned when the element was not present in the article or was not considered; and a score of "0" was awarded when the item could not be determined. Studies with scores greater than or equal to 20 are considered good; those with scores between 15 and 19 are considered fair; and those with scores of 14 or below are considered poor7.

The Jadad scale evaluates three aspects: randomization (randomness in the selection of the study sample and the reliability of this aspect), blinding (whether it is appropriate) and the losses to follow-up in the study. This scale is well accepted in the international literature and aims to measure the quality of clinical trials11-13. It contains five questions and evaluates the quality of the methodology used. Scores can be as high as five (5) points, and negative values are possible. Studies with scores greater than or equal to three (3) are considered good.

Results

Each publication was analyzed considering the field of OM its title, resume and full text. After that, articles which study designs not involved the analysis of clinical intervention on orofacial myology were excluded with no exception. Figure 2 outlines the research stages and describes the exclusions in each step. The final sample comprised six articles, all classified as clinical trials.

Legend: OM = Orofacial Motricity

Figure 2: Sequence of systematic review stages 

Figure 3 shows the distribution of the scientific articles with a clinical trial design published in Brazilian journals between 2005 and 2015.

Figure 3: Distribution of clinical trials on orofacial motricity 

We observed that outcomes related to the subject area of neonatology were more frequent (Figure 4), especially for the child and infant age groups (Figure 5).

Figure 4: Percentage of clinical trials per orofacial motricity topic, n = 14 

Figure 5: Percentage of clinical trials per orofacial motricity topic, according to age group (n = 14) 

Figure 6 describes the general aspects (objective and methodological aspects) used in the clinical trials evaluated in this study.

Figure 6: Description of clinical trials published in Brazilian journals from 2005 to 2015 focusing on outcomes of interest in the orofacial motricity field 

Figure 7 shows the methodological design aspects adopted in the evaluated clinical trials.

Figure 7: Methodological designs adopted by the clinical trials published in Brazilian journals from 2005 to 2015 

Table 1 describes the methodological parameters and quality scores for the Quality Check List for RCTs and Observational Studies and the Jadad scale.

Table 1: Evaluation of clinical trial quality according to the Quality Checklist for Randomized Controlled Trials and Observational Studies (KENNELLY, 2011) and the Jadad Scale (JADAD et al., 1996) 

Author/year Quality Checklist for RCTs and Observational Studies Jadad Scale
Writing External validity Internal validity (bias) Internal validity (confounders/selection) Power Total
DEGAN & PUPPIN-RONTANI, 2005 8 3 4 3 2 20 2
DEGAN & PUPPIN-RONTANI, 2007 5 0 7 2 1 15 4
IDERIHA & LIMONGI 2007 8 1 4 2 2 17 0
TESSITORE, PASCHOAL & PFEILSTICKER 2009 6 3 0 1 2 12 0
YAMAMOTO et al.2009 6 1 6 4 1 18 4
ROSA et al. 2010 6 1 6 3 1 16 4

Discussion

The analysis of studies with a clinical trial design and outcomes related to OM that were published in Brazilian journals of interest to speech therapy is pioneering; it represents a strategy of reflecting on knowledge production that, in response to current scientific conceptions, proposes incorporating procedures and methodological techniques that can adequately respond to the research questions. In this regard, important aspects related to this study’s findings should be highlighted: (1) in ten years of Brazilian scientific production in OM, only eleven experimental studies involving human subjects have been published; and (2) of these, only six were clinical trials. These data immediately demonstrate the need to increase the number of studies that adopt an appropriate methodological design that can evaluate the effectiveness of treatments used in speech therapy within the OM field.

Clinical trials were evaluated because they represent the gold standard study design for testing hypotheses related to the efficacy of proposed treatments14. In this regard, it is necessary to recognize the importance of sample size definition, the subject allocation process, the use of control groups, randomization, blinding and adequate statistical analyses, among other aspects15,16.

In terms of the expansion of areas of practice, it is important to consider that clinical trials provide the basis for the ethical use of interventions among the population. Analyzing and reflecting on existing scientific production therefore contributes to the dissemination of information to the academic community, clinical professionals and the public and contributes to cost reduction (for both public and private funds) because actions can be rationally planned. The absence of trials demonstrating the efficacy of a treatment in target OM treatment populations continues to delay the availability of treatments to specific market niches and means that claims of treatment effectiveness are not thoroughly justified.

In 2006, 13 systematic reviews in the Cochrane Library that aimed to evaluate the efficacy of speech therapies were identified. Methodological weaknesses related to the small number of clinical trials, the use of appropriate methodology and sample size definition criteria led to a low level of scientific evidence regarding the effectiveness of treatments in various clinical settings17.

The designs of the clinical trials evaluated here considered the inclusion of a control group and are characterized as stage II clinical studies because they evaluate treatment effectiveness. This process provides sufficient information for the experiment to be replicated and permits the comparative analysis of treatment groups15. The magnitude of the experimental treatment effect compared with the control group enables the planning of later clinical trial stages, for example, stages III and IV.

The methodological limitations identified in the Brazilian studies include issues related to sample planning (sample size definition and allocation of research subjects), which directly affects the possibility of generalizing the results to the wider population. In this sense, the statistical analysis of data obtained from a non-representative portion of the population can lead to erroneous interpretation. To avoid this bias, it is necessary to consider statistical parameters related to the type of variable (discrete or continuous), the standard deviation of the variable in the population, the significance level of the estimate and the maximum sampling error tolerated. Statistical formulae for calculating sample sizes help to establish the appropriate number of individuals required for the study18.

Sample planning also incorporates procedures related to sampling techniques (e.g., the allocation of research subjects). Biostatistical parameters consider that the probabilistic method leads to the formation of a sample with greater representative power. Conversely, convenience allocation of specific outpatient subjects may not adequately represent the population18. Moreover, the decision to include or exclude a subject based on the defined inclusion and exclusion criteria should precede the randomization process19.

Randomization was not prioritized in all of the reviewed studies. The absence of this procedure may mean that the composition of treatment groups is biased, thus indicating an important selection bias. Randomization offers an individual the same change of participating in the control group as in the experimental group, making the groups more homogeneous and providing a balance of predictors of good and poor prognosis15,20,21.

Blinding was not considered in all of the evaluated studies, and when it was adopted, it did not apply to all of those involved in the research (researchers-examiners, research participants and statisticians). Blinding aims to minimize the conscious or unconscious influence of the involved parties on the study results, whether during the evaluation process or during the classification of identified effects; therefore, masking of the adopted procedures should be considered15,21,22.

The evaluation of the clinical trials' quality based on the Quality Checklist for RCTs and Observational Studies7 indicates that the trials fall into the category of regular studies, with an emphasis on the categories of external validity and power, which generally received low scores. This reflects the low generalizability of results caused by the failure to adopt detailed methodological procedures related to the issues described above.

The lack of instruments with acceptable sensitivity and specificity and well-established diagnostic criteria means that the studies' conclusions cannot be compared with concrete measures. Differences in methods for evaluating and classifying dysfunctions or disorders limit the external validity of the results, meaning that they are only representative of the investigated population.

Although it was not the object of analysis in this study, the importance of considering indicators that have been validated for the clinical evaluation of injuries related to outcomes of interest in the OM field must be emphasized. Such indicators should ideally meet the requirements of observational or experimental epidemiological research. That is, they must be validated for reproducibility and broad representativeness. This is related to the indicators’ ability to be applied to a large number of subjects and to meet ethical requirements in terms of not causing harm or damage to the investigated individuals23.

The establishment of standardized measures with acceptable reproducibility and sensitivity is the first step toward the significant advancement of research and clinical practice in the area of OM. With such definitions, similar studies can be analyzed in a comparative way, thus facilitating the application of evidence from scientific research in the field.

Conclusions

Despite advances in recent years, experimental research (clinical trials) in the OM field still have characteristics of incipient development. The use of research techniques with a good quality of evidence and the standardization and validation of methodological tools are essential steps toward evidence-based clinical practice. Multicenter studies may represent an important strategy for understanding the health problems related to the field at a population level, and the implementation of treatment approaches supported by exemplary scientific evidence will provide more effective health care that can promote quality of life for the population.

References

1. Resolução CFFa nº 310 de 17 de fevereiro de 2006. Dispõe sobre as especialidades reconhecidas pelo Conselho Federal de Fonoaudiologia e dá outras providências. Sect. 1 (2006). [ Links ]

2. Tomaz-Morais J, Lima JAS, Luckwü-Lucena BT, Batista AUD, Limeira RRT, Silva SM et al. Integral analysis of brazilian scientific production in orofacial myology: state of art and future perspectives. Rev. CEFAC. 2016;18(2):520-32. [ Links ]

3. Lof GL. Science-based practice and the speech-language pathologist. Int J Speech Lang Pathol. 2011;13(3):189-96. [ Links ]

4. Peccin A. Research in physicaltherapy: the evidence ground ed practice and study. Fisioter. pesqui. 2005;12(1):43-8. [ Links ]

5. Carvalho APV, Silva V, Grande AJ. Avaliação do risco de viés de ensaios clínicos randomizados pela ferramenta da colaboração Cochrane. Diagn Tratamento. 2013;18(1):38-44. [ Links ]

6. Alves MRM. A produção fonoaudiológica nacional em Motricidade Orofacial - 1970 a 2000. [Dissertação] Curitiba (PR): Faculdade de Ciências Biológicas e da Saúde, da Universidade Tuiuti do Paraná; 2002. [ Links ]

7. Kennelly J, Handler A, Peacock N. Methodological approach to assessing the evidence. Reducing racial/ethinic disparities in reproductive and perinatal outcomes: the evidence from population-based interventions. Springer Science + Business Media; 2011. [ Links ]

8. National Collaborating Centre for Methods and Tools: quality checklist for health care intervention studies. Hamilton, ON: McMaster University; 2008. [ Links ]

9. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled clinical trials. 1996;17(1):1-12. [ Links ]

10. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377-84. [ Links ]

11. Kristensen NB, Bryrup T, Allin KH, Nielsen T, Hansen TH, Pedersen O. Alterations in fecal microbiota composition by probiotic supplementation in healthy adults: a systematic review of randomized controlled trials. Genome Med. 2016;10(1):8-52. [ Links ]

12. Mittal N, Mittal R, Kumar H, Medhi B. Sodium glucose co-transporter 2 inhibitors for glycemic control in type 2 diabetes mellitus: quality of reporting of randomized controlled trials. Perspect Clin Res. 2016;7(1):21-7. [ Links ]

13. Khan S, Dennison A, Garcea G. Medical therapy for polycystic liver disease. Ann R CollSurg Engl. 2016;98(1):18-23. [ Links ]

14. Medronho R, Bloch KV, Luiz RR, Werneck GL. Epidemiologia. 2 ed. São Paulo: Atheneu; 2009. [ Links ]

15. Friedman LM, Furberg CD, DeMets DL. Fundamentals of clinical trials. 4 Edition, New York: Springer, 2006. [ Links ]

16. Friedman LM, Furberg CD, DeMets DL, Reboussin DM, Granger CB. Fundamentals of clinical trials (Vol. 4). New York: Springer. 2010. [ Links ]

17. El-Dib RP, Atallah AN. Evidence-based speech, language and hearing therapy and the Cochrane Library's systematic reviews. Sao Paulo Med J. 2006;124(2):51-4. [ Links ]

18. Miot HA. Tamanho da amostra em estudos clínicos e experimentais. J Vasc Bras. 2011;10(4):275-8. [ Links ]

19. Coutinho ESF, Cunha GM. Conceitos básicos de epidemiologia e estatística para a leitura de ensaios clínicos controlados. Rev Bras Psiquiatr. 2005;27(2):146-51. [ Links ]

20. Escosteguy CC. Tópicos metodológicos e estatísticos em ensaios clínicos controlados randomizados. Arq Bras Cardiol. 1999;72(2):139-43. [ Links ]

21. Nobre MRC, Bernardo WM, Jatene FB. A prática clínica baseada em evidências: Parte III avaliação crítica das informações de pesquisas clínicas. Rev Assoc Med Bras. 2004;50(2):221-8. [ Links ]

22. Chalmers TCH, Smith JR, Blackburn B. A method for assessing the quality of a randomized control trial. Controlled Clinical Trials. 1981;2(1):31-49. [ Links ]

23. Pereira MG. Epidemiologia: teoria e prática. Rio de Janeiro: Guanabara Koogan, 1995. [ Links ]

Received: February 26, 2018; Accepted: April 27, 2018

Mailing address: Rebecca Rhuanny Tolentino Limeira, Universidade Federal da Paraíba, Campus I, CEP: 58.051-900 - João Pessoa, Paraíba, Brasil, E-mail: rebecca.rhuanny@hotmail.com

Conflict of interests: Nonexistent

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License