Use of sample size calculation in scientific research in Brazilian Physical Education journals

Emprego do cálculo amostral em pesquisas científicas de periódicos nacionais de Educação Física

Abstracts

The mistaken use of biostatistics in scientific research involves methodological errors both in the research itself as in its analyses. Among these, the lack of sample size calculation, considered essential for validation and credibility of results, is often found. The aim of the study was to determine how often sample size calculation is used in articles published in Physical Education national journals. The study included only national scientific journals ranked as A1 and B2 by the Coordination of Improvement of Higher Education Personnel (CAPES), which analyzed only articles available in full and published in the period from 2010 to 2012. Review articles, letter to the editor, points of view and original articles that did not perform any statistical analysis were excluded. Approximately 15 % of articles analyzed performed sample size calculation and no difference between these proportions over the years were identified. Differences between Physical Education subareas (education, health and sport) were observed, being lower in sport (p = 0.001) compared to other subareas. Journals classified as B2 showed higher use of sample size calculation in relation to those classified as B1 (p = 0.013) and A2 (p = 0.007). The use of sample size calculation in scientific research published in Physical Education national journals in the period from 2010 to 2012 was not satisfactory, especially in sport subarea. Moreover, no evolution on the use of sample size calculation over the years analyzed was observed.

Physical education; Sample size; Statistics

O emprego equivocado da bioestatística nas pesquisas científicas implica em erros metodológicos tanto na condução quanto nas análises das mesmas. Dentre estes, a ausência do cálculo amostral, considerado primordial para validação e credibilidade dos resultados, é frequentemente encontrado. O objetivo do estudo foi verificar com que frequência o cálculo amostral é empregado nos artigos publicados em periódicos nacionais de Educação Física. Foram incluídos no estudo somente periódicos científicos nacionais classificados entre A1 e B2 pela Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), dos quais foram analisados somente artigos originais disponíveis na íntegra e publicados no período de 2010 a 2012. Foram excluídos do estudo artigos de revisão, carta ao editor, ponto de vista e artigos originais que não realizaram nenhum tipo de análise estatística. Aproximadamente 15% dos artigos analisados realizaram cálculo amostral; não houve diferença entre as proporções destes ao longo dos anos. Observou-se diferença entre as subáreas da Educação Física (educação, saúde e esporte), sendo menor na subárea esporte (p = 0,001) em relação às demais. Os periódicos classificados como B2 apresentaram maior uso do cálculo amostral em relação àqueles classificados como B1 (p = 0,013) e A2 (p = 0,007). O emprego do cálculo amostral em pesquisas científicas publicadas em periódicos nacionais de Educação Física no período de 2010 a 2012 não foi satisfatório, especialmente na subárea esporte. Além disso, não foi observada uma evolução do uso do cálculo amostral ao longo dos anos analisados.

Educação física; Estatística; Tamanho da amostra

INTRODUCTION

Like any area of ​​knowledge, the health area has also experienced a rapid assimilation of new technologies for the treatment of scientific data. The medical sciences have made wide use of mathematical and statistical knowledge in the interpretation of scientific problems¹. Research linked to the Physical Education area seems to follow the same trend.

The mistaken use of biostatistics in clinical investigation can occur when there is a gap between researchers and the area of exact sciences, usually caused by difficulties in the communication between areas. This barrier is due, at least in part, to the complexity and characteristic terms of statistical language, quite unusual in biological sciences². In addition, statistics is little tangible and dissociated from biological problems and reduced class hours, especially in courses not associated with area sciences, which are also responsible for increasing its understanding and application³. Accordingly, Silva et al.⁴ reported that the greater the understanding of statistical concepts by students, the greater the change in attitude towards statistics, positively reflecting on the understanding and use of such knowledge.

In this context, methodological errors in the conduction and analysis of research are often found. One of the mistakes often found in the methodology used in studies related to the health science area is the absence of prior sample size calculation, considered fundamental methodology in the development of most types of scientific research¹. Thus, researchers in the area of humanities and health sciences have made inappropriate use of inferential statistical procedures⁵.

The sample size directly influences the interpretation and statistical validity of a result found^{6 , 7}. When the sample size is calculated, a suitable number of observations for certain variable can be estimated¹. Thus, the probability of error in the interpretation of the final result is minimized. Moreover, in the absence of sample size calculation, the research protocol can be performed with insufficient n, reducing its statistical power and increasing the likelihood of misinterpretations^{8 , 9}.

In the health area, where physical education is included, most studies involve the participation of humans or animal experimentation as the sampling unit. Accordingly, due to the biological variability inherent to this type of sample unit, sample size calculation becomes essential¹⁰. Likewise, it is noteworthy that not only biological outcomes (psychological, behavioral variables, etc) require sample size calculation. However, despite its methodological importance, no studies investigating the use of sample size calculation in scientific research related to the national Physical Education area were found. Therefore, the aim of this study was to determine how often sample size calculation is used in the articles published in Physical Education national journals.

METHODOLOGICAL PROCEDURES

Information survey

The study included only national scientific journals ranked as A1 and B2 by the Coordination of Improvement of Higher Education Personnel (CAPES), which analyzed only articles available in full and published in the period from 2010 to 2012. The present study included the following journals: Movimento (A2), Motriz: Revista de Educação Física (A2), Revista Brasileira de Medicina do Esporte (A2), Revista Brasileira de Ciências do Esporte (B1), Revista Brasileira de Educação Física e Esporte (B1), Revista Brasileira de Cineantropometria e Desempenho Humano (B1), Revista Brasileira de Atividade Física e Saúde (B2), Revista Brasileira de Ciência e Movimento (B2) and Revista da Educação Física/UEM (B2). The selection of journals was performed by Qualis-CAPES 2011 listing in the Physical Education area (area 21). A flowchart of the process of inclusion and exclusion of journals in the study is shown in Figure 1.

Figure 1
Flowchart of the process of inclusion and exclusion of journals in the study.

The analysis of published articles was performed by five experienced evaluators graduated in Physical Education (Master and PhD students). Initially, a thorough reading of the introduction and abstract was conducted to rank them subjectively into one of three Physical Education subareas (education, health or sport) in accordance with its objectives and contents (main theme). Articles that fit into more than one subarea were classified by the predominance of content and greater focus on one of the subareas. Subsequently, it was observed if the article described some statistical analysis of results. If so, a thorough reading of methods and results was performed by a single examiner in order to determine whether sample size calculation was performed. The presence of sample size calculation was defined by its description or not throughout the article. There was no judgment on the level of appropriateness of the sample size calculation, statistical methods used and / or reliability of results. Review articles, letter to the editor, points of view and original articles that did not perform any statistical analysis were excluded.

Statistical Analysis

Data description was performed by absolute and relative frequency. The chi- square test (2x2 contingency tables and linear trend) and binary logistic regression (expressed as odds ratio [OR] values and confidence intervals of 95 % [95% CI]) were used to compare proportions. P-values ​​< 0.05 were considered significant. The Biostat 5.0 statistical software was used for analyses.

RESULTS

The absolute quantification of articles published in the journals analyzed is shown in Table 1 considering: journal, Qualis-CAPES and publication year.

Table 1
Absolute quantification of articles included (I) and excluded (E) in the analysis of the use of sample size calculation in the national literature related to the Physical Education area.

Journal	Qualis*	Publication year						Total
		2010		2011		2012
		I	E	I	E	I	E
Movimento	A2	1	46	2	35	3	38	125
Motriz	A2	41	39	40	31	49	25	225
RBME	A2	69	20	71	16	76	7	259
RBCE	B1	30	28	25	38	36	35	192
RBEFE	B1	37	10	41	23	38	28	177
RBCDH	B1	58	15	59	14	61	12	219
RBAFS	B2	30	12	42	18	43	21	166
RBCM	B2	40	21	33	22	45	15	176
REF	B2	34	26	40	20	26	20	166
Total		340	217	353	217	377	201	1705

• ^*
  : Qualis Capes listing of 2011
• ^RBME
  : Revista Brasileira de Medicina do Esporte
• ^RBCE
  : Revista Brasileira de Ciências do Esporte
• ^RBEFE
  : Revista Brasileira de Educação Física e Esporte
• ^RBCDH
  : Revista Brasileira de Cineantropometria e Desempenho Humano
• ^RBAFS
  : Revista Brasileira de Atividade Física e Saúde
• ^RBCM
  : Revista Brasileira de Ciência e Movimento
• ^REF
  : Revista da Educação Física/UEM.

The use the sample size calculation had a prevalence of 15.2 % (95% CI: 13.1% - 17.4%) in the analyzed period (2010-2012). There was no difference (p = 0.862) in use (%) of sample size calculation between years of 2010, 2011 and 2012 (Figure 2).

Figure 2
Use (%) of sample size calculation in studies in the Physical Education area in the period between 2010 and 2012

However, differences were observed between Physical Education subareas (education, health and sport) (Figure 3), in which the use (%) of sample size calculation in health (OR = 10.1 [5.27-19.5], p = 0.001) and education subareas (OR = 7.37 [2.62-20.6], p = 0.001) was higher than in the sport subarea.

Figure 3
Use (%) of sample size calculation in studies included in the Physical Education subareas between 2010 and 2012. a Significant difference in relation to Education. b Significant difference in relation to Health

Differences in use (%) of sample size calculation were found between Qualis - CAPES (A2, B1, B2) in which they were included. Journals ranked as B2 showed higher use (%) of the sample size calculation in relation to those ranked as B1 (RC = 00:50 [0.33-0.75], p = 0.001) and A2 (RC = 00:48 [0.31-0.72], p = 0.001) (Figure 4).

Figure 4
Use (%) of sample size calculation in accordance with the Qualis-CAPES of journals. a Significant difference compared to A2. b Significant difference compared to B1

DISCUSSION

The present study showed no significant difference between publication years in the proportions of use of the sample size calculation in national journals. There are no studies in literature with similar proposal in the ​​Physical Education area, which impairs the comparison of our findings with those of other studies. Moreover, it is noteworthy that the non increase in the use of sample size calculation in the last three years identifies limitation without current trend of evolution.

In assessing the Physical Education subareas (Education, Health and Sport), the number of studies investigating the use of appropriate statistic is greater in the health subarea. This difference is attributed, at least in part, to the proximity of these researchers in medical literature, which has strongly suggested the use of this procedure ^{11 - 13}. Still in the Health area, Charles et al.¹⁴ compared the years of 1980 and 2002 and reported a 79% increase in the number of studies using sample size calculation in international journals of high impact factor. However, they found that the parameters required for sample size calculation are often absent in studies. In addition, when sample size was recalculated in these studies, two-thirds had errors. Zlowodzky and Bhandari¹⁵ observed a tendency to use small samples in most studies and reported that ethical and methodological issues are also involved. This sample size restriction has the effect of lowering the statistical power, decreasing the capacity of proving the hypotheses being tested.

In the Education area, Grácio and Garrutti¹⁶ reported the need to bring qualitative and quantitative approaches close to each other in order to enable a more comprehensive visualization of the results and strengthen their arguments, often supported only on qualitative analyses. Grácio and Garrutti¹⁷ also found that only 18 % of articles published in the education area performed statistical analyses. Among these, only 23 % worked with samples. In our research, the number of studies in this area excluded for lack of statistical analysis followed this trend; however, from a minority included in the analysis, approximately 15 % performed sample size calculation.

There are no studies questioning the non-use of sample size calculation in research conducted in the sport area. This can be justified due to the fact that studies with homogeneous samples (low variability) are sufficient for the application of statistical tests with safety¹⁸, as is the case of research with athletes. In this sense, Mourão-Junior¹⁸ claim that it is not the sample size that guarantees good results, but its quality, i.e., its ability to represent the population studied. On the other hand, Charles et al.¹⁴ suggest that in studies that do not use sample calculation, samples are often based on inaccurate and arbitrary assumptions, mainly driven by the study feasibility (convenience), resulting in inaccurate interpretations of results.

Regarding the Qualis - CAPES classification, it was expected that in the best classified journals, the number of articles published using sample size calculation was greater. However, the findings of this study indicate that this statistical method was used more frequently in journals ranked as B2. In agreement with these results, Andrade and Abreu-e-Silva¹⁹ reported that even in reputable journals with excellent editorial board, most of the times the application of the method and / or sample calculation description is not mentioned in the articles, which is an important methodological error⁹. Moreover, Andrade and Abreu-e-Silva¹⁹ reported that there is a great tendency of scientific journals not to value the presence of the sample calculation as well as the correct description of the calculation methodology. However, in this study, this trend was not confirmed by analyzing national Physical Education journals in the period between 2010 and 2012. Nevertheless, this result can be explained by the higher proportion of articles focused on health published in journals classified as B2, since this subarea showed the highest relative frequency of use of sample calculation (22.1 %), and in additional analyses, the association between the outcome and the Qualis - CAPES lost significance when adjusted for the health subarea. Moreover, in the Physical Education area, there is a process of "induction" of some journals in higher strata of the Qualis - CAPES (even not fulfilling all the requirements for such inclusion, the area chooses some journals and ranks them into higher strata [A2]), which is a process that apparently needs to be further discussed.

Although it is an important tool in the process of scientific research, the use of sample size calculation needs to be analyzed from other perspectives. For example, studies with animal models typically do not use sample calculation because they understand that the strict control (genetic, food, etc) that these samples are submitted leads to the reduction of variability of results and thus directly affects the power to detect differences¹⁸. Epidemiological studies covering an entire population are another example²⁰, in which, for obvious reasons, sample size calculation does not apply (however, they are scarce in our area). Additionally, it is noteworthy that only the proper use of sample size calculation does not guarantee the quality of the research, since other methodological procedures such as the quality of the random sampling and the eligibility criteria are essential^{7 , 18}. In fact, trying to calculate a priori (before the start of the experiment), the ideal sample size for this to be representative of the entire population being studied may be impractical in some cases and useless. For example, the study of Mourão - Junior¹⁸ indicates that determining the sample size is a dynamic process that may change as data are collected and analyzed.

One limitation of the study worth mentioning is the absence of a more detailed analysis of the sample calculations used, i.e., it was not possible to determine whether it was done correctly. Furthermore, the study period (three years) was not long enough to establish a perspective on the use of the sample size calculation in national Physical Education scientific journals.

CONCLUSION

In short, it was concluded that the occurrence of sample size calculation in scientific studies published in national journals in the Physical Education area in the period from 2010 to 2012 is low, especially in the sport subarea, showing no evolution in recent years and no positive relationship with the higher strata of the journal in the Qualis - CAPES.

REFERENCES

• ¹
  Pereira JCR. Tamanho de amostra: uma necessidade real ou um capricho cultural? Arteríola 2002;4(1):13-6.
• ²
  Watts DG. Why is introductory statistics difficult to learn? What can we do to make it easier? The American Statistician 1991;45(4):290-1.
• ³
  Metz AM. Teaching statistics in biology: using inquiry-based learning to strengthen understanding of statistical analysis in biology laboratory courses. CBE Life Sci Educ 2008 Fall;7(3):317-26.
• ⁴
  Silva CB, Brito MRF, Cazorla IM, Vendramini CMM. Atitudes em relação à estatística e à matemática. Psico-USF 2002;7(2):219-28.
• ⁵
  Minayo MCS, Sanches O. Quantitative and qualitative methods: opposition or complementarity? Cad Saúde Pública 1993;9(3):239-62.
• ⁶
  Torgerson DJ, Miles JN. Simple sample size calculation. J Eval Clin Pract 2007;13(6):952-3.
• ⁷
  Miot HA. Tamanho da amostra em estudos clínicos e experimentais. J Vasc Bras 2011;10(4):275-8.
• ⁸
  Azevedo RS. Qual o tamanho da amostra ideal para se realizar um ensaio clínico? Rev Assoc Med Bras 2008;54(4):283-97.
• ⁹
  Normando D, Almeida MAO, Quintão CCA. Análise do emprego do cálculo amostral e do erro do método em pesquisas científicas publicadas na literatura ortodôntica nacional e internacional. Dental Press J Orthod 2011;16(6):33.e1-9.
• ¹⁰
  Weyne GRS. Determinação do tamanho da amostra em pesquisas experimentais na área de saúde. Arq Méd ABC 2004;29(2):87-90.
• ¹¹
  Strasak AM, Zaman Q, Pfeiffer KP, Göbel G, Ulmer H. Statistical errors in medical research--a review of common pitfalls. Swiss Med Wkly 2007;137:44-9.
• ¹²
  Machin D, Campbell M, Fayers P, Pinol A. Sample size tables for clinical studies. 2nd ed. Oxford: Blackwell Science; 1997.
• ¹³
  Schulz KF, Grimes DA. Sample size calculations in randomized trials: mandatory and mystical. Lancet 2005;365(9467):1348-53.
• ¹⁴
  Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P. Reporting of sample size calculation in randomized controlled trials: review. BMJ 2009;338:b1732.
• ¹⁵
  Zlowodzky M, Bhandari M. Outcome measure and implications for sample size calculation. J Bone Joint Surg Am 2009;91(Suppl 3):35-40.
• ¹⁶
  Grácio MCC, Garrutti EA. Estatística aplicada à educação: uma análise de conteúdos programáticos de planos de ensino e de livros didáticos. Rev Mat Estat 2005;23(3):107-26.
• ¹⁷
  Grácio MCC, Garrutti EA. A disciplina estatística na área de educação: seleção e organização de conteúdos. Educ Matem Rev 2003;5:12-20.
• ¹⁸
  Mourão-Junior CA. Questões em bioestatística: o tamanho da amostra. Rev Interdisciplin Estud Exp Anim Hum 2009;1(1):26-8.
• ¹⁹
  Andrade MAP, Abreu-e-Silva GM. Como calcular o tamanho da amostra. In: Cristante AF, Kfuri M. Como escrever um trabalho científico. São Paulo: Sociedade Brasileira de Ortopedia e Traumatologia (SBOT); 2010.
• ²⁰
  Hartwig TW, da Silva MC, Reichert FF, Rombaldi AJ. Condições de saúde de trabalhadores de academias da cidade de Pelotas-RS: um estudo de base populacional. Rev Bras Ativ Fis Saude 2012;17(6):500-11.

Publication Dates

• Publication in this collection
  Sept-Oct 2014