Acessibilidade / Reportar erro

To P or not to P, is that the question? Rethinking experimental design and data analysis to improve biological significance beyond the statistical significance

In a assay published in Nature, Valentin Amrhein (University of Basel) and its colleagues Sander Greenland (University of California) & Blake McShane (Northwestern University), present a series of arguments against the stablished “P value-based statistical significance dichotomania”.11- Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305-7. The authors use some insightful practical examples, such as data related to unintended effects of anti-inflammatory drugs risks, and consider the influence of human cognitive trend to simplistically bucket results into “statistically significant” and “statistically non-significant” categories and to consider it definitely different.

Importantly, the essay clearly states that the authors are not advocating a ban on P values or statistical measures, but that P values should not be treated categorically or to support dichotomization as statistically significant or not. Similarly, a 2016′ statement of the American Statistical Association22- Wasserstein RL, Lazar NA. The ASA's statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129-31. warns against the misuse of statistical significance and P values, which include as a recommendation “don't say statistically significant”. On the other hand, John P. A. Ioannidis (Stanford University), argue that “retiring statistical significance would give bias a free pass' and that ‘irrefutable nonsense would rule’”.33- Ioannidis JP. Retiring statistical significance would give bias a free pass. Nature. 2019;567(7749):461. The author states that dichotomous conclusions can be useful for pinning down discoveries in different fields, but the analysis of effect sizes “can often be better than determining whether an effect exists”.

While such discussion is at least provoking and mind challenging, this Editorial goal is to take advantage of the “statistical dogma” questioning to draw some attention to steps that precede the statistical analysis and the generation of a given P value. Indeed, a proper study design can even improve the statistical findings power in a measurable way, but also several aspects of study design and the subsequent data analysis may have a significant (forget P values for a moment), but not statistically quantifiable, impact in the data “significance”. The big mistake would be the overvaluation of statistical analysis methods with the undervaluation of experimental design. In the sequence, some practical examples (derived from our research group data) will be used to illustrate how study design can increase the both “statistical significance” and “biological significance”.

Genetic studies are usually based in a classic control approach, were controls and subjects presenting a given condition are compared in regards of the occurrence and frequency of genetic variants. In this context, the P value derived from the unaffected and affected individuals' comparison is essential to draw any conclusion. In such studies, the number of individual in such groups, but also the frequency of the target genetic variation, the frequency of the studied condition, impact the study power and the determination of the P value, and evidently, the conclusions derived from such data. However, experimental design features, apparently incomputable in the study power determination, can also present a significant impact in the analysis outcome. Thus, in this situation, stratified sampling considering these possible confounding factors could balance the study groups and minimize the effect of external variables on the final data analysis.

In the periodontal genetic studies, generally affected individuals (presenting some form of periodontitis) are compared with periodontally healthy subjects.44- Trevilatto PC, Souza Pardo AP, Scarel-Caminaga RM, Brito RB Jr, Alvim-Pereira F, Alvim-Pereira CC, et al. Association of IL1 gene polymorphisms with chronic periodontitis in Brazilians. Arch Oral Biol. 2011;56(1):54-62.77- Schäfer AS, Jepsen S, Loos BG. Periodontal genetics: a decade of genetic association studies mandates better study designs. J Clin Periodontol. 2011;38(2):103-7. However, in this context, the possibility to control microbial exposure by oral hygiene methods interfere with the exposure factor, and consequently a periodontally healthy population is comprised by subjects that properly perform oral hygiene methods as a routine. Therefore, irrespective of the putative susceptible and resistant genotypic nature, such subjects will not develop the disease phenotype due the proper oral care. Indeed, such unique feature clearly differ from the usual characteristics of infectious diseases genetic studies, where affected and unaffected individuals are typically recruited from endemic areas where groups are naturally exposed to a pathogenic challenge, and the resistant and susceptible phenotypes are consequently exposed.88- Zondervan KT, Cardon LR, Kennedy SH. What makes a good case-control study? Design issues for complex traits such as endometriosis. Hum Reprod. 2002;17(6):1415-23.,99- Sakthianandeswaren A, Foote SJ, Handman E. The role of host genetics in leishmaniasis. Trends Parasitol. 2009;25(8):383-91. Therefore, the absence of the microbial factor in a periodontally healthy population, clearly disregard the case-control study architype, which determine if an exposure is associated with an outcome.88- Zondervan KT, Cardon LR, Kennedy SH. What makes a good case-control study? Design issues for complex traits such as endometriosis. Hum Reprod. 2002;17(6):1415-23.,99- Sakthianandeswaren A, Foote SJ, Handman E. The role of host genetics in leishmaniasis. Trends Parasitol. 2009;25(8):383-91.

In other words, the absence of an archetypical control with a defined resistance phenotype may limit the odds of the identification of genotypic differences when compared with a susceptible group. In order to adapt the study design to the exposure concept, in periodontitis genetic studies the control group should comprise a microbially exposed group with a distinct phenotypic outcome than chronic periodontitis.1010- Garlet GP, Trombone AP, Menezes R, Letra A, Repeke CE, Vieira AE, et al. The use of chronic gingivitis as reference status increases the power and odds of periodontitis genetic studies: a proposal based in the exposure concept and clearer resistance and susceptibility phenotypes definition. J Clin Periodontol. 2012;39(4):323-32. Such features can be found in individuals presenting chronic gingivitis, characteristically exposed to a periodontal microbial challenge associated with a reversible low severity disease form characterized by the absence of attachment, which in theory, represent “resistance” phenotype/genotype. Indeed, the “resistant versus susceptible” phenotype analysis, when compared to the traditional “healthy versus diseased” approach, significantly impacted the study power and odds of identification of genetic factors involved in PD.1010- Garlet GP, Trombone AP, Menezes R, Letra A, Repeke CE, Vieira AE, et al. The use of chronic gingivitis as reference status increases the power and odds of periodontitis genetic studies: a proposal based in the exposure concept and clearer resistance and susceptibility phenotypes definition. J Clin Periodontol. 2012;39(4):323-32. The overall impact in the study power was the boost to >85% of a previous <30% power, while the overall odds ratio values seems to double in this approach; being such impact derived from the proper observation of the archetypical study design, supported by exposure-based phenotypes determination, comprehensively used in infectious diseases genetic studies.

Phenotypic variation can also be a supporting factor for data analysis and interpretation in order to determine the possible involvement of a given factor in a pathological process. Still in periodontitis context, molecules that control osteoclastogenesis process have been regarded as potential determinants of disease onset, progression and severity. In this framework, the osteoclastogenic factor RANKL and its endogenous inhibitor OPG, have been in the focus of numerous studies in the field over the last years.1111- Garlet GP. Destructive and protective roles of cytokines in periodontitis: a re-appraisal from host defense and tissue destruction viewpoints. J Dent Res. 2010;89(12):1349-63. Briefly, RANKL levels are supposed to be locally upregulated by chronic inflammatory immune response elements, leading to the bone resorption that characterizes periodontitis, while anti-inflammatory mediators present a counteracting effect via OPG upregulation. A common approach in host factors focused studies is to comparatively measure the coding mRNA or protein levels, or to perform the staining of cells positive for such targets, in diseased and healthy tissues. However, such approach can be also limiting in terms of data interpretation and “biological significance” by the reasons similar to those highlighted for the genetic studies.

Figure 1
Exposure factor vs phenotype/genotype determination

A healthy tissue represents homeostasis and a diseased tissue represents pathology. However, both allegedly destructive (RANKL) and protective (OPG) factors are upregulated in diseased tissues, similarly to numerous inflammatory and also anti-inflammatory mediators.1111- Garlet GP. Destructive and protective roles of cytokines in periodontitis: a re-appraisal from host defense and tissue destruction viewpoints. J Dent Res. 2010;89(12):1349-63.,1212- Garlet GP, Martins W Jr, Fonseca BA, Ferreira BR, Silva JS. Matrix metalloproteinases, their physiological inhibitors and osteoclast factors are differentially regulated by the cytokine profile in human periodontal disease. J Clin Periodontol. 2004;31(8):671-9. Indeed, one could argue how increased levels of anti-inflammatory and anti-osteoclastogenic factors, associated to “highly significative” P values (in healthy vs diseased tissues comparison), are presented in inflamed and diseased periodontal sites? It is also important to consider that if a given molecule, with unknown role, would identified as upregulated in diseased periodontal tissue, the immediate assumption regarding its role in the disease pathogenesis probably will be to label such molecule as “destructive”.

In this scenario, the use of additional distinct and phenotypes can allow additional data analysis and provide some insightful information about how host inflammatory immune response mediators can impact periodontitis outcome. One possible approach is to compare periodontitis variants, such as aggressive and chronic, each one characterized by its unique features, such as early vs late onset and different progression rates.1212- Garlet GP, Martins W Jr, Fonseca BA, Ferreira BR, Silva JS. Matrix metalloproteinases, their physiological inhibitors and osteoclast factors are differentially regulated by the cytokine profile in human periodontal disease. J Clin Periodontol. 2004;31(8):671-9. In such comparison, it is possible to observe variations in RANKL/OPG ratio, which can explain the possible variations between the forms, but the sole comparison of each form with healthy tissue would not allow such inference, despite the “very significant” P value, “more significant” than the aggressive vs chronic comparison. Importantly, the comparison on distinct disease forms points to a differential balance in the levels of pro- vs anti-osteoclastogenic and pro- vs anti-inflammatory mediators as determinants of disease outcome. However, the determination of the “tipping point” that separates homeostasis from pathology may require additional approaches, which will be explored further below.

One may argue that phenotypic variation may not be necessary, since it would be possible to perform some severity disease stratification within an individual disease form, based in clinical readouts such as bleeding on probing, probing depth and attachment loss. However, it is mandatory to consider the intrinsic limitations imposed by periodontitis features, which include the cumulative nature of disease severity, and the uncertainness about disease progression patterns, impose critical limitations to data interpretation.1313- Gilthorpe MS, Zamzuri AT, Griffiths GS, Maddick IH, Eaton KA, Johnson NW. Unification of the “burst” and “linear” theories of periodontal disease progression: a multilevel manifestation of the same phenomenon. J Dent Res. 2003;82(3):200-5.,1414- Nomura Y, Morozumi T, Nakagawa T, Sugaya T, Kawanami M, Suzuki F, et al. Site-level progression of periodontal disease during a follow-up period. PloS One. 2017;12(12):e0188670. Importantly, the definition of active bone resorption is quite complex in periodontitis, which seems to progress via active disease bursts followed by quiescent periods, being the current clinical tools ineffective in the determination of the actual status of disease during the analysis/sampling, and even to determine the progression model.1111- Garlet GP. Destructive and protective roles of cytokines in periodontitis: a re-appraisal from host defense and tissue destruction viewpoints. J Dent Res. 2010;89(12):1349-63.,1414- Nomura Y, Morozumi T, Nakagawa T, Sugaya T, Kawanami M, Suzuki F, et al. Site-level progression of periodontal disease during a follow-up period. PloS One. 2017;12(12):e0188670. Indeed, a 9 mm deep periodontal pocket in a supposed stable/remission state can present distinct features of a 4 mm deep conjectural active site. Indeed, in general, “immediate” parameters such as bleeding on probing and/or probing depth have been found to not be directly correlated with cytokines, irrespectively of its pro-inflammatory (where a positive correlation could be expected) and/or with anti-inflammatory features (where a negative correlation could be expected).1515- Trombone AP, Cardoso CR, Repeke CE, Ferreira SB Jr, Martins W Jr, Campanelli AP, et al. Tumor necrosis factor-alpha −308G/A single nucleotide polymorphism and red-complex periodontopathogens are independently associated with increased levels of tumor necrosis factor-alpha in diseased periodontal tissues. J Periodontal Res. 2009;44(5):598-608.1818- Claudino M, Trombone AP, Cardoso CR, Ferreira SB Jr, Martins W Jr, Assis GF, et al. The broad effects of the functional IL-10 promoter-592 polymorphism: modulation of IL-10, TIMP-3, and OPG expression and their association with periodontal disease outcome. J Leukoc Biol. 2008;84(6):1565-73. A similar picture can be observed in periapical lesions, where the lesion size does not necessarily correlates with a “higher” activity signature (i.e. expression of tissue destructive mediators) than smaller lesions (at the moment of sample collection).1919- Silva TA, Garlet GP, Lara VS, Martins W Jr, Silva JS, Cunha FQ. Differential expression of chemokines and chemokine receptors in inflammatory periapical diseases. Oral Microbiol Immunol. 2005;20(5):310-6.2222- Garlet GP, Horwat R, Ray HL Jr, Garlet TP, Silveira EM, Campanelli AP, et al. Expression analysis of wound healing genes in human periapical granulomas of progressive and stable nature. J Endod. 2012;38(2):185-90.

Therefore, the comparison of periodontitis-derived data with other conditions can also support better interpretation of RANKL/OPG ratio association with active or inactive bone resorption. While the definition of active bone resorption is complex in periodontitis, in orthodontic tooth movement such patterns are more straightforwardly distinguishable.2323- Taddei SR, Moura AP, Andrade I Jr, Garlet GP, Garlet TP, Teixeira MM, et al. Experimental model of tooth movement in mice: a standardized protocol for studying bone remodeling under compression and tensile strains. J Biomech. 2012;45(16):2729-35.2525- Taddei SR, Queiroz-Junior CM, Moura AP, Andrade I Jr, Garlet GP, Proudfoot AE, et al. The effect of CCL3 and CCR1 in bone remodeling induced by mechanical loading during orthodontic tooth movement in mice. Bone. 2013;52(1):259-67. Categorically, the bone resorption activity is a hallmark of pressure side, and can be comparatively analyzed in the tension side counterpart, where the bone formation activity prevail.2626- Andrade I Jr, Taddei SR, Garlet GP, Garlet TP, Teixeira AL, Silva TA, et al. CCR5 down-regulates osteoclast function in orthodontic tooth movement. J Dent Res. 2009;88(11):1037-41.2828- Garlet TP, Coelho U, Silva JS, Garlet GP. Cytokine expression pattern in compression and tension sides of the periodontal ligament during orthodontic tooth movement in humans. Eur J Oral Sci. 2007;115(5):355-62. Such data can provide a theoretical cutoff or threshold value that distinguish presence of absence of active bone resorption, which can be applied to periodontitis or other inflammatory osteolytic conditions, such as periapical lesions, to support additional analysis or assumptions.2222- Garlet GP, Horwat R, Ray HL Jr, Garlet TP, Silveira EM, Campanelli AP, et al. Expression analysis of wound healing genes in human periapical granulomas of progressive and stable nature. J Endod. 2012;38(2):185-90.,2929- Araujo-Pires AC, Francisconi CF, Biguetti CC, Cavalla F, Aranha AM, Letra A, et al. Simultaneous analysis of T helper subsets (Th1, Th2, Th9, Th17, Th22, Tfh, Tr1 and Tregs) markers expression in periapical lesions reveals multiple cytokine clusters accountable for lesions activity and inactivity status. J Appl Oral Sci. 2014;22(4):336-46.,3030- Colavite PM, Cavalla F, Garlet TP, Azevedo MC, Melchiades JL, Campanelli AP, et al. TBX21-1993T/C polymorphism association with Th1 and Th17 response at periapex and with periapical lesions development risk. J Leukoc Biol. 2019;105(3):609-19.

In an additional example on how the use of distinct phenotypes can provide “biological significance” that can overcome “statistical significance” in providing data interpretation support, let's consider that a family of molecules, collectively called SOCS. SOCS are intracellular proteins that acts as suppressors of inflammatory cytokine signaling, and therefore, are putatively involved in the control of inflammatory response in periodontitis, which ultimately interfere in the RANKL/OPG ratio modulation and in the disease outcome.2121- Menezes R, Garlet TP, Trombone AP, Repeke CE, Letra A, Granjeiro JM, et al. The potential role of suppressors of cytokine signaling in the attenuation of inflammatory reaction and alveolar bone loss associated with apical periodontitis. J Endod. 2008;34(12):1480-4.,3131- Souza JA, Nogueira AV, Souza PP, Cirelli JA, Garlet GP, Rossa C Jr. Expression of suppressor of cytokine signaling 1 and 3 in ligature-induced periodontitis in rats. Arch Oral Biol. 2011;56(10):1120-8. A study demonstrate that SOCS levels are upregulated in diseased tissues when compared with healthy ones, associated with a “very significative” P value (P<0.001).3232- Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4. Similarly to previously mentioned to OPG and anti-inflammatory mediators, one could argue how increased levels of inflammation suppressors, associated to “highly significative” P values (in healthy vs diseased tissues comparison), are presented in inflamed and diseased periodontal sites? However, such study also demonstrates that SOCS levels are higher in chronic gingivitis than in chronic periodontitis, which provides some interesting additional biological clue, but such association present a “less significative” P value (P<0.05) than the healthy vs diseased approach.3232- Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4. Based in the initial comparison, it is possible to recognize that SOCS are generally absent in healthy tissues being upregulated in response to inflammation. However, the second scenario allow us to infer that a more pronounced upregulation in chronic gingivitis could suggest a more efficient suppressive feedback, which could account for some phenotypic variation between gingivitis and periodontitis. Even considering that the P value from “healthy vs diseased” analysis (P<0.001) is higher than in the “phenotypic variation” comparison (P<0.05), the second comparison may be biologically “more significant” or more relevant for data interpretation.3232- Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4. It is also important to note that oftentimes, a result that indicates statistically significant differences has little or no biological impact. Therefore, it is essential that the researcher knows that although methodologically important in study, above the statistics should be his biological knowledge and interpretation about the results. In this context, it is possible to consider that the “biological significance” can overcome the “statistical significance” in providing support the data interpretation, allowing a broader picture of the immunoregulatory scenario.

Indeed, the analysis of data generated from “phenotypic variation” other than the simple “healthy vs diseased” dichotomy allows a series of correlation analysis that would result in false positive results in the “healthy vs diseased” analysis. Please remember that diseased periodontal tissues are characterized by high levels of theoretically destructive elements, such as osteoclastogenic factors (including RANKL) and pro-inflammatory molecules, but also for high levels of supposedly protective elements, such as OPG and anti-inflammatory molecules (such as IL-10), when compared to healthy tissues. Since a high variation in the levels of such molecules is present between health and disease conditions, correlation analysis including samples from both groups presents a high trend to biased “false correlations”.2121- Menezes R, Garlet TP, Trombone AP, Repeke CE, Letra A, Granjeiro JM, et al. The potential role of suppressors of cytokine signaling in the attenuation of inflammatory reaction and alveolar bone loss associated with apical periodontitis. J Endod. 2008;34(12):1480-4.,3232- Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4. It is known from experimental studies (whose importance will be considered in the sequence) that IL-10 induces OPG upregulation and RANKL downregulation. However, a ”healthy vs diseased” correlation analysis can result in positive correlations between IL-10 and OPG, but also between IL-10 and RANKL, with “very significative R and P values” (unpublished data). When such correlation is performed observing the “phenotypic variation”, or performed only with a single disease from samples, the positive correlation between IL-10 and OPG is sustained, but with “less significative R and P values”. Additionally, such analysis also reveals that IL-10 and RANKL are non-correlated, being RANKL levels actually correlated with pro-inflammatory mediators (unpublished data); being this data in accordance with IL-10 properties previously mentioned.2121- Menezes R, Garlet TP, Trombone AP, Repeke CE, Letra A, Granjeiro JM, et al. The potential role of suppressors of cytokine signaling in the attenuation of inflammatory reaction and alveolar bone loss associated with apical periodontitis. J Endod. 2008;34(12):1480-4.,3232- Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4. Also, similar patterns (positive correlations between IL-10 and OPG) were observed when lesions are stratified as theoretically active of inactive based in RANKL/OPG threshold, being IL-10 and OPG expression predominant in inactive lesions.2222- Garlet GP, Horwat R, Ray HL Jr, Garlet TP, Silveira EM, Campanelli AP, et al. Expression analysis of wound healing genes in human periapical granulomas of progressive and stable nature. J Endod. 2012;38(2):185-90.,2929- Araujo-Pires AC, Francisconi CF, Biguetti CC, Cavalla F, Aranha AM, Letra A, et al. Simultaneous analysis of T helper subsets (Th1, Th2, Th9, Th17, Th22, Tfh, Tr1 and Tregs) markers expression in periapical lesions reveals multiple cytokine clusters accountable for lesions activity and inactivity status. J Appl Oral Sci. 2014;22(4):336-46.,3030- Colavite PM, Cavalla F, Garlet TP, Azevedo MC, Melchiades JL, Campanelli AP, et al. TBX21-1993T/C polymorphism association with Th1 and Th17 response at periapex and with periapical lesions development risk. J Leukoc Biol. 2019;105(3):609-19.,3333- Malaspina TS, Gasparoto TH, Costa MR, Melo EF Jr, Ikoma MR, Damante JH, et al. Enhanced programmed death 1 (PD-1) and PD-1 ligand (PD-L1) expression in patients with actinic cheilitis and oral squamous cell carcinoma. Cancer Immunol Immunother. 2011;60(7):965-74.

Moreover, when biological events are evaluated such as those mentioned above, it may be naive to trust that a single independent variable would be responsible for the outcome response evaluated. Correlation analyzes may present significant p values between OPG/RANKL with IL-10, but most likely are not the only factors influencing the such ratio values. Therefore, any consideration about the subject that is not based on a broader spectrum of possible influencing factors (independent variables) is naturally weakened, even if based on significant p values. Thus, for biological analyzes involving complex mechanisms, multiple regression models (linear, logistic, Poisson, etc.) could be extremely useful in assessing the impact of a number of independent variables, such as age, gender, ethnicity, oral hygiene, specific clinical variables, presence or absence (and quantity) of a series of inflammatory mediators and pathogenic microorganisms, genetic variants, among others in the analysis of a simple outcome such as having or not having the disease. However, due to the biological complexity involved in this example, the list of independent variables could be infinite if it had no methodological impact either. For regression models to be robust in data analysis, for each independent variable included in the research, in a very simplistic way, 10 to 15 sample units (animals, patients, etc.) should ideally be included in the study. However, in some cases this theory may be beautiful on paper but impossible in practice as it could make a methodologically unviable study due to the large minimum sample size required.

Also, it is mandatory to consider that dichotomization or comparisons based in phenotypic data completely differ from dichotomization or stratification based in random scores frequently attribute to the analysis process. Indeed, it is common to receive in JAOS the submission of papers comprising the use of percentage scores, derived from the quantification of cells positively stained for a given target, such as RANKL. In this virtual scenario (roughly based in submitted papers), score zero refers to 0 to 5% of stained cells, score one refers to 5 to 25%, score two to 26 to 50%, and so on. Therefore, a 25% sample and a 26% sample would receive different scores, while 26% and 50% samples would receive the very same score. It seems that it is not necessary to apply complex statistical tools to realize that some qualitative ‘downgrade’ may not be the best option for the subsequent data analysis, especially when the quantitative data is available. This strategy leads to a weakening of the dependent variable and consequently a less robust and accurate data analysis. When the stratification is necessary, and phenotypic data is not available to guide the stratification, the use of tertiles, quartiles, deciles or cluster analysis can be more adequate than the random assignment of samples into scores or subgroups. Indeed, a cluster analysis demonstrated that the clustering of osteolytic periapical lesions was primarily based in RANKL/OPG,2929- Araujo-Pires AC, Francisconi CF, Biguetti CC, Cavalla F, Aranha AM, Letra A, et al. Simultaneous analysis of T helper subsets (Th1, Th2, Th9, Th17, Th22, Tfh, Tr1 and Tregs) markers expression in periapical lesions reveals multiple cytokine clusters accountable for lesions activity and inactivity status. J Appl Oral Sci. 2014;22(4):336-46. presenting a high match (>90%) with the inactive/inactive classification based in pressure/tension RANKL/OPG threshold. Additionally, the stratification of the lesions based in RANKL/OPG tertiles reveals a very high match between the high RANKL/OPG tertile and the theoretically active lesions.3030- Colavite PM, Cavalla F, Garlet TP, Azevedo MC, Melchiades JL, Campanelli AP, et al. TBX21-1993T/C polymorphism association with Th1 and Th17 response at periapex and with periapical lesions development risk. J Leukoc Biol. 2019;105(3):609-19. At this point, is also mandatory to consider that the use of multiple analysis models can also reinforce the strength of the data, despite the lack of direct influence in P values of independent analysis.

Another non-mathematical tool that can be supporting factor for increasing data strength refers to the combination of different approaches in the study design, such as combining clinical and experimental data. Return to the unclear nature of bone activity or inactivity in periodontitis, and the possibility of determine a theoretical threshold value that distinguish presence of absence of active bone resorption based in tension/pressure sides of orthodontic movement. Despite comprising and valid and interesting approach, one may argue that orthodontic tooth movement features may be not applied to periodontal disease. It is possible to argue that such assumptions are purely associative, and are not definitive demonstration of lesions activity/inactivity status. In this context, experimental models appear as a very important experimental tool do provide non-mathematical/non-statistical “biological significance” support to the clinical data. The possibility of performing longitudinal and controlled analysis, and to implement cause-and-effect experiments, with the induction or inhibition of a given factor (by genetic or pharmacological ways), can provide definitive demonstration of such factor role in a determined condition.

Still in the periodontitis activity/inactivity setting, experimental periodontitis in mice is characterized by sequential stages, where the initial response involves a major influx of inflammatory cells, and rapid and evident bone resorption and a high RANKL/OPG ratio.1111- Garlet GP. Destructive and protective roles of cytokines in periodontitis: a re-appraisal from host defense and tissue destruction viewpoints. J Dent Res. 2010;89(12):1349-63.,3434- Garlet GP, Cardoso CR, Silva TA, Ferreira BR, Avila-Campos MJ, Cunha FQ, et al. Cytokine pattern determines the progression of experimental periodontal disease induced by Actinobacillus actinomycetemcomitans through the modulation of MMPs, RANKL, and their physiological inhibitors. Oral Microbiol Immunol. 2006;21(1):12-20.3636- Repeke CE, Ferreira SB Jr, Claudino M, Silveira EM, Assis GF, Avila-Campos MJ, et al. Evidences of the cooperative role of the chemokines CCL3, CCL4 and CCL5 and its receptors CCR1+ and CCR5+ in RANKL+ cell migration throughout experimental periodontitis in mice. Bone. 2010;46(4):1122-30. The subsequent stage is characterized by a minor progression of bone loss, a change in the pattern of immunoregulatory elements and a low RANKL/OPG ratio. The modulation of RANKL/OPG ratio, specifically upregulation of OPG by the selective attraction of immunoregulatory cells, such as Tregs, is associated with the prevention of bone loss.3737- Araujo-Pires AC, Vieira AE, Francisconi CF, Biguetti CC, Glowacki A, Yoshizawa S, et al. IL-4/CCL22/CCR4 axis controls regulatory T-cell migration that suppresses inflammatory bone loss in murine experimental periodontitis. J Bone Miner Res. 2015;30(3):412-22.4040- Garlet GP, Cardoso CR, Mariano FS, Claudino M, Assis GF, Campanelli AP, et al. Regulatory T cells attenuate experimental periodontitis progression in mice. J Clin Periodontol. 2010;37(7):591-600. Conversely, the inhibition of immunoregulatory feedback and the maintenance of high RANKL/OPG over time result in increased disease severity.4141- Francisconi CF, Vieira AE, Azevedo MCS, Tabanez AP, Fonseca AC, Trombone APF, et al. RANKL triggers Treg-mediated immunoregulation in inflammatory osteolysis. J Dent Res. 2018;97(8):917-27. Among the immunoregulatory elements, IL-10, a potent anti-inflammatory molecule which is extensively produced by Tregs, was also demonstrated to inhibit inflammation and to arrest experimental periodontitis progression.1818- Claudino M, Trombone AP, Cardoso CR, Ferreira SB Jr, Martins W Jr, Assis GF, et al. The broad effects of the functional IL-10 promoter-592 polymorphism: modulation of IL-10, TIMP-3, and OPG expression and their association with periodontal disease outcome. J Leukoc Biol. 2008;84(6):1565-73.,4242- Garlet GP, Giozza SP, Silveira EM, Claudino M, Santos SB, Avila-Campos MJ, et al. Association of human T lymphotropic virus 1 amplification of periodontitis severity with altered cytokine expression in response to a standard periodontopathogen infection. Clin Infect Dis. 2010;50(3):e11-8.4444- Zhang X, Teng YT. Interleukin-10 inhibits gram-negative-microbe-specific human receptor activator of NF-kappaB ligand-positive CD4+-Th1-cell-associated alveolar bone loss in vivo. Infect Immun. 2006;74(8):4927-31. Full circle, one of the anti-inflammatory mechanisms of IL-10 comprise the upregulation of SOCS, which in turn limits the action of pro-inflammatory mediators, in parallel with the capacity of IL-10 to upregulate OPG. Taken together, this experimental data suggests that periodontal lesions inactivity may be under the control of IL-10, which in turn upregulate OPG, induces SOCS and consequently limits pro-osteoclastic and pro-inflammatory activity.3232- Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4.,4444- Zhang X, Teng YT. Interleukin-10 inhibits gram-negative-microbe-specific human receptor activator of NF-kappaB ligand-positive CD4+-Th1-cell-associated alveolar bone loss in vivo. Infect Immun. 2006;74(8):4927-31.,4545- Araujo AA, Varela H, Brito GA, Medeiros CA, Araujo LS, Nascimento JH, et al. Azilsartan increases levels of IL-10, down-regulates MMP-2, MMP-9, RANKL/RANK, Cathepsin K and up-regulates OPG in an experimental periodontitis model. PloS One. 2014;9(5):e96750.

While experimental data does not provide any kind of additional “statistical significance” to the associative data derived from human studies, including those previously mentioned along this essay, the “biological significance”, despite being numerically unmeasurable, is remarkable. Despite the unprivileged position in the scientific evidence pyramid, the “pre-clinical” research is essential in unraveling mechanistic evidences for biological and pathological processes, and to provide the basis for subsequent clinical interventions.4646- Souza JA, Rossa C Jr, Garlet GP, Nogueira AV, Cirelli JA. Modulation of host cell signaling pathways as a therapeutic approach in periodontal disease. J Appl Oral Sci. 2012;20(2):128-38. Mice with opposing maximal and minimal inflammatory responsiveness genotypes and phenotypes present distinct susceptibility/resistance patterns when exposed to periodontopathogens, reinforcing the genotypic impact over the phenotype unraveled under “exposure” conditions,4747- Trombone AP, Ferreira SB Jr, Raimundo FM, Moura KC, Avila-Campos MJ, Silva JS, et al. Experimental periodontitis in mice selected for maximal or minimal inflammatory reactions: increased inflammatory immune responsiveness drives increased alveolar bone loss without enhancing the control of periodontal infection. J Periodontal Res. 2009;44(4):443-51. PMID.4848- Trombone AP, Claudino M, Colavite P, Assis GF, Avila-Campos MJ, Silva JS, et al. Periodontitis and arthritis interaction in mice involves a shared hyper-inflammatory genotype and functional immunological interferences. Genes Immun. 2010;11(6):479-89. The marked susceptibility of IL-10 deficient mice to periodontitis development4343- Claudino M, Garlet TP, Cardoso CR, de Assis GF, Taga R, Cunha FQ, et al. Down-regulation of expression of osteoblast and osteocyte markers in periodontal tissues associated with the spontaneous alveolar bone loss of interleukin-10 knockout mice. Eur J Oral Sci. 2010;118(1):19-28. supported the investigation of genetic variants that modulate IL-10 levels in humans,1818- Claudino M, Trombone AP, Cardoso CR, Ferreira SB Jr, Martins W Jr, Assis GF, et al. The broad effects of the functional IL-10 promoter-592 polymorphism: modulation of IL-10, TIMP-3, and OPG expression and their association with periodontal disease outcome. J Leukoc Biol. 2008;84(6):1565-73. and its role as risk factors for periodontal disease development with potential application for diagnosis and treatment planning. The possibility of modulating the inflammatory environment nature in periodontium by IL-10, derived from Tregs recruitment,3737- Araujo-Pires AC, Vieira AE, Francisconi CF, Biguetti CC, Glowacki A, Yoshizawa S, et al. IL-4/CCL22/CCR4 axis controls regulatory T-cell migration that suppresses inflammatory bone loss in murine experimental periodontitis. J Bone Miner Res. 2015;30(3):412-22.4040- Garlet GP, Cardoso CR, Mariano FS, Claudino M, Assis GF, Campanelli AP, et al. Regulatory T cells attenuate experimental periodontitis progression in mice. J Clin Periodontol. 2010;37(7):591-600. was demonstrated to the useful in pre-clinical models, which support its potential application in future clinical trials. The discovery of RANKL/OPG axis in experimental models leaded to the development of OPG-mimicking/analogues therapeutics currently used in humans for diverse bone pathologies.

Finally, we reiterate that this piece (similarly to the Amrhein's Nature assay) are not advocating a ban on P values or statistical measures, which remains essential. However, it is crucial to consider that statisticians are not mystical magical creatures and therefore there are no statistical miracles to marvelously overcome poor experimental design.

Thus, despite any major change in the way of P values or statistical tests are applied and interpreted, we must consider that the attention to the proper study design may have an immediate and positive influence in the numerically unmeasurable data “biological significance”.

References

  • 1
    - Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305-7.
  • 2
    - Wasserstein RL, Lazar NA. The ASA's statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129-31.
  • 3
    - Ioannidis JP. Retiring statistical significance would give bias a free pass. Nature. 2019;567(7749):461.
  • 4
    - Trevilatto PC, Souza Pardo AP, Scarel-Caminaga RM, Brito RB Jr, Alvim-Pereira F, Alvim-Pereira CC, et al. Association of IL1 gene polymorphisms with chronic periodontitis in Brazilians. Arch Oral Biol. 2011;56(1):54-62.
  • 5
    - Kinane DF, Shiba H, Hart TC. The genetic basis of periodontitis. Periodontol 2000. 2005;39:91-117.
  • 6
    - Laine ML, Crielaard W, Loos BG. Genetic susceptibility to periodontitis. Periodontol 2000. 2012;58(1):37-68.
  • 7
    - Schäfer AS, Jepsen S, Loos BG. Periodontal genetics: a decade of genetic association studies mandates better study designs. J Clin Periodontol. 2011;38(2):103-7.
  • 8
    - Zondervan KT, Cardon LR, Kennedy SH. What makes a good case-control study? Design issues for complex traits such as endometriosis. Hum Reprod. 2002;17(6):1415-23.
  • 9
    - Sakthianandeswaren A, Foote SJ, Handman E. The role of host genetics in leishmaniasis. Trends Parasitol. 2009;25(8):383-91.
  • 10
    - Garlet GP, Trombone AP, Menezes R, Letra A, Repeke CE, Vieira AE, et al. The use of chronic gingivitis as reference status increases the power and odds of periodontitis genetic studies: a proposal based in the exposure concept and clearer resistance and susceptibility phenotypes definition. J Clin Periodontol. 2012;39(4):323-32.
  • 11
    - Garlet GP. Destructive and protective roles of cytokines in periodontitis: a re-appraisal from host defense and tissue destruction viewpoints. J Dent Res. 2010;89(12):1349-63.
  • 12
    - Garlet GP, Martins W Jr, Fonseca BA, Ferreira BR, Silva JS. Matrix metalloproteinases, their physiological inhibitors and osteoclast factors are differentially regulated by the cytokine profile in human periodontal disease. J Clin Periodontol. 2004;31(8):671-9.
  • 13
    - Gilthorpe MS, Zamzuri AT, Griffiths GS, Maddick IH, Eaton KA, Johnson NW. Unification of the “burst” and “linear” theories of periodontal disease progression: a multilevel manifestation of the same phenomenon. J Dent Res. 2003;82(3):200-5.
  • 14
    - Nomura Y, Morozumi T, Nakagawa T, Sugaya T, Kawanami M, Suzuki F, et al. Site-level progression of periodontal disease during a follow-up period. PloS One. 2017;12(12):e0188670.
  • 15
    - Trombone AP, Cardoso CR, Repeke CE, Ferreira SB Jr, Martins W Jr, Campanelli AP, et al. Tumor necrosis factor-alpha −308G/A single nucleotide polymorphism and red-complex periodontopathogens are independently associated with increased levels of tumor necrosis factor-alpha in diseased periodontal tissues. J Periodontal Res. 2009;44(5):598-608.
  • 16
    - Repeke CE, Trombone AP, Ferreira SB Jr, Cardoso CR, Silveira EM, Martins W Jr, et al. Strong and persistent microbial and inflammatory stimuli overcome the genetic predisposition to higher matrix metalloproteinase-1 (MMP-1) expression: a mechanistic explanation for the lack of association of MMP1-1607 single-nucleotide polymorphism genotypes with MMP-1 expression in chronic periodontitis lesions. J Clin Periodontol. 2009;36(9):726-38.
  • 17
    - Ferreira SB Jr, Trombone AP, Repeke CE, Cardoso CR, Martins W Jr, Santos CF, et al. An interleukin-1beta (IL-1beta) single-nucleotide polymorphism at position 3954 and red complex periodontopathogens independently and additively modulate the levels of IL-1beta in diseased periodontal tissues. Infect Immun. 2008;76(8):3725-34.
  • 18
    - Claudino M, Trombone AP, Cardoso CR, Ferreira SB Jr, Martins W Jr, Assis GF, et al. The broad effects of the functional IL-10 promoter-592 polymorphism: modulation of IL-10, TIMP-3, and OPG expression and their association with periodontal disease outcome. J Leukoc Biol. 2008;84(6):1565-73.
  • 19
    - Silva TA, Garlet GP, Lara VS, Martins W Jr, Silva JS, Cunha FQ. Differential expression of chemokines and chemokine receptors in inflammatory periapical diseases. Oral Microbiol Immunol. 2005;20(5):310-6.
  • 20
    - Menezes R, Garlet TP, Letra A, Bramante CM, Campanelli AP, Figueira RC, et al. Differential patterns of receptor activator of nuclear factor kappa B ligand/osteoprotegerin expression in human periapical granulomas: possible association with progressive or stable nature of the lesions. J Endod. 2008;34(8):932-8.
  • 21
    - Menezes R, Garlet TP, Trombone AP, Repeke CE, Letra A, Granjeiro JM, et al. The potential role of suppressors of cytokine signaling in the attenuation of inflammatory reaction and alveolar bone loss associated with apical periodontitis. J Endod. 2008;34(12):1480-4.
  • 22
    - Garlet GP, Horwat R, Ray HL Jr, Garlet TP, Silveira EM, Campanelli AP, et al. Expression analysis of wound healing genes in human periapical granulomas of progressive and stable nature. J Endod. 2012;38(2):185-90.
  • 23
    - Taddei SR, Moura AP, Andrade I Jr, Garlet GP, Garlet TP, Teixeira MM, et al. Experimental model of tooth movement in mice: a standardized protocol for studying bone remodeling under compression and tensile strains. J Biomech. 2012;45(16):2729-35.
  • 24
    - Taddei SR, Andrade I Jr, Queiroz-Junior CM, Garlet TP, Garlet GP, Cunha FQ, et al. Role of CCR2 in orthodontic tooth movement. Am J Orthod Dentofacial Orthop. 2012;141(2):153-60.
  • 25
    - Taddei SR, Queiroz-Junior CM, Moura AP, Andrade I Jr, Garlet GP, Proudfoot AE, et al. The effect of CCL3 and CCR1 in bone remodeling induced by mechanical loading during orthodontic tooth movement in mice. Bone. 2013;52(1):259-67.
  • 26
    - Andrade I Jr, Taddei SR, Garlet GP, Garlet TP, Teixeira AL, Silva TA, et al. CCR5 down-regulates osteoclast function in orthodontic tooth movement. J Dent Res. 2009;88(11):1037-41.
  • 27
    - Garlet TP, Coelho U, Repeke CE, Silva JS, Cunha FQ, Garlet GP. Differential expression of osteoblast and osteoclast chemmoatractants in compression and tension sides during orthodontic movement. Cytokine. 2008;42(3):330-5.
  • 28
    - Garlet TP, Coelho U, Silva JS, Garlet GP. Cytokine expression pattern in compression and tension sides of the periodontal ligament during orthodontic tooth movement in humans. Eur J Oral Sci. 2007;115(5):355-62.
  • 29
    - Araujo-Pires AC, Francisconi CF, Biguetti CC, Cavalla F, Aranha AM, Letra A, et al. Simultaneous analysis of T helper subsets (Th1, Th2, Th9, Th17, Th22, Tfh, Tr1 and Tregs) markers expression in periapical lesions reveals multiple cytokine clusters accountable for lesions activity and inactivity status. J Appl Oral Sci. 2014;22(4):336-46.
  • 30
    - Colavite PM, Cavalla F, Garlet TP, Azevedo MC, Melchiades JL, Campanelli AP, et al. TBX21-1993T/C polymorphism association with Th1 and Th17 response at periapex and with periapical lesions development risk. J Leukoc Biol. 2019;105(3):609-19.
  • 31
    - Souza JA, Nogueira AV, Souza PP, Cirelli JA, Garlet GP, Rossa C Jr. Expression of suppressor of cytokine signaling 1 and 3 in ligature-induced periodontitis in rats. Arch Oral Biol. 2011;56(10):1120-8.
  • 32
    - Garlet GP, Cardoso CR, Campanelli AP, Martins W Jr, Silva JS. Expression of suppressors of cytokine signaling in diseased periodontal tissues: a stop signal for disease progression? J Periodontal Res. 2006;41(6):580-4.
  • 33
    - Malaspina TS, Gasparoto TH, Costa MR, Melo EF Jr, Ikoma MR, Damante JH, et al. Enhanced programmed death 1 (PD-1) and PD-1 ligand (PD-L1) expression in patients with actinic cheilitis and oral squamous cell carcinoma. Cancer Immunol Immunother. 2011;60(7):965-74.
  • 34
    - Garlet GP, Cardoso CR, Silva TA, Ferreira BR, Avila-Campos MJ, Cunha FQ, et al. Cytokine pattern determines the progression of experimental periodontal disease induced by Actinobacillus actinomycetemcomitans through the modulation of MMPs, RANKL, and their physiological inhibitors. Oral Microbiol Immunol. 2006;21(1):12-20.
  • 35
    - Garlet GP, Cardoso CR, Campanelli AP, Ferreira BR, Avila-Campos MJ, Cunha FQ, et al. The dual role of p55 tumour necrosis factor-alpha receptor in Actinobacillus actinomycetemcomitans-induced experimental periodontitis: host protection and tissue destruction. Clin Exp Immunol. 2007;147(1):128-38.
  • 36
    - Repeke CE, Ferreira SB Jr, Claudino M, Silveira EM, Assis GF, Avila-Campos MJ, et al. Evidences of the cooperative role of the chemokines CCL3, CCL4 and CCL5 and its receptors CCR1+ and CCR5+ in RANKL+ cell migration throughout experimental periodontitis in mice. Bone. 2010;46(4):1122-30.
  • 37
    - Araujo-Pires AC, Vieira AE, Francisconi CF, Biguetti CC, Glowacki A, Yoshizawa S, et al. IL-4/CCL22/CCR4 axis controls regulatory T-cell migration that suppresses inflammatory bone loss in murine experimental periodontitis. J Bone Miner Res. 2015;30(3):412-22.
  • 38
    - Glowacki AJ, Gottardi R, Yoshizawa S, Cavalla F, Garlet GP, Sfeir C, et al. Strategies to direct the enrichment, expansion, and recruitment of regulatory cells for the treatment of disease. Ann Biomed Eng. 2015;43(3):593-602.
  • 39
    - Glowacki AJ, Yoshizawa S, Jhunjhunwala S, Vieira AE, Garlet GP, Sfeir C, et al. Prevention of inflammation-mediated bone loss in murine and canine periodontal disease via recruitment of regulatory lymphocytes. Proc Natl Acad Sci U S A. 2013;110(46):18525-30.
  • 40
    - Garlet GP, Cardoso CR, Mariano FS, Claudino M, Assis GF, Campanelli AP, et al. Regulatory T cells attenuate experimental periodontitis progression in mice. J Clin Periodontol. 2010;37(7):591-600.
  • 41
    - Francisconi CF, Vieira AE, Azevedo MCS, Tabanez AP, Fonseca AC, Trombone APF, et al. RANKL triggers Treg-mediated immunoregulation in inflammatory osteolysis. J Dent Res. 2018;97(8):917-27.
  • 42
    - Garlet GP, Giozza SP, Silveira EM, Claudino M, Santos SB, Avila-Campos MJ, et al. Association of human T lymphotropic virus 1 amplification of periodontitis severity with altered cytokine expression in response to a standard periodontopathogen infection. Clin Infect Dis. 2010;50(3):e11-8.
  • 43
    - Claudino M, Garlet TP, Cardoso CR, de Assis GF, Taga R, Cunha FQ, et al. Down-regulation of expression of osteoblast and osteocyte markers in periodontal tissues associated with the spontaneous alveolar bone loss of interleukin-10 knockout mice. Eur J Oral Sci. 2010;118(1):19-28.
  • 44
    - Zhang X, Teng YT. Interleukin-10 inhibits gram-negative-microbe-specific human receptor activator of NF-kappaB ligand-positive CD4+-Th1-cell-associated alveolar bone loss in vivo Infect Immun. 2006;74(8):4927-31.
  • 45
    - Araujo AA, Varela H, Brito GA, Medeiros CA, Araujo LS, Nascimento JH, et al. Azilsartan increases levels of IL-10, down-regulates MMP-2, MMP-9, RANKL/RANK, Cathepsin K and up-regulates OPG in an experimental periodontitis model. PloS One. 2014;9(5):e96750.
  • 46
    - Souza JA, Rossa C Jr, Garlet GP, Nogueira AV, Cirelli JA. Modulation of host cell signaling pathways as a therapeutic approach in periodontal disease. J Appl Oral Sci. 2012;20(2):128-38.
  • 47
    - Trombone AP, Ferreira SB Jr, Raimundo FM, Moura KC, Avila-Campos MJ, Silva JS, et al. Experimental periodontitis in mice selected for maximal or minimal inflammatory reactions: increased inflammatory immune responsiveness drives increased alveolar bone loss without enhancing the control of periodontal infection. J Periodontal Res. 2009;44(4):443-51.
  • 48
    - Trombone AP, Claudino M, Colavite P, Assis GF, Avila-Campos MJ, Silva JS, et al. Periodontitis and arthritis interaction in mice involves a shared hyper-inflammatory genotype and functional immunological interferences. Genes Immun. 2010;11(6):479-89.

Publication Dates

  • Publication in this collection
    07 Oct 2019
  • Date of issue
    2019
Faculdade De Odontologia De Bauru - USP Serviço de Biblioteca e Documentação FOB-USP, Al. Dr. Octávio Pinheiro Brisolla 9-75, 17012-901 Bauru SP Brasil, Tel.: +55 14 32358373 - Bauru - SP - Brazil
E-mail: jaos@usp.br