Non-inferiority clinical trials: importance and applications in health sciences

Abstract

Non-inferiority randomized clinical trials are indicated when it is intended to prove that an experimental group is not inferior to a control group by more than a margin of non-inferiority. However, this type of study differs from traditional randomized clinical trials (superiority studies) because they have particularities that impact on the formulation of hypothesis to be tested, experimental design (non-inferiority margin determination, adapted sample size calculation, sensitivity of the study and data final analysis) and also on the presentation of data when writing the manuscript. Therefore, this article aims to present and discuss the particularities of non-inferiority clinical studies, since these requirements are fundamental to guarantee the validity of the conclusions of this type of study.

Randomized Controlled Trial; Equivalence Trial; Study Characteristics

Introduction

Randomized clinical trials (RTC) are on the second to last step of the evidence-based pyramid. When compared to basic science/case series and case–control/cohort studies, RTC offers greater reliability in promoting changes of the current clinical protocols by the establishment of new ones.¹

Traditionally, researchers seek to verify, by means of a rigid clinical protocol, the superiority of an experimental group in relation to a positive control (gold standard). However, for a standard protocol (positive control) to be replaced by a new one, the experimental group does not necessarily have to be superior to control and in this case non-inferiority clinical trials gain importance. These are especially indicated when it is not possible to use a placebo group because active controls are necessary or because the experimental group has several advantages over the positive control that allow its use, even if it is not superior to control.²

However, the researcher often does not know that his research fits into a non-inferiority design and does not consider all its particularities in planning, execution and communication of this type of study design, compromising the credibility of the obtained results. Thus, this article aims to define non-inferiority clinical trials presenting their particularities and differentiating them from traditional randomized clinical trials.

Definition

Randomized clinical trials are close to the top of the evidence-based pyramid.¹ Within this placement there are subdivisions, since randomized clinical trials are not all the same, and can be classified in clinical studies of:

1. Superiority: evaluates if there is superiority of a new intervention in relation to a control group;

2. Equivalence: searches to establish if a new intervention (Experimental) is equivalent to the reference intervention;

3. Non-inferiority: searches to establish if a new intervention (Experimental) is not less effective than the reference intervention.

The main focus of this article will be given to the last item of this classification. The question that rises is: When to conduct a Non-inferiority Randomized Clinical Trial? The answer is simple, when the use of placebo is ethically unfeasible, as for example in drug therapies for severe chronic diseases, and when an experimental group should be compared to a gold standard (positive control).³ Non-inferiority studies can also be indicated when a new intervention (Experimental) has advantages over the standard intervention (Control) such as if the new intervention is better tolerated and has fewer side effects, implies a lower number of doses, has a lower cost or the route of administration is less invasive.²,³,⁴ Thus, considering all these possible advantages, the experimental group does not need to be necessarily superior to the control group, it suffices that it is not inferior so that it could be indicated as therapy to be adopted. Thus, this type of study is designed to assess whether a new treatment (Experimental) is not less effective than a standard treatment (Control), by more than a margin of tolerance, known as non-inferiority margin.⁵ Considering this main peculiarity of non-inferiority studies, it is important to highlight that the differences among superiority, equivalence and non-inferiority trials begin with the formulation of the hypothesis to be tested.

In a superiority study the hypotheses are:

1. H0 (null hypothesis): Value found in the Experimental group - Value found in the Control group ≤ 0, that is, the experimental group is not superior to the control group;

2. H1 (alternative hypothesis): Value found in the Experimental group - Value found in the Control group> 0, that is, the experimental group is superior to the control group.

A non-inferiority study intends to demonstrate that the experimental group is not inferior to the control group by more than the non-inferiority margin, the hypotheses to be tested are:

1. H0 (null hypothesis): Value found in the Experimental group - Value found in the Control group ≤ non-inferiority margin, that is, the experimental group is inferior to the control group;

2. H1 (alternative hypothesis): Value found in the Experimental group - Value found in the Control group> non-inferiority margin, that is, the experimental group is not inferior to the control group.

First, it is important to learn how to define the margin of non-inferiority, since this is the main factor to be established in the design of a non- inferiority clinical trial. However, not only the margin of non-inferiority is important in this type of clinical study, but also other factors that will be better explored in this article, such as sample size, study sensitivity, population of analysis and reporting of the study.

Non-inferiority margin

The non-inferiority margin quantifies the maximum loss of clinically acceptable efficacy so that the treatment under study (experimental group) can be declared non-inferior to the control.⁵ For a study of this nature, it is not overstatement to say that this is the most important factor to be defined. In the past, the determination of this margin was empirical and depended heavily on the experience of the researcher. Nowadays, there are mathematical concepts that support the determination of this non-inferiority margin, thus demanding less subjectivity of the researchers. Thus, it is fundamental to adequately dimension this limit, and it cannot be too high or too low, since the two extremes can result in problems in the design or conclusions of the study.

1. High Non-inferiority margin: increases the probability that less effective treatments are declared non-inferior;

2. Low Non-inferiority margin: is more conservative, however, requires larger samples, making studies more expensive due to a larger number of patients, in addition to obvious ethical implications.

In a non-inferiority clinical trial the measure of efficacy to be analyzed should be at 95% confidence interval of the difference between values found in the experimental group and in the control group. If the confidence interval of this difference is totally above zero, then the experimental group can be considered superior to the control group (Figure A). If the confidence interval is below zero but totally above the non-inferiority limit, then the experimental group can be considered not inferior to the control group (Figure B). On the other hand, if the confidence interval is completely below the non-inferiority limit, the experimental group should be considered inferior to the control group (Figure C). Finally, non-inferiority studies whose 95% confidence interval cross the non-inferiority limit, presenting superior and inferior values (Figure D, and E) are considered inconclusive.

Figure
Non-inferiority margin and the relationship with 95% confidence interval of the difference between experimental and control groups.

Therefore, the establishment of a non-inferiority margin with an appropriate dimension is of main importance to the study. To determine the non-inferiority margin, researchers might always search for a comparison between a control and a placebo group, by means of pilot tests or by previously published data. Two methods are traditionally used to determine the non-inferiority margin. The first, known as the fixed margin method, uses a value ranging from 50 to 75% of the difference between the values found in control and placebo.²,³,⁴,⁵ The second method, advocated by the FDA, uses 50% of the lower limit of the 95% confidence interval of the difference between the values found in the control and in the placebo groups.²,³,⁴,⁵,⁶ Regardless of the chosen method, it is essential that the authors report the strategy used to obtain the non-inferiority margin and the value resulted from the calculations, since it will guide all the conclusions of the study.

Sample size

The sample calculation must be performed in all randomized clinical trials and the non-inferiority studies follow this rule. However, the calculation might be adapted to a study of this nature, since the non-inferiority margin has to be considered. The non-inferiority margin has a direct impact on the sample size, since if the non-inferiority margin is increased, the sample size is reduced, and if the non-inferiority margin is reduced the sample size is increased.⁵

Several statistical software and online calculators are able to make the sample size calculation according to this type of study. A simple and complete tool can be found at http://powerandsamplesize.com. This online calculator contemplates the most commonly dependent variables used in non-inferiority clinical studies, such as dichotomous and quantitative variables. For example, to calculate the sample size of a study comparing two means, the investigator will need to define the following parameters: estimated mean of the control group, estimated mean of the placebo group, standard deviation, non-inferiority margin, relationship between sample size of the 2 groups, α error, and β error. At the calculated value, the researcher should also add a percentage referring to the possible losses of the clinical study (20% for example). All these parameters should also be described in the manuscript, to evidence how the sample size of the non-inferiority clinical trial was calculated, as performed in Llanos et al.⁷

Sensitivity of the study

Sensitivity of a study is the ability of a clinical trial to discriminate an effective placebo treatment or, more generally, it is the property of detecting differences between treatments when they actually exist. The lack of sensitivity of the study may introduce a non-difference bias (e.g., increase in the false negative rate) in the comparisons, caused by the increase in type 2 or β error. This bias, in non-inferiority studies, may lead to ineffective treatments to be considered non-inferior²,⁴. Sensitivity is not only the power of the study (the ability to detect differences when they actually exist). It goes much further because it can be influenced by factors such as overall quality of the randomized clinical trial execution, patient adherence, patient selection criteria, magnitude of treatment effect, and excess variability of study data.

However, the most important factor to guarantee the sensitivity of a non-inferiority study is a historical data from the literature showing that in fact the control group to be tested is superior to the negative control. Ideally, any randomized clinical trial should be performed with 3 groups: experimental, positive control and placebo. However, the inclusion of placebo is not always feasible from a financial point of view and is often contraindicated according to the ethical principals involved.

Before conducting a clinical study comparing an experimental group with a control group, it is important to ensure that the control group is in fact superior to a placebo. Therefore, 2 factors are indispensable:

1. Historical evidence of treatment effect sensitivity and consistency of results: corresponds to properly conducted clinical studies in the past showing that the control group is regularly and consistently superior to placebo. The results cannot be volatile, so the efficacy of the control over placebo is maintained for similar studies.²,⁴

2. Quality of the study: it is evaluated through several factors that can reduce differences between treatments and / or increase variability, causing bias of no difference.²,⁴ Therefore, the most important thing to ensure this factor is the use checklists that check for quality attributes such as the CONSORT Checklist, for example.⁸

Population analyzed

There are two possible ways to assess data of randomized clinical trials, which is defined by the population considered in the final analyses. The intention to treat (ITT) considers the entire randomized population, even if some of them withdraw, were lost or did not adhere to the treatment. The per protocol (PP) includes only the patients that completed the treatment and adequately followed the protocol. In superiority studies, there is a consensus that intention to treat analysis should be preferred.²,⁴,⁶ It provides a more conservative result, decreasing differences between treatments and, therefore, introduces an opposite bias to non-inferiority studies. By keeping all volunteers randomized at the baseline of both groups (control and experimental) in the final analysis, the possible differences between groups diminish. Therefore, is a more conservative method to observe superiority, since if possible differences between groups appeared even with the maintenance of non-adherent patients in the final analysis, then it happened because in fact they exist.

On the other hand, the protocol analysis, which is done in a population of adherent patients, has a greater efficiency in discriminating treatments. Therefore, is the strategy to be followed first in the non-inferiority analysis, according to many authors.²,⁴,⁶ On the other side, in non-inferiority studies, the intention-to-treat analysis should also be performed. It is important to bear in mind that both analysis methods should be robust and consistent and ideally, they might not diverge for a more reliable result.²,⁴,⁶ See, for instance, the study of Arrow and Klobas⁹ in which the two methods were evaluated.

Report of a non-inferiority study

Finally, it is fundamental that there is an adequate report of the non-inferiority randomized clinical trial, considering all the particularities of this type of experiment. Therefore, it is important to follow the CONSORT guidelines,⁸ which aims to reduce the problems arising from inadequate reporting of randomized clinical trials, increasing the transparency and accuracy of research reports. In 2012, an extension of the Consort Statement was published to specifically report non-inferiority studies ⁶. Thus, from the design of the study to its publication, CONSORT should always guide the researcher in randomized clinical trials.

Conclusion

Non-inferiority randomized clinical trials should be carried out with increasing frequency, especially since in the health area, research ethics committees will no longer permit the evaluation of experimental groups compared to placebo groups, requiring the use of positive controls. However, there is still a lack of information about this type of experimental design and many protocols and therapies are incorrectly considered to be not inferior. Thus, discussing this topic in scientific events and scientific journals is of fundamental importance for the properly use of superiority, equivalence and non-inferiority randomized clinical trials by the scientific community.

Acknowledgments

The authors thank the Pro-Rector of Research of USP, Research Commission of FO, FOB and FORP and the Organizing Committee for participating in the “Grandes Desafios em Pesquisa Clínica na Odontologia” event. Also, the author thanks CAPES Foundation (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, financial code 001) and CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico, financial codes 304128/2016-3 and 304405/2018-3).

References

• ¹
  Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016 Aug;21(4):125-7. https://doi.org/10.1136/ebmed-2016-110401
  » https://doi.org/10.1136/ebmed-2016-110401
• ²
  Pinto VF. Estudos clínicos de não-inferioridade: fundamentos e controvérsias. J Vasc Bras. 2010;9(3):145-51. https://doi.org/10.1590/S1677-54492010000300009
  » https://doi.org/10.1590/S1677-54492010000300009
• ³
  Hahn S. Understanding noninferiority trials. Korean J Pediatr. 2012 Nov;55(11):403-7. https://doi.org/10.3345/kjp.2012.55.11.403
  » https://doi.org/10.3345/kjp.2012.55.11.403
• ⁴
  Sasse AD, Wada CY, Andrade JM, Bermudez JV, Lotufo PA, Pinto VF. Requerimentos mínimos para o planejamento e análise de estudos clínicos de não inferioridade. Rev Bras Clin Med. 2010 Nov-Dec;8(6):538-41.
• ⁵
  Althunian TA, Boer A, Groenwold RH, Klungel OH. Defining the noninferiority margin and analysing noninferiority: an overview. Br J Clin Pharmacol. 2017 Aug;83(8):1636-42. https://doi.org/10.1111/bcp.13280
  » https://doi.org/10.1111/bcp.13280
• ⁶
  Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG, CONSORT Group; CONSORT Group. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012 Dec;308(24):2594-604. https://doi.org/10.1001/jama.2012.87802
  » https://doi.org/10.1001/jama.2012.87802
• ⁷
  Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010 Aug;63(8):e1-37. https://doi.org/10.1016/j.jclinepi.2010.03.004
  » https://doi.org/10.1016/j.jclinepi.2010.03.004
• ⁸
  Llanos AH, Sapata VM, Jung RE, Hämmerle CH, Thoma DS, César Neto JB, et al. Comparison between two bone substitutes for alveolar ridge preservation after tooth extraction: cone-beam computed tomography results of a non-inferiority randomized controlled trial. J Clin Periodontol. 2019 Mar;46(3):373-81. https://doi.org/10.1111/jcpe.13079
  » https://doi.org/10.1111/jcpe.13079
• ⁹
  Arrow P, Klobas E. Child oral health-related quality of life and early childhood caries: a non-inferiority randomized control trial. Aust Dent J. 2016 Jun;61(2):227-35. https://doi.org/10.1111/adj.12352
  » https://doi.org/10.1111/adj.12352

Publication Dates

• Publication in this collection
  07 Aug 2020
• Date of issue
  2020

History

• Received
  2 Sept 2019
• Accepted
  22 Sept 2019
• Reviewed
  8 June 2020