Introducing the CONSORT extension to pilot trials: enhancing the design, conduct and reporting of pilot or feasibility trials

Abstract

This editorial provides a brief overview of the importance of pilot or feasibility trials or studies, the challenges with current practices in their conduct and reporting, an introduction to the Consolidated Standards of Reporting Trials (CONSORT) extension to pilot trials aimed at improving their reporting, along with some key resources on aspects related to pilot and feasibility studies.

What are pilot studies and why they are important?

A randomized controlled trial (RCT) is considered the best type of design for studying causal links between interventional exposure and outcomes in all clinical areas including treatment of tropical diseases and procedures related to toxinology. Designing and conducting RCTs is a complex undertaking, and it is frequently time-consuming and expensive. Therefore, conducting a feasibility or pilot study/trial prior to the main RCT is often highly recommended so as to enhance the likelihood of success of the main study, and thereby saving time and costs [¹]. Pilot or feasibility study can be defined as a small-scale investigation designed to test the feasibility of methods and procedures for later use in the design of a large scale study [²]. Thus, by definition, the term ‘pilot’ or ‘feasibility’ implies an intention for further definitive work in the future [³].

It is important to emphasize that the primary objectives and outcomes of pilot or feasibility study and the main study are different. The pilot or feasibility trial aims to address questions about prevailing uncertainty regarding the processes or aspects of the main study, such as recruitment, randomization, retention, assessment procedures, validity of tools, new methods and implementation of an intervention. On the other hand, a definitive or main trial has a primary objective of determining the efficacy or effectiveness of a new intervention [⁴, ⁵].

Another important distinction between a pilot trial and a main RCT is that in the main study we typically use a statistically significant result based on the minimal clinically important difference (MCID) on the primary outcome of interest as the criterion for declaring success for the trial. In contrast, in a pilot trial, researchers need to establish analogous criteria for determining success of feasibility to guide decisions about moving on to the largescale trial. Such criteria are typically based on the primary feasibility objectives and outcomes [¹].

What are the current challenges with pilot studies?

Although there is an increasing number of pilot/feasibility studies in the literature, there is substantive evidence that the reporting and conduct is very suboptimal [¹, ³, ⁶, ⁷]. There is, for example: too much emphasis on hypothesis testing; no clear feasibility objectives/outcomes; inadequate descriptions of analytic plans; and no explicit mention that they were actually done to inform the design of future studies [⁷, ⁸]. Some reports of pilot trials show that the study is called “pilot” because of the small sample size, or because it was conducted in single center and without the clear intention of advancing to a main study [¹]. Most journals have no editorial policies on pilot studies – making it harder for reviewers and editors to make optimal and consistent editorial decisions on manuscripts reporting such studies [³, ⁹].

What is the CONSORT extension to pilot trials?

The CONSORT was first published in 1996 and it was last updated in 2010. It is a guideline designed to improve the transparency and quality of the reporting of RCTs, and it was based on the “standard” two-group parallel design [¹⁰]. However, there are several variations to the standard trial methodology, and nowadays there are some CONSORT extensions, for example to cluster trials, non-inferiority and equivalence trials and to pilot and feasibility studies. These extensions of main CONSORT were done to help improve the reporting of these trials.

The CONSORT 2010 statement extension to randomized pilot and feasibility trials that are conducted in advance of a future definitive RCT was published in 2016 [¹¹]. The publication of this extension was motivated by the growing number of studies described as feasibility or pilot and by research that identified deficiencies in their reports and conduct.

As mentioned before, pilot/feasibility trials differ from other forms of RCTs in their objectives – their focus is on assessing feasibility rather than effectiveness or efficacy. Therefore, there are some key differences between the main CONSORT statement [¹⁰] and CONSORT statement extension to pilot trials [¹¹]:

• First, an adapted CONSORT extension to abstracts of pilot trials checklist is provided with illustration of how to use the checklist.

• Second, the items of standard CONSORT and those of the CONSORT extension to pilot trials are put side-by-side to make it easier for the researcher to identify items that:

  • remain unchanged (items 4a; 4b; 5; 7b; 8a; 9; 10; 11a; 11b; 13b; 14a; 15; 19; 25);

  • have been modified (items 1a; 1b; 2a; 2b; 3a; 3b; 6a; 6b; 7a; 8b; 12a; 13a; 14b; 16; 17a; 18; 20; 21; 22; 23; 24);

  • were added (items 4c; 6c; 19a; 22a; 26); and

  • were deleted (12b; 17b) – because they were deemed not relevant or applicable for pilot or feasibility trials.

• Third, the guideline also provides some explanation and elaboration for each item – to explain why each included item is important with examples of best practices, and why each excluded item is not relevant to pilot trials. Finally, it recommends an adapted flow diagram of progress of the study with the number of participants that were screened, enrollment, intervention allocation, follow up, and assessment and reporting of each feasibility outcome of the study.

The latest versions on the CONSORT statement of RCT and its extensions are available at www.consort-statement.org and the Equator Network website www.equator-network.org.

Additional resources for improving the conduct and reporting of pilot trials

Table 1 provides a list of some useful resources that researchers can explore for more information about definitions, challenges with reporting, sample size and reporting pilot studies.

Table 1
Some key references about different topics related to pilot or feasibility trials

Main topic	Key references
Definitions	[4, 8, 9]
Challenges with reporting	[1, 3, 6, 7, 12–14]
Sample size estimation	[5, 15]
Reporting	[11, 16, 17]

Conclusions

Pilot or feasibility studies provide a good opportunity to assess feasibility of large full-scale evaluation studies and they can also enhance the success probability of the main study. For enhancing the transparency and quality of pilot/feasibility trials, they should be scrutinized the same way as the main trials: they should be registered, and they should be reviewed for ethics approval like all research studies. All researchers have the ethical, scientific and economic responsibility and obligation to ensure that their findings about feasibility are widely disseminated in a timely manner. Adoption, adherence and enforcement of the CONSORT extension by journals would certainly make it easier for all stakeholders including authors, journal editors, manuscript reviewers, and researchers to enhance their reporting. Many journals have endorsed the CONSORT statement and its extensions, and recommend adherence to these guidelines in their instructions to authors.

• Funding
  Not applicable.
• Publisher's Note
  Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
• Published online: 02 February 2018

Abbreviations
• MCID:  minimal clinically important difference;
• RCT:  randomized controlled trial

Acknowledgments

Not applicable.

References

• ¹
  Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1.
• ²
  Everitt BS. Medical statistics from a to Z: a guide for clinicians and medical students. 2nd ed. Cambridge: Cambridge University Press; 2007.
• ³
  Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010;10:67.
• ⁴
  Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9.
• ⁵
  Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013;66(2):197–201.
• ⁶
  Shanyinde M, Pickering RM, Weatherall M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Med Res Methodol. 2011;11:117.
• ⁷
  Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.
• ⁸
  Whitehead AL, Sully BG, Campbell MJ. Pilot and feasibility studies: is there a difference from each other and from a randomised controlled trial? Contemp Clin Trials. 2014;38(1):130–3.
• ⁹
  Loscalzo J. Pilot trials in clinical research: of what value are they? Circulation. 2009;119(13):1694–6.
• ¹⁰
  Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. J Pharmacol Pharmacother. 2010;1(2):100–7.
• ¹¹
  Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud. 2016;2:64. eCollection 2016
• ¹²
  Kaur N, Figueiredo S, Bouchard V, Moriello C, Mayo N. Where have all the pilot studies gone? A follow-up on 30 years of pilot studies in clinical rehabilitation. Clin Rehabil. 2017;31(9):1238–48.
• ¹³
  Kistin C, Silverstein M. Pilot studies: a critical but potentially misused component of interventional research. JAMA. 2015;314(15):1561–2.
• ¹⁴
  Kraemer HC, Mintz J, Noda A, Tinklenberg J, Yesavage JA. Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry. 2006;63(5):484–9.
• ¹⁵
  Eldridge SM, Costelloe CE, Kahan BC, Lancaster GA, Kerry SM. How big should the pilot study for my cluster randomised trial be? Stat Methods Med Res. 2016;25(3):1039–56.
• ¹⁶
  Craig P. A new CONSORT extension should improve the reporting of randomized pilot and feasibility trials. J Clin Epidemiol. 2017;84:30–2.
• ¹⁷
  Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One. 2016;11(3):e0150205.

Publication Dates

• Publication in this collection
  2018

History

• Received
  03 Jan 2018
• Accepted
  18 Jan 2018