Compliance with clinical trial registration and reporting guidelines by Latin American and Caribbean journals

Adherencia a las iniciativas de registro de ensayos clínicos y guías de reporte por revistas de América Latina y Caribe

Adesão às iniciativas de registro de ensaios clínicos e normas de notificação por periódicos da América Latina e do Caribe

ARTIGO ARTICLE

Compliance with clinical trial registration and reporting guidelines by Latin American and Caribbean journals

Adherencia a las iniciativas de registro de ensayos clínicos y guías de reporte por revistas de América Latina y Caribe

Adesão às iniciativas de registro de ensaios clínicos e normas de notificação por periódicos da América Latina e do Caribe

Ludovic Reveiz^I; Eleana Villanueva^I; Chimaraoke Iko^I; Iveta Simera^II

^IHealth Systems Based on Primary Health Care, Pan American Health Organization, Washington DC, U.S.A.

^IICentre for Statistics in Medicine, University of Oxford, Oxford, U.K.

^{Correspondence}

ABSTRACT

The objective of this study was to determine to what extent Latin American and Caribbean biomedical journals have endorsed and complied with clinical trial registration and reporting guidelines. A search of randomized clinical trials was carried out using the LILACS database. The randomized clinical trials identified through the search were assessed to determine whether trial registration and CONSORT guidance was mentioned. Information regarding endorsement of the ICMJE, trial registration and other reporting guidelines was extracted from the online instructions for authors of the journals included in the study. The search identified 477 references. We assessed a random sample of 240 titles of which 101 were randomized clinical trials published in 56 journals. Trial registration was reported in 19.8% of the randomized clinical trials, 6.9% were prospectively registered and 3% mentioned CONSORT. The ICMJE was mentioned by 68% of the journals and 36% of journals required trial registration. Fewer journals provided advice on reporting guidelines: CONSORT (13%), PRISMA (1.8%), STROBE (1.8%), and the EQUATOR network (3.6%). Wider endorsement of trial registration and adherence to reporting guidelines is necessary in clinical trials conducted in Latin America and the Caribbean.

Editorial Policies; Research Design; Periodicals

RESUMEN

Se evaluó el nivel de apoyo y cumplimiento a las iniciativas de registro de ensayos clínicos y de guías de reporte de revistas biomédicas de América Latina y Caribe. Se realizó una búsqueda de ensayos clínicos aleatoria en LILACS. Los ensayos clínicos aleatorios fueron evaluados para determinar si los autores informaron del registro de ensayos clínicos y mencionaron la guía CONSORT. Se evaluaron las instrucciones para los autores de las revistas que publicaron los ensayos clínicos aleatorios con el fin de determinar las indicaciones, en relación al registro de ensayos clínicos, las guías de reporte y el ICMJE. Se identificaron 477 citas; una muestra aleatoria de 240 de ellas detectó 101 ensayos clínicos aleatorios. Un 19,8% de los ensayos clínicos aleatorios informaron del registro de ensayos clínicos y un 6,9% fueron registrados prospectivamente; un 3% de los ensayos clínicos aleatorios aludieron a CONSORT. Un 68% de las 56 revistas mencionaron a ICMJE y un 36% requiere el registro de ensayos clínicos. La indicación acerca de otras guías de reporte fue infrecuente: CONSORT 13%, PRISMA 1,8% y STROBE 1,8% y la red EQUATOR 3,6%. Es necesario un mayor apoyo a las iniciativas de registro de ensayos clínicos y guías de reporte en América Latina y Caribe.

Políticas Editoriales; Proyectos de Investigación; Publicaciones Periódicas

RESUMO

Avaliou-se o nível de apoio e cumprimento às iniciativas de registro de ensaios clínicos e aos guias de notificação de periódicos científicos biomédicos da América Latina e do Caribe. Realizou-se uma busca de ensaios clínicos randomizados no LILACS. Esses ensaios foram avaliados para determinar se os autores notificaram o registro de ensaios clínicos e mencionaram as normas do CONSORT. Avaliou-se as instruções para os autores dos periódicos que publicaram ensaios clínicos randomizados a fim de determinar as normas em relação ao registro de ensaios clínicos, os guias de notificação e ao ICMJE. Identificou-se 477 citações; uma amostra aleatória de 240 delas detectou 101 ensaios clínicos randomizados. Dezenove vírgula oito por cento dos ensaios clínicos randomizados reportaram o registro de ensaios clínicos e 6,9% foram registrados antecipadamente; 3% destes ensaios clínicos aludiram ao CONSORT. Sessenta e oito por cento das 56 revistas mencionaram o ICMJE e 36% requerem o registro de ensaios clínicos. Indicação acerca de outros guias de notificação não foi frequente: CONSORT 13%, PRISMA 1,8%, STROBE 1,8% e a rede EQUATOR 3,6%. É necessário um maior apoio às iniciativas de registro de ensaios clínicos e de guias de notificação na América Latina e no Caribe.

Políticas Editoriais; Projetos de Pesquisa; Publicações Periódicas

Introduction

Adherence to guidelines for transparent and accurate reporting of research studies ¹ and clinical trial registration ² by researchers and biomedical journals increases the value and utility of health literature. The Pan American Health Organization (PAHO), regional office for the Americas of the World Health Organization (WHO), promotes the generation of high-quality health research, the enhancement of research standards and the dissemination of research findings ^3,4. PAHO/WHO have taken several steps to advance the quality of health research, including the development of standards for clinical trial registration, through the launch, support and promotion of the International Clinical Trial Registry Platform (ICTRP) ⁵, and promoting the use of reporting guidelines ⁶.

Clinical trial registration is now an essential part of the scientific process and is assumed to reduce bias in health literature, promote higher levels of research accountability, address selective reporting, and make trial data accessible for completed and ongoing trials. Trial registration engages a wide network of researchers within the scientific community by distributing crucial information between researchers, ethics committees, sponsors, care providers and trial participants ^1,7. In 2007, the Latin American and Caribbean Center for Information and Health Sciences (BIREME), a specialized regional center of the PAHO, recommended that journals indexed in LILACS and SciELO follow the WHO orientations on clinical trial registration ^8,9,10 and 56 Latin American and Caribbean journals publicly endorsed trial registration ^{11,12,13,14,15}. However, despite a growing number of trial registrations in the last decade in Latin American and Caribbean countries, there is still room for improvement ¹⁵.

Another factor that often diminishes the benefits of published research evidence and prevents its effective use by the wider scientific community is poor quality reporting. Once a study has been completed, the only way of assessing design quality and how the study was conducted is through the published article. Incomplete, selective or misleading reporting may impact the perceived validity of results and undermine the utility of such research ¹.

Since July 2010, the PAHO has partnered with the EQUATOR Network (http://www.equator-network.org/) to improve health research reporting in the Americas. As part of this partnership, the EQUATOR site is now available in Spanish (http://www.espanol.equator-network.org/), benefiting countries from the Latin America and the Caribbean region ¹⁶. The EQUATOR site offers more than 200 specialized reporting guidelines which are easily accessible online ². Several are also available in Spanish, including CONSORT (for reporting of randomized controlled trials) ¹⁷, STARD (reporting of diagnostic accuracy studies) ¹⁸, STROBE (reporting of observational studies in epidemiology) ¹⁹, and PRISMA (reporting of systematic reviews) ²⁰. Despite a general trend towards improving the quality of reporting randomized controlled trials, currently few Latin American and Caribbean journals endorse the EQUATOR-listed reporting guidelines ²¹ and the quality of reporting is still below acceptable international standards ²².

There is a fundamental need for clinical trial registration and adherence to reporting guidelines for the improvement of health research in Latin America and the Caribbean. The objective of our study was therefore to determine to what extent Latin American and Caribbean biomedical journals have endorsed and implemented clinical trial registration and reporting guidelines.

Methods

Study design

We conducted a cross-sectional study to evaluate the extent to which LILACS indexed Latin American and Caribbean biomedical journals that publish randomized controlled trials have endorsed and adhered to clinical trial registration and reporting guidelines.

Sample

• Eligibility criteria

The following inclusion criteria were adopted: (1) articles published between January 1st and December 31st 2011 in a LILACS indexed journal; (2) the article was published as an original article; (3) the study groups were randomly assigned (the report had to explicitly use the word "random" or variations thereof); (4) only studies randomizing individuals or clusters of individuals were included (for example randomization of extracted teeth, biopsies etc., were excluded); (5) randomized clinical trials were conducted in at least one Latin American or Caribbean site; (6) duplicate randomized clinical trials were excluded.

• Selection of randomized clinical trials and data extraction

LILACS was used to identify randomized controlled trials published in Latin American and Caribbean journals. LILACS contains 856 journals from 18 Latin American and Caribbean countries, 82 of which overlap with MEDLINE ²³. A structured search (Figure 1) developed by LILACS ¹⁸ was used to identify randomized clinical trials reported in these journals between January 1st and December 31st 2011. The search identified 477 references of which 240 citations were randomly selected a priori using a random number sequence generated in Excel (Microsoft Corp., USA). The titles and abstracts of these articles were screened for inclusion. In cases of doubt the full article was obtained to assess whether it met the inclusion criteria. Data was extracted from full papers into pre-prepared and piloted data extraction forms. Journal and study selection, and data extraction was performed by two reviewers and disagreements were resolved by consensus.

Outcomes

• Data extracted from the randomized clinical trials included in the study sample

Information was collected for the following items: reference; first author's country affiliation within Latin America and the Caribbean; registration in an International Clinical Trials Registry recognized by the ICTRP network mentioned in the abstract; and the use of or mention of the CONSORT statement. Each registered randomized clinical trial was searched via the ICTRP search portal (http://apps.who.int/trialsearch/) to assess if the study was prospectively or retrospectively registered ¹⁵.

• Data extracted from journals' online instructions for authors

The following information was collected: journal name; journal country; information regarding the endorsement of the Uniform Requirements for Manuscripts Submitted to Biomedical Journals developed by the International Committee of Medical Journal Editors (ICMJE); mention of major reporting guidelines such as the CONSORT statement, PRISMA, STARD and STROBE ^{16,17,18,19,20} (all available in Spanish in the EQUATOR Network website); and requirement for trial registration ^24,25. No restriction was made with respect to language.

Criteria for establishing categories

We divided journals into those that required trial registration and those that did not. No further categories were created since few journals specifically supported the CONSORT statement.

Statistical analysis

We conducted a descriptive analysis of all evaluated articles and journals. Data obtained was analyzed using SPSS version 17.0 (SPSS, Inc., Chicago, USA). Chi-square (χ² statistic) was used to determine the degree of independence between categorical variables. The difference was considered statistically significant when the result was p < 0.05.

Results

A total of 240 clinical trials were randomly selected from a total search result of 477 references. A total of 101 articles concerning studies conducted in eight different countries and published in 56 different journals from 10 countries met the inclusion criteria. The majority of articles were by Brazilian researchers (72.3%) and published in Brazilian journals (73.3%).

The ICMJE was mentioned in the online author instructions by 68% (34/56) of the journals while 36% (20/56) of the journals required prospective trial registration. Trial registration was reported by at least one randomized clinical trial in 20% of journals (11/56). In only one journal (which had over two randomized clinical trials) were all randomized clinical trials registered. Trial registration was mentioned in the abstract of 8.9% (9/56) of the articles and in the full text (full manuscript including the abstract) in 19.8% of the papers included in the study. We found that 6.9% (7/56) of the randomized clinical trials were prospectively registered in a clinical trials registry recognized by the ICTRP.

Advice on major reporting guidelines was rarely given in the online author instructions – CONSORT (7 journals; 13%), PRISMA (1 journal; 1.8%), STROBE (1 journal; 1.8%) – and the EQUATOR network was acknowledged by only 2 journals (3.6%). CONSORT was mentioned in only three (3.0%) randomized clinical trials.

There was no significant difference in the proportion of registered trials between journals that mentioned the ICMJE guidelines and those that did not. Although a trend was noted, there was no significant difference in the proportion of registered trials between journals that required trial registration and those that did not (p = 0.073).

Discussion

To the best of our knowledge, this study is one of the first in-depth investigations of endorsement of and compliance with clinical trial registration and reporting guidelines by Latin American and Caribbean biomedical journals. Results show that the level of endorsement and compliance is variable and generally low, corroborating similar findings of studies carried out in other regions of the world ²⁶.

The results show that some Latin American and Caribbean biomedical journals do not seem to prioritize adherence to trial registration and reporting guidelines or appear to create barriers to adherence and/or compliance. We found that although 68% of the journals included in this study mentioned the ICMJE, evidence of prospective trial registration was found in only a few randomized clinical trials published by these journals. Furthermore, advice on reporting guidelines was rarely provided by journals in the instructions to authors. These results clearly indicate that Latin American and Caribbean biomedical journals should strive to standardize trial registration and the use of reporting guidelines. Currently, journals are not enforcing trial registration and as a result content may suffer ²⁷. Reporting guidelines are also necessary for adequate scientific reporting as scientific journal editors and authors are concerned about the ethical principles of their journals ²⁸.

Several steps need to be taken to ensure trial registration and compliance with reporting guidelines by Latin American and Caribbean biomedical journals. First, the importance of trial registration should be ingrained into the journal's culture. Journals should create a system for the submission of manuscripts which requires authors to comply with a checklist of requests to successfully submit an article, including a declaration of registration and statement of responsibility citing that the author is complying with all requirements, thus ensuring author's accountability. Furthermore, ethics review committees should impose trial registration as a condition for final approval of informed consent ²⁹.

More efforts should be made to promote the use of reporting guidelines available in the EQUATOR Network library and training should be offered to Latin American and Caribbean peer reviewers, editors and authors in the use of these reporting guidelines. Reviewers and editors should act as gatekeepers, ensuring that all accepted articles adhere to the stated reporting guidelines. Databases such as SciELO and LILACS have a fundamental role to play in promoting the use of reporting guidelines and improving the standardization of reporting requirements detailed in authors' instructions.

Our study has a number of limitations. Although approximately 900 Latin American and Caribbean biomedical journals are indexed in LILACS, the sample does not represent all Latin American and Caribbean journals and countries such as Brazil are overrepresented. In addition, the search strategy for identifying randomized trials in the LILACS database has not yet been validated and some randomized clinical trials and journals may have been missed. Finally, we did not formally evaluate the quality of reporting among the randomized clinical trials. The fact that authors did not explicitly mention CONSORT does not necessarily imply that the checklist of recommendations was not adopted by the study and the fact that CONSORT was cited does not mean that it was correctly implemented.

We hope that this paper will encourage researchers from the region to conduct investigations into the quality of reporting and trial registration to monitor compliance, identify barriers and assess potential solutions.

Contributors

All authors contributed to study conception and planning, data interpretation and drafting of the final manuscript. L. Reveiz carried out the search and was responsible for extracting, evaluating, analyzing and summarizing data from the studies. C. Iko carried out the search and was responsible for extracting data from studies.

Conflict of interests

None.

Submitted on 18/Dec/2012

Final version resubmitted on 14/Feb/2013

Approved on 20/Feb/2013

• 1. Krlea-Jeriç K, Lemmens T, Reveiz L, Cuervo LG, Bero LA. Prospective registration and results disclosure of clinical trials in the Americas: a roadmap toward transparency. Rev Panam Salud Pública 2011; 30:87-96.
• 2. Simera S, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med 2010; 8:24.
• 3. Pan American Health Organization. PAHO's policy on research for health. http://new.paho.org/hq/index.php?option=com_docman&task=doc_download&gid=14557&Itemid=&lang=en (accessed on Oct/2012).
• 4. World Health Organization. The WHO strategy on research for health (2010). http://www.who.int/phi/WHO_Strategy_on_research_for_health.pdf (accessed on Oct/2012).
• 5. Ghersi D, Pang T. En route to international clinical trial transparency. Lancet 2008; 372:1531-2.
• 6. Simera I, Reveiz L. New alliance between the Pan American Health Organization and the EQUATOR Network to improve writing and publication of articles on health outcomes research. Med Clin (Barc) 2011; 137:208-10.
• 7. Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F, et al. Clinical trial registration looking back and moving ahead. N Engl J Med 2007; 356:2734-6.
• 8. Latin American and Caribbean Center on Health Sciences Information. Registry of clinical trials (2007). http://metodologia.lilacs.bvsalud.org/php/level.php?lang=en&component=34&item=43 (accessed on Oct/2012).
• 9. Packer A. Recommendation for publishing clinical trials in journals indexed in LILACS and SciELO. Rev Chil Pediatr 2007; 78:334.
• 10. Gozzani JL. Comitê de ética, conflito de interesses e registro de ensaios clínicos. Rev Bras Anestesiol 2008; 58:91-4.
• 11. Rocha FA, Xavier RM. Registro de protocolos clínicos. Rev Bras Reumatol 2007; 47:149.
• 12. Stegemann H. The International Clinical Trials Registry Platform ICTRP. Arch Latinoam Nutr 2007; 57:311-2.
• 13. Caramelli B. The mandatory public record of clinical trials. Rev Assoc Med Bras 2005; 51:301.
• 14. Palacios M, Ramírez JH. Editorial: clinical trials registration and the "instead of" research phenomenon. Colomb Med 2008; 38:5-6.
• 15. Reveiz L, Bonfill X, Glujovsky D, Pinzon CE, Asenjo-Lobos C, Cortes M, et al. Trial registration in Latin America and the Caribbean. J Clin Epidemiol 2012; 65:482-7.
• 16. EQUATOR Network. Library for health research reporting. http://www.equator-network.org/resource-centre/library-of-health-research-reporting/ (accessed in Oct/2012).
• 17. Schulz KF, Altman DG, Moher D, Fergusson D. CONSORT 2010 changes and testing blindness in RCTs. Lancet 2010; 375:1144-6.
• 18. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007; 147:573-7.
• 19. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. BMJ 2003; 326:41-4.
• 20. Moher D, Liberati A, Tetzlaff J, Altman DG; The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. J Clin Epidemiol 2009; 62:1006-12.
• 21. Costa L, Maher C, Lopes A, Noronha M, Costa L. Transparent reporting of studies relevant to physical therapy practice. Rev Bras Fisioter 2011; 15:268.
• 22. Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ 2010; 340:c723.
• 23. Latin-American and Caribbean Center on Health Sciences. LILACS search tutorial. http://wiki.bireme.org/en/index.php/LILACS_search_tutorial (accessed in Oct/2012).
• 24. DeAngelis C, Drazen JM, Frizelle FA, Haud C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med 2004; 351:1250-1.
• 25. DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors. JAMA 2005; 293:2927-9.
• 26. Zhang D, Freemantle N, Cheng K. Are randomized trials conducted in China or India biased? A comparative empirical analysis. J Clin Epidemiol 2011; 64:90-5.
• 27. van de Wetering FT, Scholten RJPM, Haring T, Clarke M, Hooft L. Trial registration numbers are underreported in biomedical publications. PLoS One 2012; 7:e49599.
• 28. Fernandez M, Quieroz M, Moraes M, Barbosa M, Sousa A. Ethical standards adopted by Brazilian journals of medical specialties. Rev Assoc Med Bras 2011; 57:264-8.
• 29. Reveiz L, Cardona AF, Ospina EG. Clinical trial registration. N Engl J Med 2005; 352:198-9.

Publication Dates

• Publication in this collection
  04 Mar 2013
• Date of issue
  June 2013

History

• Received
  18 Dec 2012
• Accepted
  20 Feb 2013