Publication of studies presented as free papers at a Brazilian national orthopedics meeting

Abstracts

OBJECTIVE:

To evaluate the rate and others factors related with the publication of free papers presented at a national orthopedic meeting.

METHODS:

Using virtual databases we reviewed the studies presented at the 2004 Brazilian Congress of Orthopedics and Traumatology that were published, as well as related factors such as institution, sub-specialty, year, level of evidence, results and comparison between abstract presented and published.

RESULTS:

There were 58 studies published from 267 presented (21.73%). Seven (12.1%) were published in international and 51 (87.9%) in national journals, mainly RBO (55%). The publication rate was higher in the year of the event and the following year (37.9%). The sub-specialties of spine and knee showed the best correlation between the numbers of papers presented and published (respectively 40.9% and 37.9%). Most of the studies were cohort (65%) and experimental studies have been 34.5%. There was a tendency to publish positive results or statistically significant. Three institutions were responsible for most of the publications (53.4%). Works with higher level of evidence showed the highest rate of publication. The abstract published was modified in 68.5% of the form the presentation.

CONCLUSIONS:

This study presents data that compromises the quality of scientific of the abstracts presented at orthopedic meeting: most have a lower level of evidence and nearly 80% are not published.

Research ; Congresses ; Publications

OBJETIVO:

Avaliar o índice de publicação e os fatores relacionados de trabalhos apresentados na forma de temas livres em congresso ortopédico.

MÉTODOS:

Em consulta a bases de dados virtuais foram identificados os temas livres do Congresso Brasileiro de Ortopedia e Traumatologia de 2004, os quais resultaram publicados. Também foram analisados os fatores relacionados, como instituição, subespecialidade, ano, nível de evidência, resultados e comparação entre resumo apresentado e publicado.

RESULTADOS:

Foram publicados 58 artigos a partir dos 267 apresentados (21,73%), sete (12,1%) em revistas internacionais e 51 (87,9%) em nacionais, a mais frequente a Revista Brasileira de Ortopedia (RBO) (55%). A maior taxa de publicação ocorreu até o fim do ano seguinte do evento (37,9%). As subespecialidades de coluna e joelho apresentaram a melhor correlação entre número de trabalhos apresentados e publicados (respectivamente 40,9% e 37,9%). A maioria dos estudos foi de coorte (65%) e estudos experimentais corresponderam a 34,5%. Houve tendência de publicação de resultados positivos ou estatisticamente significativos. Três instituições foram responsáveis pela maioria das publicações (53,4%). Trabalhos com maior nível de evidência e estudos anatômicos apresentaram maior índice de publicação. O texto do resumo publicado foi modificado em 68,5% dos trabalhos, mas sem alterar o teor da conclusão.

CONCLUSÕES

: Este estudo apresenta dados que comprometem a qualidade científica dos temas livres apresentados em congresso da Sociedade Brasileira de Ortopedia e Traumatologia (SBOT): a maioria tem menor nível de evidência e quase 80% não são publicados.

Pesquisa ; Congressos ; Publicações

Introduction

The presentation of free papers is considered to be an opportunity to demonstrate research lines, enhance and transfer knowledge, and seek professional improvement. Although a series of factors that may or may not be related to scientific quality might influence the value of the free papers that are presented, the publication rate is the primary instrument that is used to measure the scientific quality of such conferences.

According to a recent study, fewer than 50% of the abstracts presented at the 2001. American Academy of Orthopaedic Surgery (AAOS) annual meeting were published.¹ A similar study was conducted in the United Kingdom, where the publication rate was 33%.² In a study conducted by the American Orthopaedic Society for Sports Medicine (AOSSM) and the Arthroscopy Association of North America (AANA), 59.5% of the oral presentations were published.³

Brazilian studies in the fields of vascular surgery and sports medicine suggest that Brazilian indices are lower, with rates of 6.3% and 12%, respectively.^4-5 Although free papers are considered to be an important element of the Brazilian Congress of Orthopedics and Traumatology, we could not find studies that discussed the quality of these presentations. It is also essential to determine whether the studies maintain the data in the published version or why they have not been published.^6,7

The objectives of this study are as follows: 1) to evaluate the publication rate of and other factors related to the presentation of free papers in a Brazilian Conference of Orthopedics and Traumatology and 2) to check for possible textual changes in the published versions of these papers.

Materials and Methods

This study was conducted based on the official program of the 36^th Brazilian Congress of Orthopedics and Traumatology (2004), in which 270 free papers were presented. To identify possible publications, we searched online databases (Bireme, LILACS, SciELO, Medline-PubMed, Google Scholar and online RBO) for studies using the authorsâ€(tm) names and the abstract titles in Portuguese and English.

All of the papers published in scientific journals, masterâ€(tm)s or doctoral theses and online databases were included. The search included all publications that were published through the date of this study (2012).

We determined the absolute number of publications, the year and journal (national or international), the related subspecialty (trauma, knee, spine, shoulder, pediatric orthopedics, ankle and foot, hip, hand and oncology), the lead institutions, the type of study and the positivity of the results. The studies that showed statistically significant results (p ≤ 0.05) were considered to be positive in the case of experimental or comparative studies and favorable (classified as excellent, good or satisfactory) in the case of cohort studies.

The papers were classified by the level of evidence according to the classification published in the Journal of Bone and Joint Surgery, and the publication rate (percentage of papers published) was obtained for each level of evidence. Anatomical and experimental studies were considered separately. Finally, the abstracts of the presentations were compared with the papers published. We evaluated changes in the title, authors, objectives, materials and methods, results and conclusions.

The present study was approved by the Education and Research Commission of the Universidade Federal do Paraná (UFPR).

Results

Of the 270 presentations of free papers, we found that three papers were duplicate publications. A total of 267 papers were searched for in the database, and we found that 58 papers had been published (21.7%).

Three papers (5.2%) were published in the period before the congress (until 2003). In the year when the congress occurred, 12 papers (20.7%) were published; in the following year (2005), 10 papers (17.2%); in 2006, 12 papers; in 2007, seven papers (12.1%) were published; in 2008 and 2009, two papers (3.4% per year) each year; in 2010, seven papers (12.1%); in 2011, only one paper (1.7%); and in 2012, two papers (3.4%).

Of all papers, 51 were published in national publications (87.9%): 28 in the Revista Brasileira de Ortopedia (RBO) (55%) and nine in the Acta Ortopédica Brasileira (17.7%). Two papers were published in the Revista Brasileira de Medicina, Clinics e Coluna/Columna (corresponding to 4% each). The following journals also published one study each: Revista Brasileira de Medicina do Esporte, Acta Cirúrgica Brasileira and Revista da Associação Médica Brasileira (1.9% each). Five publications were masterâ€(tm)s or doctoral theses (9.8%). The other seven papers (12.1%) were published in international journals: ^{European Journal of Orthopaedic Surgery and Traumatology, Journal of Spinal Cord Medicine,} Journal of Bone and Joint Surgery, Journal of Hand Surgery, Techniques in Shoulder and Elbow Surgery, Transplantation Society Journal and Journal of Physical Therapy.

The trauma and knee subspecialties had the highest absolute number of publications (11 each). However, the spine subspecialty showed the best relation between the number of oral presentations and the number of publications: of 22 free papers, nine (40.9%) were published. The other fields that were examined showed the following relations: the knee subspecialty included 29 presentations and 11 publications (rate of 37.9%); the pediatric orthopedics had 20 and 7, respectively (35%); the trauma subspecialty had 39 and 11 (28.2%); the foot surgery subspecialty had 22 and 5 (22.7%); the shoulder subspecialty had 40 and 8 (20%); the hip subspecialty had 23 and 4 (17.4%); the hand surgery subspecialty had 24 and 2 (8.3%); and the tumor subspecialty, 18 and 1 (5.5%).

Regarding the institutions from which studies were published, Santa Casa de Misericórdia de São Paulo presented the highest number of publications, with 14 papers (24.2%). The other institutions with multiple publications were as follows: the Universidade Estadual de São Paulo, with 11 papers (18.9%); the Universidade Federal do Estado de São Paulo São Paulo, with six papers (10.3%); the Instituto Nacional de Traumatologia e Ortopedia (The Brazilian National Institute of Orthopedics and Traumatology, with five papers (8.6%); and the Universidade de Campinas and the Universidade Estadual de São Paulo in Ribeirão Preto, with three papers each (5.2%). Sixteen additional institutions contributed, with one publication each.

Concerning the positivity of the results, of the 58 presentations, 20 (34.5%) were experimental studies, of which 75% presented statistically significant results (p < 0.05). Of the other 38 presentations, eight were anatomical or observational studies; therefore, the results could not be analyzed. The remaining 30 presentations were cohort studies or case series, of which 29 showed positive results and one showed non-positive results.

Regarding the level of evidence, level IV was most prevalent, with 207 papers, followed by level II, with 27 studies. Eight studies were classified as level V, and seven studies were classified as level III. Level I studies were not observed. Eighteen studies were anatomical studies. The level of evidence was established for the published papers and for the presentations. The publication rate for level II was 59.2% (16 publications and 27 presentations); for level III, the publication rate was 57.1% (four publications and seven presentations); for level IV, the publication rate was 14.5% (30 publications and 207 presentations); and for level V (0 publication and 8 presentations), the publication rate was 0%. For the anatomical studies, the publication rate was 44.4% (8 publications and 18 presentations), and for the experimental studies, the publication rate was 58.8% (20 publications and 34 presentations).

Comparing the abstract presented at the congress with the final published abstract, 40 papers (68.9%) revealed changes in the final text regarding authorship, titles, objectives, materials and methods, results and conclusions. Most of the changes were observed in the titles and authorship (22 and 31, respectively); however, three papers had changes in the objectives section, seven had changes in the materials and methods section, nine had changes in the results section, and nine had changes in the conclusions. No publication presented conflicting conclusions compared with the congress presentations.

Discussion

It is important to present free papers in congresses as it promotes a faster propagation of the study. Other professionals may change their medical conduct based on the data presented in this type of presentation. Furthermore, 53% to 63% of textbook chapters include data derived from the presentations of free papers, although such presentations do not always provide the data necessary for qualitative evaluation.⁷ However, the greatest objective of a study is publication in a scientific journal, which permits broader visibility and more permanent promotion, with better quality. Doubts may arise regarding the quality and scientific value of free papers and whether the data will be maintained in a future publication.⁷

One of the simplest ways to evaluate the quality of congress presentations is to use their publication rate. International studies have shown publication rates to vary between 34% and 61%.^1-6 An assessment of the presentations at the 2004 American Academy of Orthopaedic Surgery (AAOS) showed a 52% publication rate in five years for the oral presentations and a 47% publication rate for the poster presentations.¹ Although the authors considered these rates to be low, the rates were shown to be higher than a previous finding published by Bhandari et al.,⁷ with 34% of the abstracts from a 1996 conference being published, up to five years after the event. An even lower rate (35%) was found in an Australian study.⁸

In our study, the 21.7% publication rate was lower than the rates reported in international studies; however, it was higher than those reported in several other Brazilian studies on this topic. For a congress organized by the Centro de Estudos do Laboratório de Aptidão Física de São Caetano do Sul (SP) (Study Center of the Physical Capability in São Caetano do Sul, in São Paulo), the publication rates were 12% for oral presentations and 2% for posters.4 For the Brazilian Congresses of Angiology and Vascular Surgery in 2001 and 2003, the average publication rate was 6.3%.⁵ Another study only conducted a survey of journals that publish findings in the fields of orthopedics and traumatology.⁹

One of the factors that may have influenced the higher publication rate was the greater follow-up gap (eight years, from 2004 to 2012) in the studies mentioned. Although most of the papers were published within five years after the congress, 10 papers (17%) were published after this period. In our survey, we included masterâ€(tm)s and doctoral theses as publications, but these types of papers are not considered to be publications in international studies. If we had excluded these types of publications, our publication rate would have been lower than 20%.

This study did not explore the reasons for not publishing papers after their presentations. Sprague et al.¹⁰ conducted a study on the authors who did not publish full texts following their abstract presentations at the 1996 AAOS. According to the 199 authors who completed the questionnaire, the main reason for not submitting the full text for publication was the lack of time for additional research (47%). Another 31% reported that their study was still in progress, whereas other authors mentioned that the manuscript was someone elseâ€(tm)s responsibility or that the pursuit of publication was not a priority. The lack of time was also mentioned by 42% of the authors in a similar study conducted by Weber et al.¹¹ on emergency medicine. The factors related to the type of study were not mentioned by the authors. Although methodological failures were not mentioned by the authors as a reason for nonpublication, such failures can influence nonpublication.^6,7,10

According to the literature, authors tend to publish studies that demonstrate results that they regard to be positive.⁶ In a systematic review, Scherer et al.¹² noticed an evident association between publications and positive results (significant results or results that show the experimentâ€(tm)s positivity).⁶ Our study confirms this fact, both for experimental works, for which 75% of the results were considered to be significant, and cohort studies. Of 24 published papers, only one did not show positive results.

As expected, nearly all of the papers examined in the present study were published in national journals, especially in RBO, and only 12% were published in international journals. The subspecialties with the highest rate of publication were spine and knee orthopedics. Additional studies must be conducted to monitor whether these publication rates remain constant in subsequent years because the number of presentations per specialty is not high, and thus, a small variation in the presentation/publication ratio can substantially alter the publication rate. As expected, a majority of the published studies were conducted at institutions in the state of São Paulo.

In the present survey, half of the publications were cohort studies, followed by experimental studies. The lack of higher quality studies, such as prospective studies, controlled studies or even comparative studies, demands our attention. We have noticed that studies with a higher quality of evidence, as well as anatomic and experimental studies, tend to have a higher presentation rate. Studies with level II evidence showed a 36% publication rate, whereas studies with level IV evidence (the most common evidence level among the studies analyzed, with more than 80% of the studies having level IV evidence) showed a publication rate of only 17%. In a survey with 79 studies, Scherer et al. reported strong relation between experimental studies with better quality (randomized trials or controlled clinical studies) and eventual publication.¹²

It is also important to verify whether the data presented in a publication differ from the data in the oral presentation. Bhandary et al.⁷ found inconsistencies between the materials and methods sections in 14% of studies and inconsistencies between the results sections in 19% of studies. They considered these inconsistencies to be important changes. In our assessment, approximately 2/3 of the papers presented changes; most of the papers had discrepancies between the titles and the lists of authors. The percentage of studies with changes in the materials and methods sections was comparable to the percentage reported by Bhandary et al.⁷ Although no publications presented different conclusions compared with the presentations, the high number of studies with other changes suggests that the texts were revised. Such changes in content are another factor against the scientific reliability of the studies that are presented as free papers.

It should be taken into account that the findings in our study are based on the data from one event that occurred eight years ago. Data may vary from year to year, and new surveys must be conducted to consolidate results and evaluate whether there are general trends, as verified in the American Academy of Orthopaedic Surgery (AAOS) congresses by Voleti et al.¹³ When evaluating the quality of free papers presented between 2001 and 2010, these authors found a substantial increase (17% to 36%) in level I and II studies; they also found a 50% decrease in level IV studies, which used to represent 62% in the same period. Our study showed much lower results, with only 11% of studies classified as level I or II and more than 80% of studies classified as level IV. Comparing our results with those of the AAOS event that occurred in the same year, there is an evident disadvantage, with respective rates of 21% and 54%. We hope that the tendency toward an increasing proportion of studies with higher evidence levels at our events results in a greater number of publications.

In summary, although the orthopedic publication rate reported herein is higher than that reported in other national studies, it is still unsatisfactory. The advancement of new knowledge occurs more widely, permanently and efficiently through publication in medical journals, and the publication of a text with a study review confers enhanced scientific prestige and credibility to both the article and the authors. The fact that 80% of orthopedic presentations are not published â€" and that those that are published often have content changes â€" calls attention to the likelihood of a lack of quality in the scientific studies that are presented as free papers. The change or adoption of new medical procedures based on studies presented as free papers, especially cohort studies or case series, should be further exploerd.

REFERENCES

• ¹
  Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468(5):1428-35.
• ²
  Guryel E, Durrant AW, Alakeson R, Ricketts DM. From presentation to publication: the natural history of orthopaedic abstracts in the United Kingdom. Postgrad Med J. 2006;82(963):70-2.
• ³
  Yoo S, Oh G, Wang JC. Publication rates of presentations made at annual meetings of the American Orthopaedic Society for Sports Medicine and the Arthroscopy Association of North America. Am J Orthop (Belle Mead NJ). 2002;31(6):367-9.
• ⁴
  Fernandes IA, Alvaresi TS, Arteiro SM, Vianna LC, Araújo CGS. Apresentação de temas livres em eventos científicos de ciências e medicina do esporte e publicação em periódicos indexados. Rev Bras Cineantrop Desem Hum. 2008;10(1):50-55.
• ⁵
  Bonetti W, Holmo NF, Corregliano GT, Baldon KM, Souza e Silva N. Publicações indexadas geradas a partir de resumos de congressos de angiologia e cirurgia vascular no Brasil. J Vasc Bras. 2008;7(4):293-7.
• ⁶
  Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994;272(2):158-62.
• ⁷
  Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84(4):615-21.
• ⁸
  Walby A, Kelly AM, Georgakas C. Abstract to publication ratio for papers presented at scientific meetings: how does emergency medicine compare? Emerg Med (Fremantle). 2001;13(4):460-4.
• ⁹
  Camargo OP, Leme LEG. Revisão de artigos de ortopedia e medicina esportiva publicados em periódicos brasileiros. Acta Ortop Bras. 2009;17(6) 344-9.
• ¹⁰
  Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta P 3rd, Cook DJ, et al. Barriers to full-text publication following presentation of abstracts at annual orthopedic meetings. J Bone Joint Surg Am. 2003;85(1):158-63.
• ¹¹
  Weber EJ, Callaham ML, Wears RL, Barton C, Young G. Unpublished research from a medical specialty meeting: why investigators fail to publish. JAMA. 1998;280(3):257-9.
• ¹²
  Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;(2):MR000005.
• ¹³
  Voleti PB, Donegan DJ, Baldwin KD, Lee GC. Level of evidence of presentations at American Academy of Orthopaedic Surgeons annual meetings. J Bone Joint Surg Am. 2012;94(8):e50.

Publication Dates

• Publication in this collection
  May/June 2013

History

• Received
  04 Sept 2012
• Accepted
  03 Oct 2012