Brazilian Scientific Journals in Surgery: quality control in the abstract structure of non-experimental articles

14 - ORIGINAL ARTICLE

BIBLIOMETRICS

Brazilian Scientific Journals in Surgery. Quality control in the abstract structure of non-experimental articles¹

Carlos Alberto Guimarães^I; Rosely de Fátima Pellizzon^II; Marcia Kiyomi Koike^III

^IPhD, Associate Professor, Department of Surgery, School of Medicine, Rio de Janeiro Federal University, Brazil. Acquisition of data, supervised all phases of the study, manuscript writing and critical revision

^IILibrarian, Central Library, UNIFESP, Sao Paulo-SP, Brazil. Acquisition of data, selection of descriptors (MeSH/DeCS), critical revision, references correction

^IIIPhD, Associate Professor, University City of Sao Paulo (UNICID), Brazil. Technical procedures, data analysis, processing of study information, critical revision

^{Correspondence}

ABSTRACT

PURPOSE: To evaluate the quality of abstracts of original non-experimental research articles in Brazilian Journals in Surgery.

METHODS: Convenience sample of 471 abstracts of original research articles from six Brazilian surgical journals indexed in Thomson Reuters (ISI) Web of Knowledge. The quality of abstracts was measured against a checklist of eight evaluation criteria, which were divided into 32 categories. The total score for each abstract was obtained by summing the score of all criteria present. The overall mean score was also determined.

RESULTS: The overall mean score of abstract quality was that of a good abstract. Most of the abstracts contained some information from each of the eight basic categories of an abstract. All abstracts were structured ones.

CONCLUSION: The overall quality, for abstracts of original articles of six Brazilian non-experimental journals in surgery, was classified as good.

Key words: Abstracting and Indexing as Topic. Periodicals as Topic. Quality Improvement. Quality Control. General Surgery. Bibliometrics

Introduction

An article's abstract is a powerful tool for the reader as well as the author¹. Except for the title, the part of a scientific article that will be seen and read by the most people is the abstract. Abstracts should provide all the necessary and important information on the research performed (e.g., the study's purpose, design, results and conclusions). They enable readers to review relevant features of the research without having to read the entire report. However, investigators have demonstrated that abstracts can be misleading or biased and that the entire article should still be read^2,3.

Improving the quality of scientific literature has been advocated since the inception of scientific publication, in 1665³. In the sixties, the summary and conclusions of articles were moved to the beginning of each report in the Journal of the American Medical Association (JAMA). JAMA and the Canadian Medical Journal were the first to adopt this new role for abstracts, and other journals, including Lancet, the New England Journal of Medicine and the Annals of Internal Medicine, followed suit soon thereafter. Today it is the rare biomedical publication which does not feature a summary or an abstract at the beginning of each major scientific report. These abstracts followed the IMRAD structure (Introduction, Methods, Results And Discussion⁴).

In 1987, the Ad Hoc Working Group for Critical Appraisal of the Medical Literature published a proposal for more informative abstracts of clinical articles. This structured abstract had seven topics: Objective, Design, Setting, Patients or Participant, Interventions, Measurements and Conclusions^5,6.

In 1990, Haynes et al.⁷ made a proposal of a structured abstract with eight headings for original articles (Objective, Design, Setting, Patients or participants, Interventions, Main outcomes measures, Results and Conclusions) and six headings for review articles (Purpose, Data sources, Study selection, Data extraction, Results of data synthesis and Conclusions).

A busy health care professional trying to keep abreast of the latest information will likely peruse the abstract to determine whether to read the entire article. Some studies of the quality of traditional abstracts have detected deficiencies. There is some evidence that abstracts tend to present conclusions that do not follow from the findings and that they overemphasize positive conclusions⁸.

The specific objective of this study was to evaluate the quality of abstracts of original research articles in Brazilian journals in non-experimental surgery

Methods

Sample selection

The journals selected for the study were the Brazilian non-experimental surgical journals indexed in Thomson Reuters (ISI) Web of Knowledge: Arquivos Brasileiros de Cardiologia (Brazilian Archives of Cardiology), International Brazilian Journal of Urology, Jornal Brasileiro de Pneumologia (Brazilian Journal of Pulmonology), Revista da Assocação Médica Brasileira (Journal of the Brazilian Medical Association), Revista Brasileira de Cirurgia Cardiovascular (Brazilian Journal of Cardiovascular Surgery) and São Paulo Medical Journal. All journals contain research articles on health care and require an introductory abstract of those articles. We studied all 2010 issues.

Only abstracts of original research articles were eligible. The reasons for this were that (a) most biomedical articles may be categorized as original research, (b) original research articles are an important source of new knowledge for health professionals and (c) previous work has been done in developing guidelines for the evaluation of abstracts of original research articles

Evaluation criteria

The quality of abstracts was measured against a checklist of 32 evaluation criteria, which were divided into eight categories. The key details of each section were identified and criteria selected for each important item. Each criterion was represented by a question on a checklist, against which the abstracts were evaluated. Each category was weighted equally and could be answered as Yes, No or Partially⁸.

Purpose - Yes; N; Partially

Was any information on the purpose given? Was the purpose explicitly stated?

Was the main purpose distinguished from secondary ones?

Research Design - Yes; No; Partially

Was any information on the research design given? Were technical descriptors used? If a follow-up study was the duration given?

Setting - Yes; No; Partially.

Was any information on the setting given? Was the level of clinical care (e.g., primary care) indicated?

Subjects - Yes; No; Partially

Was any information on the subjects given? Were common demographic characteristics given? Were technical descriptors of subject selection (e.g., random or convenience sample) use? Was the number of subjects indicated?

Were the response and refusal rates indicated? Was the number of dropouts and losses indicated? If the samples were matched were matching characteristics given?

Intervention - Yes; No; Partially

Was any information on intervention given? Were the commonest name and common synonyms given? Was a description given? Was the duration indicated?

Measurement of variables - Yes; No; Partially

Was any information on the measures given? Were the variables explicitly identified? Was the source of the data given? If the measurements were subjective were the observers blind to the patient groupings?

Results - Yes; No; Partially

Were any results given? Were they directly related to the purpose? Were appropriate numerical data given?

Conclusions - Yes; No; Partially

Were any conclusions made? Were they directly related to the purpose? Were they consistent with the results? Were the study's limitations mentioned? Were the study's implications mentioned? Were there recommendations for further study?

Each abstract could achieve a quality score of 2 (good information on that category), 1 (poor information on that category and 0 (no information on that category). So, each abstract could have a maximum total score of 16 (score of 2 in each category). An abstract was qualified as "good"with a score > 11. An overall mean score for the sample was calculated.

Assessment

The assessment of the abstracts was made online with all the existing identifications (journal and authors). The abstracts were reviewed independently by two of us (C.A.G. and R.F.P.). The consistent interpretation of the criteria was discussed but communication between the raters was avoided during evaluation. Any disagreements were resolved by consensus.

Results

A total of 471 abstracts were studied. The overall mean score of the abstracts was 13 (good abstract). The scores varied from12 to 16 (Table 1). Most of the abstracts contained some information from each of the eight basic categories of an abstract. All abstracts were structured ones.

The frequency with which the abstracts were classified as good is shown in Table 2.

Discussion

We assessed the quality of a convenience sample of structured abstracts from six Brazilian non-experimental journals in surgery. In this criterion-based observational study, a mean score of 13 indicates that most abstracts were identified as good, so the information expected to be found in the abstracts was present. Findings also revealed that a significant proportion (i.e., 76%) of articles contained a good abstract.

All the abstracts were structured one and we know that the frequency in meeting the specific criteria is generally higher for the structured abstracts than for the nonstructured ones

The types of research articles included for which abstracts were rated did not differ between the six journals (data available on request).

Studies of the quality of abstracts offer editors and readers insight into the shortcomings of abstracts and how they compare with abstracts in other journals. They cal also facilitate improvements in current standards of practice. In the nineties, investigators have begun to assess the quality of abstracts in biomedical journal^3,8-11.

Our study had some limitations. We do not have an independent third party serve as final arbiter of any disagreements. Knowledge of information about the authors and the title of the article can influence the evaluation of an abstract. Blinding of the raters to these factors would reduce the potential for bias.

A further limitation was that the contents of the abstract and the article were not compared to determine whether the quality of the abstract reflected the quality of the article. Studies should be done in this area.

We studied the quality of abstracts of original research. However, abstracts are used to describe other types of articles as well. It is conceivable that the quality of these types of abstracts would be amenable to study.

Conclusion

The overall quality, for abstracts of original articles of six Brazilian non-experimental journals in surgery, was classified as good.

Received: August 06, 2012

Review: October 08, 2012

Accepted: November 11, 2012

Conflict of interest: none

Financial source: none

• 1. Hahs-Vaughn DL, Onwuegbuzie AJ. Quality of abstracts in articles submitted to a scholarly journal: A mixed methods case study of the journal Research in the Schools. Lib Inf Sci Res. 2010;32(1):53-61.
• 2. Pitkin RM. The importance of the abstract [editorial]. Obst Gynecol. 1987;70:267.
• 3. Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR. Quality of nonstructured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. Can Med Assoc J. 1994;150(10):1611-5.
• 4. Soffer A. Abstract of clinical investigations: a new and standardized format. Chest. 1987;92(3):389-90.
• 5. Ad Hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical articles. Ann Intern Med. 1987;106(4):598-604.
• 6. Guimarães CA. Structured Abstracts. Narrative review. Acta Cir Bras. 2006;21(4):263-8.
• 7. Haynes RB, Mulrow CD, Huth EJ, Altmam DG, Gardner MJ. More information abstracts revised. Ann Inter Med. 1990;113(1):69-76.
• 8. Narine L, Yee DS, Einarson TR, Ilersich AL. Quality of abstracts of original research articles in CMAJ in 1989. Can Med Assoc J. 1991;144(4):449-53.
• 9. Wong HL, Truong D, Mahamed A, Davidian C, Rana Z, Einarson TR. Quality of structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association: a 10-year follow-up study. Curr Med Res Opin. 2005;21(4):467-73.
• 10. Dupuy A, Khosrotehrani K, Lebbe C, Rybojad M, Morel P. Quality of abstracts in 3 clinical dermatology journals. Arch Dermatol. 2003;139(5):589-93.
• 11. Hartley J, Sydes M. Are structured abstracts easier to read than traditional ones? J Res Reading. 1997;20(2):122-36.

1

Research performed at Nucleus of Scientific Communication in Health Science (NCCCS), Federal University of Sao Paulo (UNIFESP), Brazil.

Publication Dates

• Publication in this collection
  18 Jan 2013
• Date of issue
  Jan 2013

History

• Received
  06 Aug 2012
• Accepted
  11 Nov 2012
• Reviewed
  08 Oct 2012