Acessibilidade / Reportar erro

Identifying reasons for failure in biomedical research and publishing

Abstract

The regular assessment of Brazilian scientific output means that individual university departments need to constantly improve the quantity and quality of their scientific output. A significant proportion of this output involves the work of Master’s and Doctoral students, but getting this work published in a suitable journal can often prove to be a challenge. Although students’ lack of fluency in English is a contributing factor, many of the problems observed have an early origin in the formulation of the research problem and its relevance to current research trends in the international literature. In short, more time needs to be spent in the library and less in the laboratory, and more effort needs to be made in teaching students basic research skills such as the effective use of bibliographic databases like PubMed, Web of Science and Scopus.

Research failure; Reading; Scientific writing; Scientific literacy


Braz J Med Biol Res, July 2009, Volume 42(7) 589-592 (Concepts and Comments)

Identifying reasons for failure in biomedical research and publishing

Correspondence and Footnotes D. Bousfield

Ganesha Associates, Cambridge, UK

References

Correspondence and Footnotes Correspondence and Footnotes Correspondence and Footnotes

Abstract

The regular assessment of Brazilian scientific output means that individual university departments need to constantly improve the quantity and quality of their scientific output. A significant proportion of this output involves the work of Master's and Doctoral students, but getting this work published in a suitable journal can often prove to be a challenge. Although students' lack of fluency in English is a contributing factor, many of the problems observed have an early origin in the formulation of the research problem and its relevance to current research trends in the international literature. In short, more time needs to be spent in the library and less in the laboratory, and more effort needs to be made in teaching students basic research skills such as the effective use of bibliographic databases like PubMed, Web of Science and Scopus.

Key words: Research failure; Reading; Scientific writing; Scientific literacy

Introduction

In 2006, and then again in 2007 and 2008 I came to Brazil for three months as a visiting professor. Initially my job focussed on trying to help students and their faculty get their work published in reputable Qualis A journals (1) in fields that ranged across all of the biological and many of the clinical sciences. So in addition to editing scores of manuscripts at various stages of completion, I was also able to study the reasons for rejection and see the specific comments made by the referees from many different journals. A pattern began to emerge - manuscripts were being rejected for a small set of common reasons.

The hypothesis, when I first arrived, had been that “the students' English' was the problem, and that the Editors of international journals were rejecting good science that had been poorly translated. However, I was also being given manuscripts in Portuguese and it was possible to see the same sorts of problems associated with these.

I made a list of some of the common problems I was seeing soon after I arrived, and although I have read a lot more manuscripts since then, this list has not really changed. For example:

1. Many manuscripts made no clear attempt to define the purpose, or, better still, the main conclusion of the study. This ought to be clearly stated in the title, abstract and introduction but often the readers were left to decipher the purpose of a study for themselves by direct reference to the results!

2. Over two-thirds of the manuscripts given to me in 2006 contained few or no references to work done after 2001! This was because the literature study had been conducted before the experimental work was carried out. There was no attempt to link the results of the work to what had been going on in the field subsequently. Part of the problem here is that students were often unaware of the full range of online information resources available to them or had not received adequate training on those they were using.

3. It was quite common for articles to have been rejected because they had been sent to the wrong journal. This is actually a non-trivial issue. There are thousands of journals to choose from, each with its own niche. In today's world of multidisciplinary research, it is important to be aware of the Editor's prejudices about what is or is not interesting research. But part of the problem is also because the authors themselves have not figured out what they have actually done that is interesting and presented it as such!

4. Little thought is given to experimental design until after the experiments have been performed. It is not just the application of statistical tests that is often flawed but also the fundamentals, such as the importance of replication and controls. The fact that major problems with the interpretation of results only become apparent at the late stage of writing the manuscript suggests that there has been no critical review process during the course of the study.

5. And, yes, the English is often poor, but these problems stem from the logical structure of what is being said, which is not really a translation issue!

Based on these observations, we developed a short course on basic research and publishing skills that introduced students to the wide range of bibliographic and database resources available within the Federal university system, taught them how to use these resources to formulate robust experimental hypotheses, and explained some of the idiosyncrasies of the scientific publishing system. The course instantly became very popular attracting students from many departments at the University as well as from other institutions in Brazil. Furthermore, over the years, it has grown from being a simple 15-h lecture series to a 45-60-h juggernaut centred on a rich variety of practical exercises and group work. Its scope, too, has broadened and contains modules designed to support the students at key points during the research/publishing process, i.e., covering the use of molecular sequence or clinical molecular databases or other types of bioinformatic tools such as BLAST, Ensemble, UCSC Genome Browser, “English as an academic language', robust hypothesis formation, experimental design and statistics, how scientific journals work together with key trends in the scientific publishing industry. The course resources are all available online and there are plans to move them to a Moodle (2) environment at some point so that students can take the modules when they make most sense. But that is another story.

In 2007, I began to look more closely at the student's project documents, which were usually in Portuguese, and made the interesting discovery that many of the problems that emerged when the student started to write a paper were apparent even at this stage. The most fundamental of these problems relates to the framing of the basic research question.

Why is this problem so important? Why can the student not go straight to the lab and do some experiments and try to figure out the relevance of the results later? Well, when an editor first reads your paper, he/she is looking for a clear story that is both relevant and topical. In other words, the paper has to tell us something new about something we already have reason to believe is of broad importance - and this message needs to be very visible in the title, abstract and introduction. So, being able to state a clear and compelling reason why the study was carried out is paramount.

This statement is often missing from the project proposal. It is as though the student had been given a specific problem ex cathedra, and then did some background reading to provide a broad context for a study without providing the detailed reasoning as to why this specific question, above all others, was chosen. In other words, the hypothesis is weak, the statistical design is weak and even if P is less than 0.05, the interpretation of what this means is likely to be wrong (3).

Based on these observations, I have some comments for project supervisors and department heads:

1. Research begins, continues, and ends in the (virtual) library. Every attempt should be made to encourage and reward critical reading of current literature. Estimates vary, but most biomedical and clinical researchers should read one new research paper per working day.

2. Writing a research paper begins before the experimental work is begun, and is an integral component of the research process. It is not an optional extra to be contemplated when the experiments have been completed.

3. Provide support for improving reading and writing in English (and Portuguese). Remember, well-structured Portuguese is much easier to translate.

4. Make Professors' academic records (publications, recent post-doctoral periods of retraining, student success rates, etc.) more visible so that prospective students can choose their supervisors effectively.

5. Improve project design, planning and management of Doctoral and Master's courses. Students make the transition from course work to experimental work with little time for thinking or planning. Could the design of the experimental project be integrated into the course workflow somehow? If publication is one of the main outputs from the process, then perhaps course work should be redesigned to reflect this.

6. All projects should be reviewed at regular intervals. Experimental work should not be planned as a single large project, but should be broken down into individual modules so that progress can be monitored in a more continuous way.

7. The creation of a thesis seems counterproductive. The ones I have seen amount to long, uncritical literature reviews plus details of a few experiments. Are they necessary?

8. Courses on scientific method and publishing should be included as a basic component of the Master's and Doctoral degrees. There should also be a course introducing students to online information resources and tools.

9. Basic training on time management, managing meetings, etc. should be given to all Professors. Encourage the development of internet-based calendar programs. There is a saying, ‘If you want something done, ask a busy person'. Successful researchers manage their time effectively.

10. More effort should be put into establishing regular post-doc training periods and networking with researchers overseas.

As a closing remark, I might add that these problems are not specific to developing countries like Brazil. There is a global presupposition that because the younger generation of research scientists have grown up with the internet, they know how to use it. Not so! There is a growing realisation that research quality is a problem in developed countries and that students there also need support in developing field-independent research skills such as the ones discussed here (4,5). This goes on to have implications for the ways in which governments support and fund research. Thus, here are some further observations based on the trends in management and governance of biomedical research and teaching in the US and Europe.

1. Reassess the role of the library. As the availability of online information resources increases, academic librarians must reassess the role they are playing to support research and teaching. The physical need to catalogue and store journal and book collections is rapidly being reduced, and will be replaced by the need to reorganise information resources in a virtual space, and to play an active role in promoting and helping to develop information literacy.

2. Invest in information technology. In a bureaucratic culture, much time is spent on mundane administrative tasks. In a business environment, many of these tasks would be automated, or managed via centralised web services. For example, course registrations, meeting arrangements, booking teaching resources could all be done through Microsoft Office. Multiple choice tests and other teaching resources can be managed using educational software packages such as Blackboard or Moodle. All lecture rooms should have access to high-speed internet, some provision should be made for computer teaching labs and more places for students to work online should be provided.

3. Teach the teachers. The teaching environment today differs significantly from that of 5 or 10 years ago. Students' expectations have changed, the delivery of information has evolved from print to online and multi-media technological tools are available to create new types of teaching resources. The teachers need to be trained to use these new resources effectively. It is not safe to assume that students are computer literate - many are not.

4. Support the development of English. In order to maximise economic and social benefit from its investment in biomedical research, Brazil must compete at an international level. This means that the majority of its research should be published in journals of international standing, in English. More importantly, it means that Brazilian Master's and Doctoral students should be able to read in English from the beginning of their courses so that they may read and assess critically the latest research trends and problems when formulating their research projects. Perhaps more students should go overseas for their training. English is not an optional extra in science - it is a necessity.

5. Make scientific literacy a core competency. Reading and writing scientific English are distinct skills. Although the working vocabulary of scientific English may be smaller than for everyday language, being able to understand and think critically about scientific statements requires sound logical comprehension skills. Furthermore, scientific articles have a distinct style and structure, which may differ slightly from journal to journal. Many students do not have these skills or this knowledge, which needs therefore to be taught. Ideally, such a course would be modular and presented to students at key landmark stages, e.g., when preparing the initial project proposal, before making a preliminary presentation of results and so forth.

6. Provide training in the use of bioinformatic tools such as PubMed, Scopus and BLAST. Most students currently access their preferred journals via the CAPES Periodicals Portal, Scielo or via Google. As a consequence, they are familiar with only a fraction of the literature in their area. The CAPES Periodicals scheme enables all Brazilian students taking accredited courses to access a comprehensive up-to-date library of the very best international journals directly from computers linked to the University network. In addition, the practise of biomedical research increasingly requires students to access other types of online databases containing molecular, taxonomic or clinical information - again, most of these are open access and are available to all students. Making students aware of these tools and teaching them how to use them effectively should also be included in the teaching of scientific literacy.

It is worth repeating that these problems are not specific to Brazil, or even to developing countries. In a recent editorial in The Scientist, the editor identifies a number of cases of “bibliographic negligence', in which authors fail to cite important relevant articles, either deliberately or because they did not know that they existed (6). Furthermore, a recent report by the CIBER group at University College London (7) identifies some bad “Google generation' habits in all age groups. The teachers need to be taught the basics too!

Acknowledgments

Special thanks also for support and guidance from José Eulalio Cabral Filho, Belmira Andrade da Costa, Carlos Perez da Costa, Eduardo Oswaldo Cruz, Waldemar Ladosky, Roberto Lent, Leonor Maia, Marcos Morais, and Valdir Pessoa.

Address for correspondence: D. Bousfield, Ganesha Associates, Cambridge, UK. E-mail: david@ganesha-associates.com

Research supported by CNPq and UFPE. Received March 30, 2009. Accepted May 26, 2009.

  • 1
    Qualis. http://www.capes.gov.br/avaliacao/qualis
  • 2
    Moodle. http://www.moodlebrasil.net
  • 3. Ioannides JPA. Why most published research findings are false. http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371%2Fjournal.pmed.0020124&ct=1 PLoS Med 2005; 2: e124.
  • 4. Birkhead T. Times higher education. http://www.timeshighereducation.co.uk/story.asp?storyCode=400029§ioncode=26
  • 5. Haire L. Enhancing scientific literacy. Guardian newspaper. http://education.guardian.co.uk/conferences/story/0,,1967721,00.html
  • 6. Gallagher R. Scientists are guilty of bibliographic negligence. The Scientist 2009; 23: 13.
  • 7
    Information behaviour of the researcher of the future, CIBER Report. http://www.bl.uk/news/2008/pressrelease20080116.html
  • Correspondence and Footnotes

  • Publication Dates

    • Publication in this collection
      29 June 2009
    • Date of issue
      July 2009

    History

    • Received
      30 Mar 2009
    • Accepted
      26 May 2009
    Associação Brasileira de Divulgação Científica Av. Bandeirantes, 3900, 14049-900 Ribeirão Preto SP Brazil, Tel. / Fax: +55 16 3315-9120 - Ribeirão Preto - SP - Brazil
    E-mail: bjournal@terra.com.br