Doctorate in cardiology at FM-USP heart institute from 1994 through 2004: defense and publication

Abstracts

FUNDAMENTO: Um estudo avaliou a relação entre defesas de tese e publicações no âmbito de toda a Faculdade de Medicina da Universidade de São Paulo (FMUSP). Questiona-se a existência de diferenças entre diferentes áreas do conhecimento e no domínio do tempo. OBJETIVO: Caracterizar as publicações relacionadas às teses defendidas na pós-graduação do Instituto do Coração (InCor) da FMUSP. MÉTODOS: Realizou-se um levantamento retrospectivo junto à pós-graduação de cardiologia da FMUSP, no período de 1994 a 2004. Inicialmente foram coletados dados de alunos que defenderam teses nesse período, de seus orientadores e das próprias teses. A seguir, cruzando informações da Medline® e Web of Science®, localizaram-se publicações desses autores e dados referentes às respectivas publicações. RESULTADOS: Nesse período, foram defendidas 268 teses, que resultaram em 195 publicações, no período de até dez anos após a defesa. As publicações ocorreram com mediana de um ano e nove meses após a sua defesa, com fator impacto mediano de 2,1 e uma mediana de 4 citações por trabalho. Não houve correlação com significância estatística em nenhum dos dados estudados. CONCLUSÃO: Uma porcentagem importante das teses é publicada. A publicação tem ocorrido de maneira cada vez mais precoce após a defesa. Esse fato pode estar relacionado ao sucesso das políticas das comissões de pós-graduação e à importância com que esse tema já é debatido atualmente, contribuindo para a melhoria da qualidade da pós-graduação.

Educação de pós-graduação em medicina; fator de impacto; avaliação pelos pares para publicação

ORIGINAL ARTICLE

Doctorate in cardiology at FM-USP heart institute from 1994 through 2004: defense and publication

Julio Flávio Meirelles Marchini^II; Bruno Caramelli^I

^IInterdisciplinary Medicine in Cardiology Unit, Heart Institute of the University of São Paulo Medical School, São Paulo, SP - Brazil

^IIUniversity of São Paulo Medical School of Ribeirão Preto, Ribeirão Preto, SP - Brazil

Mailing Address

SUMMARY

BACKGROUND: A study evaluated the relationship between the defense of dissertations and their publication within the realm of the entire Medical School of the University of São Paulo (FM-USP). The existence of differences among different areas of knowledge and the time between defense of dissertations and their publication is questioned.

OBJECTIVE: To characterize publications related to dissertations defended in graduate courses of the Heart Institute (InCor) of the University of São Paulo Medical School (FM-USP).

METHODS: A retrospective survey was carried out featuring graduate work at FM/USP for the 1994 – 2004 period. Initially, data were collected on students who defended dissertations during this period, on their advisors, and on the dissertations defended in the course of the period. Then, by crossing these data with data from Medline® and Web of Science®, publications by these authors and data were located that referred to the respective publications.

RESULTS: During that period, 268 dissertations were defended, resulting in 195 publications within a period of up to 10 years after their defense. The median time for publication after defense was one year and nine months, with a median impact factor of 2.1, and a median of 4 citations per paper. There was no statistically significant correlation among any of the data studied.

CONCLUSION: A significant percentage of the dissertations were published, and in the group studied, publication took place within increasingly short times after their defense. This fact may be related to the success of the graduate commissions' policies and to the importance currently lent to debates on this theme, all of which contribute to improvement in the quality of graduate studies.

Key words: Education, medical graduate; impact factor; peer review, research.

Introduction

The graduate program training process includes the acquisition of knowledge, teaching, scientific methodology, and the production of innovative knowledge. Students must be capable of carrying out data collection, tabulation, and the presentation of findings to the target community. A survey recently carried out at the Medical School of the University of São Paulo (FM-USP) showed that publication rates¹ were low. The defense of a dissertation is not the end of a process, but rather its beginning. The process generates knowledge to be publicized and also serves as the foundation for new works. An editorial of this journal commemorating 30 years of graduate studies in cardiology at the Heart Institute (InCor) highlighted the history of this area and its contribution to the production and exportation of knowledge².

However, these are general data and may not be true for specific areas. In this paper we sought to continue this analysis within the field of cardiology, featuring it as a highlight within this same medical school. This observation occurred during a period following a previous period that was covered by a different study, which made it possible to note if there was a tendency toward change in the publication rate.

Methods

Analysis was carried out of dissertations in cardiology at FM-USP presented in the course of the eleven-year period from 1994 through 2004. This analysis took into consideration only enrolled students who defended their dissertations during the interval under study. Data were collected regarding the graduate students themselves, their advisors, and the themes of the dissertations they defended. Data regarding the students included date of birth, graduating year, graduate program entrance date, and dissertation defense date. Calculations focused on the interval between the date of graduation from medical school and enrollment in the graduate program, length of time in the program, and ages upon enrollment and defense of their dissertations.

Information regarding the registered advisors consisted of the number of dissertations each one had advised in the course of the period studied.

Four researchers (two from the clinical area and two from the basic area) analyzed the titles of the dissertations without any knowledge regarding the names of the authors or their advisors, and independently classified each dissertation according to the area of interest: basic, clinical or epidemiological. Works not carried out on humans, based on tissue samples or on models, were classified as basic area. Non-experimental studies, determining prevalence, incidence, and prognostics, were classified as epidemiological, and experimental studies with human beings were classified as clinical studies.

The second source of data used in this study was the Pubmed® archive, available on the Internet from the National Library of Medicine of the National Institute of Health (ncbi.nih.nlm.gov), and Web of Science®, accessible through CAPES. By entering the title of the dissertation, the name of the candidate and of the advisor, it was possible to identify whether the dissertation had been published in any journal registered in these sources up to the conclusion date of this paper – February 2008. If the paper was identified, the following data was extracted: its publication date, the journal in which it was published, its impact factor in the year of its publication, and the number of times the article was cited in other papers up to the time of its analysis. Also calculated was the number of citations per year from the publication date until February 2008. The impact factor used came from Thomson Scientific Journal Citation Reports® (JCR) made available by CAPES (Coordination for the Improvement of Higher Education Personnel) since 2001. For 14 publications prior to 2001, the impact factor was requested from the eight journals involved, or obtained from two publications^3,4. Impact factors for the years of 2007 and 2008 have not yet been defined – a situation applying to 11 publications – in which cases the 2006 impact factor was used. A zero impact factor was attributed to journals not included in the JCR.

Statistical analysis was carried out using the Stata 9.0® program. Kappa analysis was used to evaluate agreement in the classification of dissertations into types. Logistic regression was used to test the correlation between publication of the dissertation and the variables surveyed, and multivariate linear regression was used to correlate the impact factor obtained by the publication with the variables under study. Either non-paired single-tailed t-tests or chi-square tests were used for comparison of groups when appropriate.

Results

Two hundred and sixty-eight doctoral dissertations were presented within the eleven-year period of this study. The average age of students upon entrance to the graduate program was 37, with a standard deviation of 7.0 years. Students averaged 41 years of age with a standard deviation of 6.9 years upon defense of their dissertation. Ninety-seven percent of the students were physicians. The average duration of the graduate work, i.e., between enrollment and presentation of the dissertation, was 4 years, with a standard deviation of 1.6 years.

The average length of time between graduation from medical school and entrance in the graduate program was 13 years with a standard deviation of 6.7 years. That same interval for those who graduated prior to 1980 was 23 years with a standard deviation of 6.15 years; 13 years with a standard deviation of 6.0 years for those who graduated between 1980 and 1989; and 6 years with a standard deviation of 1.84 years for those who graduated after 1989.

We noted that 47 dissertations were defended in the first half of the graduate program period, and 221 in the second half.

Sixty-eight different advisors took part in the program within the interval of this study, each advisor with a median of 6 students, ranging from 1 to 15 students advised.

The dissertations of 50% of the students were published within two years and eight months of their defense (Figure 1). In all, 195 (73%) were published with a median of one year and nine months after their defense. The 25th and 75th percentiles were, respectively, 1.1 year and 3.1 years. The earliest publication occurred six years and four months prior to defense of the dissertation, and the latest, ten and a half years after its defense. One hundred and seven (55%) were published in journals that have impact factor. In this group, the median time to publication was 1 year and 9 months after defense and the 25th and 75th percentiles were respectively 0.9 years and 2.8 years. The impact factor of these publications ranged from 0.4 to 51.3, with a median of 2.1 and 25th and 75th percentiles amounting respectively to 1.4 and 3.6. Citations of these publications per year ranged from zero to 353 with a median of 4 and respective 25th and 75th percentiles of 1 and 11 times. The publications obtained a median of 1.0 citation per year, ranging from zero to 49.3 citations per year with respective 25th and 75th percentiles of 0.3 and 1.9.

As to publications not evaluated by JCR, publication occurred with a median of two years and one month after the defense, and the 25th and 75th percentiles were respectively 1.2 and 3.7. Table 1 shows the number of dissertations defended and published in the course of the years studied.

The chi-square test was carried out to compare the percentage of publications for each year with the average for the whole 11-year period studied. The comparison showed that there was no difference in distribution (p=0.99).

It was also observed that 16% of the total publications occurred prior to defense of the dissertation, while 17% occurred four years after defense.

Total dissertations published were divided at the median, which occurred in July 2002. The first group comprised eight and a half years with 98 dissertations published, and the second group comprised the remaining two and a half years with 97 dissertations published. The median length of time from defense until publication for the first 98 dissertations was two and a half years, with respective 25th and 75th percentiles of 0.9 and 4.2. The median for the remainder (97 dissertations) was one year and seven months with 25th and 75th percentiles of 1.1 and 2.4, respectively. The values found for the two groups were significantly different (p=0.0118) (Figure 2).

Kappa analysis of the two types of dissertations showed agreement to be 0.11, i.e., low agreement. Table 2 shows agreement by type of dissertation. It was noted that kappa agreement proved to be only average (0.2 and 0.4) for dissertations classified as basic – those that had shown the highest correlation.

In 111 dissertations (41%), no majority opinion was established among the classifiers as to the type of dissertation. In regard to the remainder, 67 (25%) were classified as clinical studies, 61 (23%) as epidemiological studies, and 29 (11%) as basic studies. If the epidemiological and clinical studies are seen as one sole classification, kappa agreement rises to 0.23. In this case, 53 dissertations (19%) remained unclassified, 194 (72%) were classified as clinical studies, and the remaining 21 (7%) as basic studies.

Correlation was sought between the factors surveyed and publication of the dissertations. The variables considered in this analysis were: year of graduation; interval between graduation and enrollment in the graduate program; length of time in the graduate program; age upon defense; and number of dissertations advised concomitantly by the respective advisors. Analysis by logistic regression showed no correlation with any variable surveyed. Multivariate linear regression was then used to analyze the relationship between the impact factor of the journal in which the dissertation was published and the factors surveyed. The factors taken into account were the same. No correlation was found between the impact factor and any of the variables. Figure 3 shows the time until publication of the dissertation in relation to the impact factor.

Discussion

A survey was carried out to show performance and evolution of the graduate program in cardiology at InCor/FM-USP. Factors were sought that might correlate the greater probability of publication and/or publication in journals featuring higher impact factors.

A period of 13 years between graduation from medical school and enrollment in a graduate program is quite long. It must also be pointed out that such interval, for those who graduated prior to 1980, was 26 years. This fact can be explained, in part, by the medical residence in cardiology which was equivalent to only a small fraction of the time between graduation and enrollment in the graduate program for those who graduated prior to 1980, but corresponds to 50% of that time for those who graduated after 1989. The two-year medical residence in clinical medicine was only initiated in 2000 and 2001 and therefore does not include the graduate students with four years of residence included in this sample. Another possibility to complement this value would be the time spent on preparation of the graduate project, including beginning it prior to formal enrollment in the program.

Publication of 195 dissertations was found within the 11 years covered by this study. Another study, published in 2005, pointed to the production of 294 publications of dissertations within the 30 years of graduate studies at the Heart Institute². Excluding the 42 publications that occurred after 2005, we can see that 75% of all the publications attributed to this institution occurred within the 11 years studied.

These results can be compared to the study made by Younes et al¹ that analyzed the first publication of PhDs who earned their doctorate at FM-USP in any graduate program area during the period of 1990-2000. They found that in five years, 50% of the graduates had not yet published any paper. In this study, the value found for publication of the dissertation by 50% of the students was two years and eight months. For the previous study, we have no analysis exclusively for students of cardiology. However, upon comparison of two subgroups of this current study (divided at their median), we noted a significant drop in the time until publication of the dissertation (Figure 2).

Classification of the dissertations by type – clinical, epidemiological, and basic – showed agreement among the four researchers to be unsatisfactory. Even the highest correlation obtained was no higher than kappa 0.23, which corresponds to average agreement. It is possible that each dissertation involves aspects that occur in all of the various types of classification. For example, a clinical study may use a molecular biology method and also feature epidemiological analysis of the study groups. There is no way of dividing graduate studies into such types.

Logistic regression found no correlation between publication of the study and any of the variables studied, nor did linear regression reveal any correlation between the impact factor and any of the variables studied.

On December 3, 1985, the Federal Education Commission defined graduate study, which was being structured in Brazil at the time⁵. The object was to position universities as generators of scientific and technological knowledge. The definition also points out that contrary to undergraduate courses, which should be open to the greatest possible number of students, graduate courses should be limited to the most capable students. Thus, graduate programs must uphold a high level of ongoing improvement, which involves knowing their characteristics, as well as proposing and implementing changes. Thereafter, a new analysis should be carried out, comparing it with the first and making it possible to draw conclusions regarding the results. Instruments and methods must be chosen for this evaluation. Self-evaluation of an institution may involve conflicts of interest since the parties of interest are the evaluators themselves. Scientific knowledge generated, upon its publication in a journal, is seen by groups outside the institution, and the graduate program itself is indirectly evaluated by its product. Evaluation of graduate programs by CAPES takes this fact into consideration⁶. At the same time, the criteria proposed by CAPES make it clear that this is not the only factor considered in the course of an evaluation.

A second level of analysis would be to evaluate the impact factor of the publications obtained. This second level reflects the view that an article is of scant worth if it is not used and cited by other researchers. The impact factor is calculated by dividing the number of citations of all the works published in a certain journal for the two previous years by all the works published during the same period by the same journal⁷. There are some important works, such as those on local diseases, their prevalence, and analyses of national costs, that are of little interest to the main hubs of world research – the United States and Europe – and that may not be published in journals with high impact factors⁸. Studies carried out by national groups may find it more difficult to have their works published in the more renowned journals. This fact has a negative influence on the impact factor indicator, and consequently on the use of the study, without necessarily reflecting on the quality of that study. The limitations of this indicator must be understood if it is to be properly interpreted.

The CAPES evaluation itself, by incorporating analysis of publications and a limited time for publication following the defense of dissertations, may constitute the main reason for the tendencies noted. In addition, the importance of publication for the graduate program is now being acknowledged by the institutions that grant doctorate degrees. Some commissions are now demanding submission of the work for publication as a requirement for conclusion of the doctorate. Candidates are allowed the possibility of repeating the study in the near future for comparison purposes.

Limitations of the study

The value of the impact factor used for years 2007 and 2008 (representing 5% of the dissertations covered in this study) was the same as that for 2006. Among the criteria for inclusion was the requirement that defense of the dissertation had been completed. Thus, dropouts were not taken into consideration but, in view of their very small number, we feel that the results were not influenced.

Acknowledgement

Our thanks to Dr. Julio Sérgio Marchini, for his suggestions and careful revision of the article.

Potential Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Sources of Funding

There were no external funding sources for this study.

Study Association

This study is not associated with any graduation program.

References

• 1. Younes RN, Deheinzelin D, Birolini D. Graduate education at the faculty of medicine of the University of Sao Paulo: quo vadis? Clinics. 2005; 60 (1): 6-8.
• 2. Hueb W, Mady C, Ramires JAF. Trinta anos de pós-graduação em cardiologia. Arq Bras Cardiol. 2005; 85 (6): 385-7.
• 3. SoRelle R. Circulation impact factor is highest ever. Circulation. 2001; 104 (13): 1450.
• 4. Marban E. Circulation research impact factor sets new record. Circ Res. 2001; 89 (2): 101.
• 5. Parecer CFE nş 977/65, aprovado em 3 dez. 1965. Rev Bras Educ. 2005; 30: 162-73.
• 6. Horta JSB, Moraes MCM. O sistema CAPES de avaliação da pós-graduação: da área de educação à grande área de ciências humanas. Rev Bras Educ. 2005; 30: 95-116.
• 7. Tobin MJ. Impact factor and the journal. Am J Respir Crit Care Med. 2003; 168: 621-2.
• 8. Kerr-Pontes LRS, Pontes RJS, Bosi MLM, Rigotto RM, Silva RMd, Bezerra Filho JG, et al. Uma reflexão sobre o processo de avaliação das pós-graduações brasileiras com ênfase na área de saúde coletiva. Physis: Rev Saúde Coletiva. 2005; 15: 83-94.

Publication Dates

• Publication in this collection
  05 Jan 2009
• Date of issue
  Nov 2008

History

• Reviewed
  07 Apr 2008
• Received
  11 Nov 2007
• Accepted
  05 May 2008