October 2004 Version
This document describes the evaluation criteria, policies and procedures adopted by the SciELO Project (http://www.scielo.org) for inclusion of scientific journals in the SciELO Brazil (http://www.scielo.br) electronic collection.
The criteria as well as the policy and procedures for application were discussed at the Seminar on Evaluation and Selection Criteria for Scientific Journals held in April 1999 at FAPESP - The State of Sao Paulo Research Foundation (http://www.fapesp.br), Sao Paulo, and later approved by FAPESP and BIREME - The Latin American and Caribbean Center on Health Sciences Information (http://www.bireme.br). They are referred hereinafter as SciELO Brazil Criteria since they are closely related to the Brazilian scientific publications both concerning their definition and application.
As the SciELO Project expands to other Latin American countries, it is expected that such criteria will be established as common for inclusion of scientific journals in national and/or regional SciELO collections.
2 SciELO Brazil Criteria Objectives
SciELO Brazil Criteria are defined in the context of the SciELO Project, which is coordinated by FAPESP and BIREME, and also supported by CNPq - The National Council for Scientific and Technological Development (http://www.cnpq.br). The general goal of the SciELO Project is to contribute to the development of national scientific research by improving and expanding its means of dissemination, publication, and evaluation through the intensive use of electronic publishing.
In the short term, the objective of the SciELO Project is to radically increase national and international visibility, accessibility and credibility of Latin American and Caribbean scientific publications by publishing on the internet national and regional collections of scientific journals. In the long term, the project expects to increase the impact of national scientific literature.
In order to meet the SciELO Project objectives, it is essential to promote the improvement of every aspect of scientific communication in the country seeking to identify, stimulate, and develop a core collection of scientific journals with the same quality standards of leading scientific journals. Therefore, evaluation criteria for journals as well as policies and procedures for their application are the main criteria for the SciELO Project.
The specific goal of SciELO Brazil Criteria is to contribute to the creation and development of the SciELO Brazil core collection of scientific journals by:
- Providing data for the evaluation of national scientific journals in order to determine if they can be included in the SciELO Brazil collection;
- Monitoring the performance of scientific journals included in the SciELO Brazil collection in order to judge if they should remain in the collection;
- Producing performance indicators for the SciELO Brazil collection as a whole and for each of its individual journals.
SciELO Brazil Criteria contents and application will follow SciELO Project dynamics and, in particular, the process of creating and developing the SciELO Brazil core collection.
3 SciELO Brazil Advisory Committee
Based on the application of SciELO Brazil Criteria, the Advisory Committee shall propose:
- The inclusion of new scientific journal titles in the SciELO collection;
- The exclusion of scientific journal titles from the SciELO collection;
- Changes in the evaluation criteria for inclusion and continuity of scientific journals in the SciELO Brazil collection;
- Performance indicators to be published regularly for journals belonging to the SciELO Brazil collection;
- The definition and enhancement of the Committee's operation, enabling the efficient fulfillment of the above objectives.
The Committee will be formed on a permanent basis by representatives of the national scientific community directly related with its communication as follows:
- The SciELO Project Operational Coordinator who will preside over the Committee as Chairman;
- A representative from ABEC - Associação Brasileira de Editores Científicos (Brazilian Association of Scientific Editors);
- A representative from the FAPESP Program for Scientific Publications;
- A representative from CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico (National Council for Scientific and Technological Development;
- Four editors of scientific publications from each of the following major areas: Agriculture, Biology, Exact Sciences and Human Sciences to be elected every 2 years from science editors participating in the SciELO Brazil collection as from the second year of life of the Advisory Committee - during the first year scientific editors will be indicated by SciELO. Scientific editors can be re-elected, with the replacement of one editor at each meeting;
- A representative from CAPES - Coodernação de Aperfeiçoamento de Pessoal de Nível Superior (Coordination for Higher Level Education Improvement).
The Committee may also invite outside consultants and specialists to take part in meetings and activities as required.
4 SciELO Brazil Collection Scope
The SciELO Brazil collection includes mainly scientific journals featuring articles resulting from original scientific research and other original works, which contribute significantly to the area of expertise of the journal. Review and approval of articles published is carried out by peers.
The collection of each individual title at SciELO Brazil starts with its approval for inclusion in the collection with the addition of previous issues backdated to 1997, or with the first issue of a title for journals published after 1997. The articles and other works may be written in English, Portuguese or Spanish.
Journals published by the SciELO collection are not necessarily printed on paper.
5 SciELO Brazil Criteria for journals admission
5.1 Criteria for automatic inclusion in the period 1999-2001
The SciELO Brazil criteria for inclusion of new journal titles take into consideration the previous evaluations that these journals have taken part in including their rating in selected international indexes and the evaluations by FAPESP and by CNPq/FINEP of 1997.
Up until 2001, a journal title would automatically qualify for inclusion in the collection if it were indexed in one of the selected indexes described in item 5.1.1 or if it had obtained a high score in the evaluation processes by FAPESP and CNPq/FINEP.
5.1.1 Journals included in international indexes
Brazilian journals which appear in one of the following international indexes are automatically qualified for initial inclusion in the SciELO Brazil collection up until the year of 2001:
- MEDLINE/Index Medicus
- PsycInfo (APA)
The Advisory Committee had sole responsibility for any changes in the international indexes list.
Journals will continue to be featured in this group of the SciELO Brazil collection subject to performance indicators as stated in item 6.
5.1.2 Pre-selected journals by FAPESP Criteria
The Brazilian journals evaluated by FAPESP in 1997 classified as of "priority" content and rated "very good" or "good" are automatically qualified for initial inclusion in the SciELO Brazil collection until 2001.
Journals will remain in this group of the SciELO Brazil collection subject to performance indicators as stated in item 6.
5.1.3 Pre-selected journals by CNPq/FINEP Criteria
The Brazilian journals classified in the Group A by the Program for Scientific Publications of CNPq/FINEP in 1997 are automatically qualified for initial inclusion in the SciELO Brazil collection until 2001.
Journals will remain in this group of the SciELO Brazil collection subject to performance indicators as stated in item 6.
5.2 Evaluation criteria for inclusion in SciELO
When a title is not automatically qualified for inclusion in SciELO, it will be evaluated according to the following factors:
5.2.1 Scientific content
Journals shall publish mainly original articles resulting from scientific and/or significant research in their areas of expertise. Journals can include other types of contributions such as reviews, reports, releases and case studies, which will not be regarded as original articles.
SciELO Advisory Committee may request the opinion of peers so as to confirm prevalence of original contributions.
5.2.2 Peer review
Evaluation and approval of the contributions submitted must be performed by peers. The journal must expressly specify the procedure used by peers for review and approval of contributions. Once a journal is accepted for inclusion in the SciELO collection the review procedure by the peers will be documented. The indication of the main dates of the arbitrage process, including dates of receipt and acceptance is mandatory.
5.2.3 Editorial board
Members of the journals editorial board shall be made public. Members shall be well-known national and international specialists duly identified in the publication.
Journals whose boards are made up mainly of members belonging to the same institution and/or whose articles come mainly from the same institution or geographical region will not be accepted for inclusion in the collection.
Periodicity is an indicator of the flow of scientific production, which depends on the area of expertise of the journal. It is also an indicator related to opportunity and speed of communication.
Defined by main subject areas of expertise, the following table indicates the minimum and desirable periodicity as well as the minimum and desirable number of articles per year a journal is expected to present:
Number of articles per year
|Agriculture and Exact Sciences||
5.2.5 Minimum number of publications
To be considered for evaluation, the journal must have been published at least four times.
The journal must come out punctually according to the stated publication dates.
5.2.7 Article titles, abstracts, and keywords in English
Articles must contain title, abstract and keywords in the original language they were written and in English if this is not the original language.
The journal shall specify the standards followed for presentation and structuring of texts, bibliographic references and descriptors so as to enable evaluation of compliance with the standards indicated.
Although the adoption of a standard is recommended for bibliographic reference - such as ABNT, ISO, Vancouver- other standards will be accepted provided the bibliographic format to be followed by authors is clearly indicated.
5.2.9 Authors' affiliation
The articles shall contain complete information on the affiliation of the authors, including institution of origin, city and country.
The journal shall present a list of citations compatible with journals in the same area of expertise, confirmed by the citations received from articles published in the SciELO Brazil collection.
5.3 Procedures for acceptance for inclusion
Featuring of a journal in the SciELO collection will be made after a favorable report is issued by SciELO Advisory Committee.
Appraisal procedures comprise three aspects: (1) format, i.e. compliance with the standards; (2) endogeny with regard to institutional and geographical concentration of the publishing board and authors; and (3) content, to assure scientific quality.
The appraisal of format includes an analysis of a set of characteristics, based on studies and international standards on publication of scientific journals. There are compulsory items but as a whole a minimum of 80% of characteristics is accepted.
The appraisal of endogeny is based on the affiliation declared by the editorial board, reviewers and authors. Judgment or decision as to institutional or geographical concentration of these elements is regarded as a negative result for inclusion of journal into the collection.
The journals that satisfactorily meet the above-mentioned requirements will be evaluated as to their content based on the opinion of a minimum of two specialists in the field of expertise of the journal, with regard to the following characteristics:
- representativeness of the editorial board, reviewers and authors;
- scientific content of periodical articles;
- review conducted by peers;
- relevance as to development of area.
Every editor representing a field of expertise examines the documentation and opinions, and submits the journal to the Advisory Committee for appreciation and final decision.
SciELO will contribute to the work of the Committee by organizing the information and documentation required. Only the journals that comply with items 5.2.1. to 5.2.10 will be submitted for analysis by the Advisory Committee.
The Committee may make recommendations for improvements and modifications to the journals, which shall be implemented in the period of time specified in the committee's report.
6 Performance evaluation criteria for continuation of journals in the SciELO collection
As a rule, all the indicators adopted in the evaluation process for inclusion of titles in SciELO Brazil are also applied for deciding on its continuation in the collection.
Adoption of the Committee's recommendations for improvement of journal will be considered as indicative for the journal to remain in the collection.
The following are indicators associated with the specific performance of journals in SciELO, and will be used to justify their continuation or not in the collection:
6.1 Punctuality in file delivery
Punctuality is measured by the arrival of the electronic files of a journal at SciELO. File submission shall strictly comply with periodicity of the journal. SciELO shall report any delays in file submission.
Journals that arrive late will be evaluated by the SciELO Advisory Committee, and will be subject to exclusion from the SciELO collection.
6.2 Journal use indicator
Journal use is measured monthly by the number of times it is accessed or viewed online. When the use of a journal is consistently low and/or tends to decrease when compared with other journals featuring the same subjects, the SciELO Advisory Committee will evaluate if it should remain in the collection. The Committee will then suggest a study to find out the reasons for this and will either propose possible solutions, or recommend the journal be excluded from the SciELO collection.
6.3 Impact factor
The impact indicator of a journal title measured by the number of citations received must be assessed together with the titles of journals in the same field of expertise.
An increase in the impact factor or standing on the same average of similar journals will be considered productive performance, which assures the journal will remain in the SciELO collection.
7 Title exclusion process report
Exclusion of a journal from the SciELO collection can only be carried out after a decision report is issued by the SciELO Advisory Committee.
In the case of unfavorable results obtained in the performance appraisal, the journal will be notified by the Committee of the aspects to be improved within the period/time stipulated.
Journal editors may appeal at any time against a decision by the SciELO Advisory Committee both in relation to inclusion and exclusion processes.
Appeals will be examined by the Advisory Committee and inclusion of the journal may be re-examined. The Committee's report will be sent to the journal editor.
9 Re-inclusion process
Journals excluded from the SciELO collection can be re-introduced provided that they meet the inclusion criteria required by SciELO Brazil. The SciELO Advisory Committee will analyze re-inclusion.
Re-inclusion of a journal title will not be made effective immediately after its exclusion from the SciELO collection. In order to be re-evaluated, a journal must demonstrate its compliance with the criteria for at least four consecutive issues or one year.