Building criteria to evaluate qualitative research papers: a tool for peer reviewers

Costa, António Pedro; Minayo, Maria Cecília de Souza

doi:10.1590/S1980-220X2018041403448

Acessibilidade / Reportar erro

Brasil

Revista da Escola de Enfermagem da USP

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

ORIGINAL ARTICLE • Rev. esc. enferm. USP 53 • 2019 • https://doi.org/10.1590/S1980-220X2018041403448 copy

Building criteria to evaluate qualitative research papers: a tool for peer reviewers

Elaborando critérios para a avaliação de artigos qualitativos: uma ferramenta para revisores

Creando criterios para evaluar trabajos de investigación cualitativa: una herramienta para revisores por pares

Authorship SCIMAGO INSTITUTIONS RANKINGS

ABSTRACT

Objective:

To present some criteria, many of which were created in the form of a checklist of items to be considered throughout the review process. Furthermore, we present a study whose main objective was to build (and validate) a review instrument by specialists that is clear, comprehensive, concise and consistent with the highest standards of excellence.

Method:

We use a survey to collect data (123 answers). To analyze the open answers we use the technique of content analysis through the Qualitative Data Analysis Software.

Results:

Besides the creation of new criteria with a focus on methodological dimensions based on 12 items, the involvement of the Ibero-American Congress on Qualitative Research scientific committee in the validation of this instrument.

Conclusion:

This option, in line with the others described, will allow to implement improvements to the editions of the events using the Qualitative Research Evaluation Tool in 2018.

DESCRIPTORS:
Qualitative Research; Checklist; Peer Review; Reference Standards; Academic Performance

	Guiding Questions	General Guidelines
Research Questions and Objectives	1- Is there an inclusion of questions related to the themes of the Congress, namely the methodology of qualitative research?	What are the themes of the congress? Where does your article fit right? Is your article transversal to more than one theme? Which? Is there in the title, abstract or in the introduction a clear indication of the methodological problems, techniques and tools used in qualitative research?
	2 - Are the research questions clearly stated?	Are research questions highlighted and clear? Is there a hierarchical system of research questions and / or sub-questions?
	3 - Is the theoretical basis updated and articulated with the theme or objectives of the study?	How many international authors are cited as the basis of work? Are “classic” publications cited in articulate fashion with other publications (e.g., last 5 years)?
Methodology	4 - Is the methodological option coherent with the problem and the research questions?	Are all the research questions listed included in the methodological design? Is it easy to see its coherence? Is there a table or figure that explains their relation?
	5 - Is the data collection clearly described (i.e., information on availability or origin of data)?	What data served as the basis for building responses to research questions? Are there justifications for the ethical processes (authorizations and informed consent of the subjects involved)?
	6 - Are the methods and techniques of data analysis adequate for the study?	What methods are used? Are the analysis techniques presented? Are they clearly described? Are elements of an analysis system presented, such as dimensions, categories and subcategories? Are they consistent with research issues?
	7 - Is the coding process explained (i.e., number of researchers involved, dimensions, categories of analysis)?	What is the process and number of researchers involved in coding? Is how the coding was validated explained? Is there a concern to show how rigorous and systematic this process was?
Results and Conclusions	8 - Do the quantity and quality of the presented data have the potential for a relevant qualitative analysis?	Do the authors indicate that the quantity and quality of the data are consistent and sufficient to answer the research questions? Is there a diversity of sources and data features?
	9 - Is the discussion of the results based on the theoretical basis and the research questions presented?	Is the power of argumentation evident in the discussion of results? Are they articulated with the theoretical rationale and the objectives and research questions of the work?
	10 - In the conclusions are there summaries of the arguments supporting the results?	Is there a summary of the results in the conclusions? Are these conclusions related to research questions? Is there a triangulation of conclusions with the literature? Are the limitations of these findings presented?
	11 - Do the findings offer any contribution to the qualitative methodologies?	What is the relevance of qualitative research in the conclusions? Is it possible to identify any conclusions directly or indirectly linked to the qualitative methodology?
	12 - Do you agree with the contributions (implications, impact) for the development of the qualitative research mentioned by the authors?	Is it possible to identify the implications of the process, results and conclusions of the work? Is the impact or possible impacts of work on different spheres of influence (training, research, society, etc.) pointed out?
		Note: read the text inserted by the authors in the field “What contribution does the article bring to Qualitative Research and to CIAIQ?” Available in EasyChair (next to the summary).

Categories/Subcategories	References	Definition
Suggestions
Evaluation	13	Process of evaluating papers regardless of the criteria presented.
Criteria	24	Proposals for changes, adjustments and improvements to the criteria.
Others	7	Crosswise observations on the evaluation process and its criteria.
Criteria
Suitable	82	The criteria are objective and relevant according to the scope / theme of the conference.
Clear	28	The criteria are clear in their proposal and presentation.
Strict	21	The exacting criteria allow for careful evaluation.
Facilitators	20	The criteria simplify and guide evaluation.
Exhaustive	4	The criteria are too dense and complex.

Universidade de São Paulo, Escola de Enfermagem Av. Dr. Enéas de Carvalho Aguiar, 419 , 05403-000 São Paulo - SP/ Brasil, Tel./Fax: (55 11) 3061-7553, - São Paulo - SP - Brazil
E-mail: reeusp@usp.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Corresponding author: António Pedro Costa Ludomedia/webQDA e CIDTFF - Centro de Investigação de Didática e Tecnologia na Formação de Formadores, Departamento de Educação e de Psicologia da Universidade de Aveiro - Campus Universitário de Santiago 3810-193 Aveiro - Portugal pcosta@ludomedia.pt / apcosta@ua.pt