Accessibility / Report Error

Development and validation of an instrument for evaluating the ludicity of games in health education

Desarrollo y validación de instrumento para evaluar la ludicidad de juegos en salud

Abstract

OBJECTIVE

Developing and validating an instrument to evaluate the playfulness of games in health education contexts.

METHODOLOGY

A methodological, exploratory and descriptive research, developed in two stages: 1. Application of an open questionnaire to 50 graduate students, with content analysis of the answers and calculation of Kappa coefficient for defining items; 2. Procedures for construction of scales, with content validation by judges and analysis of the consensus estimate byContent Validity Index(CVI).

RESULTS

53 items regarding the restless character of the games in the dimensions of playfulness, the formative components of learning and the profiles of the players.

CONCLUSION

Ludicity can be assessed by validated items related to the degree of involvement, immersion and reinvention of the subjects in the game along with the dynamics and playability of the game.

Descriptors:
Health Education; Play and Playthings; Education, Higher; Creativity; Personal Autonomy; Validation Studies

Resumen

OBJETIVO

Elaborar y validar un instrumento para la evaluación de la ludicidad de juegos en contextos educativos de salud.

MÉTODO

Investigación metodológica, exploratoria y descriptiva, desarrollada en dos etapas: 1. aplicación de cuestionario abierto a 50 estudiantes universitarios, con análisis de contenido de las respuestas y cómputo del coeficiente Kappa para cierre de los ítems; 2. procedimientos de construcción de escalas, con validación de contenido mediante análisis de los jueces y estimación de consenso por el Content Validity Index(CVI).

RESULTADOS

Se validaron 53 ítems acerca del carácter irrequieto de los juegos en las dimensiones de la ludicidad, de los componentes formativos del aprendizaje y del perfil de los jugadores.

CONCLUSIÓN

La ludicidad puede ser evaluada por ítems validados relacionados con el grado de involucración, inmersión y reinvención de los sujetos en el partido, al lado de la dinámica y la jugabilidad del juego.

Descriptores:
Educación en Salud; Juegos e Implementos de Juegos; Educación Superior; Creatividad; Autonomía Personal; Estudios de Validación

Resumo

OBJETIVO

Elaborar e validar um instrumento para a avaliação da ludicidade de jogos em contextos educativos da saúde.

MÉTODO

Pesquisa metodológica, exploratória e descritiva, desenvolvida em duas etapas: 1. aplicação de questionário aberto a 50 graduandos, com análise de conteúdo das respostas e cálculo do coeficiente Kappa para fechamento dos itens; 2. procedimentos de construção de escalas, com validação de conteúdo mediante análise dos juízes e estimativa de consenso pelo Content Validity Index(CVI).

RESULTADOS

Foram validados 53 itens acerca do caráter irrequieto dos jogos nas dimensões da ludicidade, dos componentes formativos da aprendizagem e do perfil dos jogadores.

CONCLUSÃO

A ludicidade pode ser avaliada por itens validados relacionados ao grau de envolvimento, de imersão e de reinvenção dos sujeitos na partida, ao lado da dinâmica e da jogabilidade do jogo.

Descritores:
Educação em Saúde; Jogos e Brinquedos; Educação Superior; Criatividade; Autonomia Pessoal; Estudos de Validação

Introduction

Knowledge on the interface of recreational activities with education in health often reports using games as a tool for learning. A literature review on this subject highlights the importance of games for clinical, investigative or educational activities(11 Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.

2 Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, et al. Educational games for health professionals. Cochrane Database Syst Rev. 2013;(1):CD006411.

3 Irons JY, Petocz P, Kenny DT, Chang AB. Singing as an adjunct therapy for children and adults with cystic fibrosis. Cochrane Database Syst Rev. 2014;(6):CD008036.
-44 Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database of Syst Rev. 2015;(2):CD008349.). International studies show different approaches for games in health(11 Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.

2 Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, et al. Educational games for health professionals. Cochrane Database Syst Rev. 2013;(1):CD006411.

3 Irons JY, Petocz P, Kenny DT, Chang AB. Singing as an adjunct therapy for children and adults with cystic fibrosis. Cochrane Database Syst Rev. 2014;(6):CD008036.

4 Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database of Syst Rev. 2015;(2):CD008349.

5 Morris BJ, Croker S, Zimmerman Z, Gill D, Romig C. Gaming science: the 'Gamification" of scientific thinking. Front Psychol [Internet]. 2013 [cited 2014 Feb 10];4:607. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
http://www.ncbi.nlm.nih.gov/pmc/articles...

6 Kuss JD. Internet gaming addiction: current perspectives. Psychol Res Behav Manag [Internet]. 2013 [cited 2014 Feb 10];6:125-37. Available from:Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
http://www.ncbi.nlm.nih.gov/pmc/articles...

7 Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.
-88 Mishra J, Bavelier D, Gazzaley A. How to assess gaming-induced benefits on attention and working memory.. Games Health J 2012;1(3):192-8.): a- ludic activies promoting healthy lifestyles; b- risks of suffocation and similar games; c- damage caused by video game addiction; d- the importance of games for scientific thinking; e- games in the training of health professionals; f-exergames(physical exercises motivated by computer games) for health promotion; among others. Specialized journals have multiplied, developing a corpus of its own knowledge known as Serious Games, which synthesize the understanding of entertaining characteristics into the production ofedutainment(55 Morris BJ, Croker S, Zimmerman Z, Gill D, Romig C. Gaming science: the 'Gamification" of scientific thinking. Front Psychol [Internet]. 2013 [cited 2014 Feb 10];4:607. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
http://www.ncbi.nlm.nih.gov/pmc/articles...

6 Kuss JD. Internet gaming addiction: current perspectives. Psychol Res Behav Manag [Internet]. 2013 [cited 2014 Feb 10];6:125-37. Available from:Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
http://www.ncbi.nlm.nih.gov/pmc/articles...

7 Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.
-88 Mishra J, Bavelier D, Gazzaley A. How to assess gaming-induced benefits on attention and working memory.. Games Health J 2012;1(3):192-8.).

Much has been investigated about the contribution of games towards disease prevention, behavioral changes or educational processes in health. A certain imprecision in results can be noticed in these investigations, explicating limitations or instigating recommendations, thereby impeding generalizations. In an extended analysis of this scenario, it is argued that the hermeneutic irreverence of recreational activities, such as the human spirit, resists to be restricted to utilitarian purposes, whatever they may be(99 Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.). In other words, despite the importance of investing in games for the demands of health, it is argued that this form of dialogue about ludic activity restricted to behavioral purposes inhibits its creative, inventive and autonomous driving force(99 Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.

10 Huizinga J. Homo Ludens: o jogo como elemento da cultura. São Paulo: Perspectiva; 2008.
-1111 Callois R. Os homens e os jogos: a máscara e a vertigem. Lisboa: Cotovia; 1990.).

In the dialogue of games and education, it is necessary to maintain the restless spirit of play if one wants to release the creative impulse that shapes human autonomy. Ludic activity is a cultural trait inherent to subjectivity; therefore it is to remain free of capitalist purposes if critical reflection is to be prioritized. The ludic element is moved by inventiveness, fantasy and the reinvention of reality. When recreational activities meet a conducive learning environment, it is worth asking to what extent the disciplinary power of the technique, characteristic of educational processes in health, limits the irreverence and spontaneity of the game. In other words, the more the game/activity and its playfulness remain, the more we move toward the dimension of training, the center of critical education(99 Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.

10 Huizinga J. Homo Ludens: o jogo como elemento da cultura. São Paulo: Perspectiva; 2008.

11 Callois R. Os homens e os jogos: a máscara e a vertigem. Lisboa: Cotovia; 1990.

12 Haddad AE. Nursing and the national policy of education for health care professional for the Brazilian National Health System. Rev Esc Enferm USP. 2011;45(n.spe2):1803-9.

13 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.
-1414 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.).

Thus, it is necessary to examine the issues occasioned by games in healthcare from a different angle, as it is still poorly discussed in scientific publications on the subject(11 Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.

2 Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, et al. Educational games for health professionals. Cochrane Database Syst Rev. 2013;(1):CD006411.

3 Irons JY, Petocz P, Kenny DT, Chang AB. Singing as an adjunct therapy for children and adults with cystic fibrosis. Cochrane Database Syst Rev. 2014;(6):CD008036.

4 Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database of Syst Rev. 2015;(2):CD008349.

5 Morris BJ, Croker S, Zimmerman Z, Gill D, Romig C. Gaming science: the 'Gamification" of scientific thinking. Front Psychol [Internet]. 2013 [cited 2014 Feb 10];4:607. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
http://www.ncbi.nlm.nih.gov/pmc/articles...

6 Kuss JD. Internet gaming addiction: current perspectives. Psychol Res Behav Manag [Internet]. 2013 [cited 2014 Feb 10];6:125-37. Available from:Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
http://www.ncbi.nlm.nih.gov/pmc/articles...

7 Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.
-88 Mishra J, Bavelier D, Gazzaley A. How to assess gaming-induced benefits on attention and working memory.. Games Health J 2012;1(3):192-8.). Instead of investigating whether playing contributes to the purposes of education and health, it must be asked what happens to the inventive character of the game when they are set in different contexts of spontaneity, imagination and rebelliousness that characterize them. By reversing the question, we seek to identify the exercise of creativity and autonomy for subjects of knowledge in the restlessness of the game(1313 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.-1414 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.).

There are few instruments which evaluate the playfulness of games in educational settings, given the multitude of studies that are directed to behavioral or clinical purposes(11 Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.

2 Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, et al. Educational games for health professionals. Cochrane Database Syst Rev. 2013;(1):CD006411.

3 Irons JY, Petocz P, Kenny DT, Chang AB. Singing as an adjunct therapy for children and adults with cystic fibrosis. Cochrane Database Syst Rev. 2014;(6):CD008036.

4 Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database of Syst Rev. 2015;(2):CD008349.

5 Morris BJ, Croker S, Zimmerman Z, Gill D, Romig C. Gaming science: the 'Gamification" of scientific thinking. Front Psychol [Internet]. 2013 [cited 2014 Feb 10];4:607. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
http://www.ncbi.nlm.nih.gov/pmc/articles...

6 Kuss JD. Internet gaming addiction: current perspectives. Psychol Res Behav Manag [Internet]. 2013 [cited 2014 Feb 10];6:125-37. Available from:Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
http://www.ncbi.nlm.nih.gov/pmc/articles...

7 Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.
-88 Mishra J, Bavelier D, Gazzaley A. How to assess gaming-induced benefits on attention and working memory.. Games Health J 2012;1(3):192-8.). In general, the methodologies used for game development are based in interaction design and computer science(1515 Jegers K, Wiberg C. Learning while playing: design implications for edutainment games. In: Ferding RE. Handbook of research on effective electronic gaming in education. London: IGI Global; 2008. p.1449-59.-1616 Teng CI, Huang HC.More than flow: revisiting the theory of four channels of flow. Int J Comp Games Technol [Internet]. 2012 [cited 2015 July 20];2012:724917. Available from: Available from:http://www.hindawi.com/journals/ijcgt/2012/724917/
http://www.hindawi.com/journals/ijcgt/20...
). Thus, this study aimed to develop and validate an instrument to evaluate the playfulness of games in health education contexts, based on the investigative analysis of a board game aimed at training health professionals of the SUS (National Brazilian Healthcare System).

Method

This is a methodological, exploratory and descriptive research for validating the content of items in a questionnaire about the playfulness of health education games, based on the investigative analysis of a board game focused on training health professionals of the SUS. The methodological character is justified since it describes the validation steps of a research instrument and its exploratory aspect, and there is a lack of studies on the topic(1717 Sampieri RH, Collado CF, Lúcio PB. Metodologia de pesquisa. São Paulo: McGraw-Hill; 2006.-1818 Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2009.)

Phase 1 - Closing of items and inter-rater reliability by Kappa coefficient

Initially, an open questionnaire was applied to 50 graduates. The answers were subjected to content analysis. Next, the questions were defined from the Kappa Inter-rater Reliability Test for the inclusion of items(1717 Sampieri RH, Collado CF, Lúcio PB. Metodologia de pesquisa. São Paulo: McGraw-Hill; 2006.

18 Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2009.

19 Stemler SE. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Pract Assess Res Eval [Internet]. 2004 [cited 2015 Feb 09];9(4). Available from: Available from: http://pareonline.net/getvn.asp?v=9&n=4
http://pareonline.net/getvn.asp?v=9&n=4...
-2020 Cohen J. Weighted Kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70(4):213-20.).

This phase took place in the Faculdade de Ciências da Saúde at the Universidade de Brasília (UNB). The subjects were all 50 students enrolled in the Public Health Policy Management course in (Gestão de Políticas Públicas de Saúde - GPS), offered in two consecutive semesters by the Nursing Department of UNB, who agreed to participate in a game of " Banfisa" (Banco Fim-Saúde), which involves the construction and financing of SUS healthcare networks(2121 Silva LVS, Tanaka PSL, Pires MRGM. BANFISA and (IN) DICA-SUS na graduação em saúde: o lúdico e a construção de aprendizados. Rev Bras Enferm. 2015;68(1):124-30.). These students were chosen due to the adequacy of the educational context of the course which discusses SUS management and financing, thereby being in consensus with the themes of the Banfisagame. Therefore, the choice of subjects in an environment that favors dialogue of ludic activities within an education context was prioritized, being necessary to research the items being validated.

An open questionnaire that had been developed in previous studies was used, with the dimensions and variables being(1414 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.): a) the profile of players (gender, education, age, occupation, affinity for board games, study habits); b) training components of learning (learning by association, perception of learning, self-assessment, understanding concepts, interaction and group discussion/active participation); c) emotions and attitudes during the game (willingness and satisfaction of achievement, motivation to study, ambiguity in feelings of pleasure and tension in the game, disinterest/willingness to abandon the game, concern/frustration); d) tactics of the game (understanding of the rules, quality of the cards, clarity of the board, reinventing the game).

Data collection occurred in the classroom of the GPS class, between November 2012 and June 2013. Students were asked to play Banfisaand, soon after playing they answered the aforementioned questionnaire containing 12 open items. The average answering time was 15 minutes. The 50 responses were transcribed and organized by items. Next, a floating evaluation of the material was conducted, defining a matrix for content analysis ​​according to the columns(1717 Sampieri RH, Collado CF, Lúcio PB. Metodologia de pesquisa. São Paulo: McGraw-Hill; 2006.-1818 Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2009.): a- dimensions (playfulness; training components of learning, profiles of the players); b- categories (variables of the previously referred to dimensions); c- empirical subcategories (extracted from the content analysis of responses from the open questionnaire); d- number of responses that confirm the subcategory (unit of analysis that ratifies the subcategory).

The content analysis was done by two researchers of educational technologies for health. The responses were grouped according to their empirical subcategory, following the criteria of a single rating response, followed by the simple frequencies of these units of analysis. The consensus estimate was calculated by the Kappa coefficient, estimating the percentage of inter-observer agreement beyond what would be expected solely by chance(2020 Cohen J. Weighted Kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70(4):213-20.). This measure has one as a maximum value and values ​​close to or below zero (no agreement or as expected by chance). For inclusion of the item, an agreement above 0.60 (p<0.005) was considered substantial(1717 Sampieri RH, Collado CF, Lúcio PB. Metodologia de pesquisa. São Paulo: McGraw-Hill; 2006.

18 Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2009.

19 Stemler SE. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Pract Assess Res Eval [Internet]. 2004 [cited 2015 Feb 09];9(4). Available from: Available from: http://pareonline.net/getvn.asp?v=9&n=4
http://pareonline.net/getvn.asp?v=9&n=4...
-2020 Cohen J. Weighted Kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70(4):213-20.).

Phase 2 - Construction and content validity of the items

For the construction and content validity of items, the procedures of scale construction were used with a focus on the theoretical pole, which addresses the underlying theory of the research(2222 Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista ACP, et al. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.). The items from Step 1 with a Kappa higher than 0.60 were included for constructing the items. The items were reviewed in light of the concepts:

a- Playfulness/Ludic character: free expression inherent to the game, constitutive of human culture. It is characterized by irreverence, imagination, being external to everyday life, being not-for-profit and immersing the players in the game. The playing occurs in boundaries of space and time, according to certain rules accepted among participants in an enthusiastic, festive or sacred environment in the midst of tension and pleasure, ambiguously. Characteristics inherent to the game are repetition, situation of choosing, proper language and transience between seriousness and triviality;

b- Formative Learning Components: aspects of learning centered on the subjects of knowledge which are capable of their own projects. The necessary attributes for transforming education and other social relations in a given training context, based on the characteristics of autonomy, creativity, capacity to elaborate, and critical reflection on the reality of those who learn;

c- Profile of players: specificities of the players such as their study habits and the degree of affinity for games in general. It includes identifying variables as gender, age, education and occupation.

The validation of item content to verify the degree to which each element of an instrument is representative of a theoretical concept was performed by a judges' analysis(2323 Rubio DM, Ber-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003; 27(2):94-111.-2424 Polit DF, Beck TC, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67.). The judges were specialized professors and researchers from public universities with production in the area of education on health. The analysis was done through an electronic form constructed in Google docs containing the described concepts, followed by the related items. The questionnaire was sent to 14 experts, and 10 were returned completed. Each item was assessed on a four point scale, according to relevance criteria (importance, correspondence) and clarity (unambiguous meaning, objective) with respect to the concept of the item(2222 Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista ACP, et al. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.-2323 Rubio DM, Ber-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003; 27(2):94-111.).

For data analysis, Content Validity Index (CVI) andReliability Index or Inter-rater Agreement(IRA) were used. CVI assesses the content of the items (CVIA - Content Validity IndexApplied to each item) and of the instrument (CVI) in relation to the representability of the measurement and it is considered valid with an agreement above 0.80. IRA evaluates the extent to which judges are reliable in rating the items (above 0.80 of agreement on the total dimension). To overcome the limitation of the CVI which does not include agreements by chance, the following literature recommendations were followed(2323 Rubio DM, Ber-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003; 27(2):94-111.-2424 Polit DF, Beck TC, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67.): a- number of judges between 8 and 12; b- CVI of the item higher than 0.78; c- calculating IRA. The criteria for the final review were(2222 Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista ACP, et al. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.): a- CVI of the item; b- redundancy; c- balance between positive and negative statements in each dimension; d- frequency of spontaneous responses to the open-ended questionnaire (Phase 1); e- coherence between the item and the theories that the playfulness of games are based on in the literature.

The study was approved by the Ethics Committees of the UNB Faculdade de Ciências da Saúde (number 322.709/13) of the Research and Education Foundation of the Federal District (Fepecs-SES-DF N193/11) and integrates the research project"Violets game: ludic and educational technology in confronting violence against women " financed by CNPq (process number 405302/2012-6).

Results

Construction of items based on responses of the open-ended questionnaire

Among the respondents of the open questionnaire, women (n=45; 90%) were more frequent than men (n=5; 10%). Most were young, aged between 18 and 22 years of age (n=45; 90%) and with just a few being older than 23 (n=4, 10%). All subjects were undergraduates in health, some had already completed a technical course (n=2; 4%), and others were post-graduates (n=4, 8%). From the content analysis of the open answers, 44 empirical subcategories were defined that reported possible items for the instrument on the playfulness of games for health education. Regarding the consensus estimate of empirical subcategories extracted from the content analysis ( Table 1), almost all obtained Kappa above 0.60, with the exception of the self-assessment (k=0.485; p<0.001).

Table 1
Frequency and Kappa concordance of the open-ended questionnaire responses to define the items on the playfulness of games - Brasilia, DF, Brazil, 2014

Preliminary version of instrument and content validity of the items

The review of 43 defined items in the first stage resulted in a preliminary version with 62 items, submitted to analysis by 10 experts. Regarding the relevance criteria ( Table 2), 54 (87%) items obtained an estimated CVIA and IRA above 0.80, and eight were below 0.80. The items were clear (IRA above 0.90), with the exception of 5. The content of the items was validated as relevant (n=54; 87%) and clear (n=62; 98.3%). The experts were confident in the assessments (IRA above 0.80). The items of the instrument measured by the CVA obtained values higher than recommended (CVIA relevance 0.88; CVIA clarity 0.96).

Table 2
Agreement of judges according to the relevance and clarity of the items by the instrument dimensions, based on CVI and IRA tests - Brasilia, DF, Brazil, 2014

Of the 62 items submitted to the validation, nine were excluded for the following reasons ( Table 2): a- one CVI below 0.80 (items: 5; 11; 28; 32; 48; 60); b- redundancy with other items (items: 11 and 41) and c- low response frequency of the students in Phase 1 (item 56). Two items (items: 53 and 54) had the headings modified, one to make the sentence shorter and the other to change the statement from positive to negative in view of it being balanced in the same respective dimension. The final instrument contains 53 items ( Chart 1), distributed in the dimensions ludic characteristics (items 1-26), training components of learning (items 27-45) and profile of the players (items 46-53).

Chart 1
Items of the instrument to evaluate the playfulness of games in health education contexts - Brasília, DF, Brazil, 2015

Discussion

The exploratory study in the first phase is relevant to defining the possible items on the playfulness of games in health education contexts, given the lack of this kind of instruments in the literature(11 Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.

2 Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, et al. Educational games for health professionals. Cochrane Database Syst Rev. 2013;(1):CD006411.

3 Irons JY, Petocz P, Kenny DT, Chang AB. Singing as an adjunct therapy for children and adults with cystic fibrosis. Cochrane Database Syst Rev. 2014;(6):CD008036.

4 Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database of Syst Rev. 2015;(2):CD008349.

5 Morris BJ, Croker S, Zimmerman Z, Gill D, Romig C. Gaming science: the 'Gamification" of scientific thinking. Front Psychol [Internet]. 2013 [cited 2014 Feb 10];4:607. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
http://www.ncbi.nlm.nih.gov/pmc/articles...

6 Kuss JD. Internet gaming addiction: current perspectives. Psychol Res Behav Manag [Internet]. 2013 [cited 2014 Feb 10];6:125-37. Available from:Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
http://www.ncbi.nlm.nih.gov/pmc/articles...

7 Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.
-88 Mishra J, Bavelier D, Gazzaley A. How to assess gaming-induced benefits on attention and working memory.. Games Health J 2012;1(3):192-8.). The establishment of an open questionnaire in order to define items from the answers is recommended in order to build phrases closer to the spontaneity of statements of the determined target audience, providing greater identity between the items and the statements(1717 Sampieri RH, Collado CF, Lúcio PB. Metodologia de pesquisa. São Paulo: McGraw-Hill; 2006.

18 Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2009.
-1919 Stemler SE. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Pract Assess Res Eval [Internet]. 2004 [cited 2015 Feb 09];9(4). Available from: Available from: http://pareonline.net/getvn.asp?v=9&n=4
http://pareonline.net/getvn.asp?v=9&n=4...
).

The profile of subjects, mostly young undergraduates of health care courses, contributed to contextualize the restlessness of a playful learning environment(1313 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.-1414 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.). The high estimate of intra-judge agreement measured by the kappa coefficient is based on previous research that confirmed the variables used in this study(1414 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.,2121 Silva LVS, Tanaka PSL, Pires MRGM. BANFISA and (IN) DICA-SUS na graduação em saúde: o lúdico e a construção de aprendizados. Rev Bras Enferm. 2015;68(1):124-30.). The low agreement of the judges in the subcategory of self-assessmentis consistent with the literature, since this feature is not inherent in the ludic/playfulness component. Unlike the discipline required in self-assessment, the attraction of the game lies in the fact that it is inherently spontaneous, fun and uncertain, distinct from any purpose other than to strengthen his innate restlessness(99 Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.

10 Huizinga J. Homo Ludens: o jogo como elemento da cultura. São Paulo: Perspectiva; 2008.

11 Callois R. Os homens e os jogos: a máscara e a vertigem. Lisboa: Cotovia; 1990.

12 Haddad AE. Nursing and the national policy of education for health care professional for the Brazilian National Health System. Rev Esc Enferm USP. 2011;45(n.spe2):1803-9.
-1313 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.).

In the second phase, eight items were not considered relevant to the study by the judges. From these, six items were excluded from the final version of the instrument for not expressing significant relevance to the concept of playfulness or they displayed redundancy with other items that were evaluated as better. In the critical review, the item "1- Rules were reinvented during the game" and its opposite, "2- We followed the rules of the game completely," remained on the instrument, despite having reached an agreement below the adopted cutoff point. This decision is justified on the theoretical concepts that are the basis of ludic activities, in which the inventive, creative and indeterminate character of the game is widely recognized(11 Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.,77 Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.,99 Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.

10 Huizinga J. Homo Ludens: o jogo como elemento da cultura. São Paulo: Perspectiva; 2008.

11 Callois R. Os homens e os jogos: a máscara e a vertigem. Lisboa: Cotovia; 1990.

12 Haddad AE. Nursing and the national policy of education for health care professional for the Brazilian National Health System. Rev Esc Enferm USP. 2011;45(n.spe2):1803-9.

13 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.

14 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.

15 Jegers K, Wiberg C. Learning while playing: design implications for edutainment games. In: Ferding RE. Handbook of research on effective electronic gaming in education. London: IGI Global; 2008. p.1449-59.
-1616 Teng CI, Huang HC.More than flow: revisiting the theory of four channels of flow. Int J Comp Games Technol [Internet]. 2012 [cited 2015 July 20];2012:724917. Available from: Available from:http://www.hindawi.com/journals/ijcgt/2012/724917/
http://www.hindawi.com/journals/ijcgt/20...
,2525 Savi R, Wangenheim CGV, Ulbricht V, Vanzin T. Proposta de um modelo de avaliação de jogos educacionais. Rev Renote Novas Tecnol Educ [Internet]. 2010 [citado 2015 fev. 09];8(3). Disponível em: Disponível em: http://seer.ufrgs.br/renote/article/view/18043
http://seer.ufrgs.br/renote/article/view...
-2626 Keller JM. Motivational design for learning and performance: the ARCS model approach. New York: Springer; 2009.). That is, the reinvention of the rules by the participants during a match is inherent to the ludic activity, since it is from this ambivalence between normativity and subversion that is needed for the game(1212 Haddad AE. Nursing and the national policy of education for health care professional for the Brazilian National Health System. Rev Esc Enferm USP. 2011;45(n.spe2):1803-9.

13 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.
-1414 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.).

The subjectivity expressed in the subjection to the rules and the renunciation of impulsive action as a gateway to the imaginary world fostered by the game is constituted in a reinvention of reality that is immersed in the desire and castration of relationships, inherent to the formation of the human psyche(2727 Vigotsky LS. A formação social da mente. São Paulo: Martins Fontes; 1991.). Moreover, the perspective of playfulness sought in this study focused on the disruptive and insubordinate character of the game - a trait necessary to train subjects of knowledge. So, the more players reinvent rules while immersed in ludic activities, the greater the chance of imaginative, restless and creative character being revealed - and these are the attributes of the ludic activity that are intended to be measured, aimed at the formation of autonomous individuals (99 Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.

10 Huizinga J. Homo Ludens: o jogo como elemento da cultura. São Paulo: Perspectiva; 2008.

11 Callois R. Os homens e os jogos: a máscara e a vertigem. Lisboa: Cotovia; 1990.

12 Haddad AE. Nursing and the national policy of education for health care professional for the Brazilian National Health System. Rev Esc Enferm USP. 2011;45(n.spe2):1803-9.

13 Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.

14 Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.

15 Jegers K, Wiberg C. Learning while playing: design implications for edutainment games. In: Ferding RE. Handbook of research on effective electronic gaming in education. London: IGI Global; 2008. p.1449-59.
-1616 Teng CI, Huang HC.More than flow: revisiting the theory of four channels of flow. Int J Comp Games Technol [Internet]. 2012 [cited 2015 July 20];2012:724917. Available from: Available from:http://www.hindawi.com/journals/ijcgt/2012/724917/
http://www.hindawi.com/journals/ijcgt/20...
,2121 Silva LVS, Tanaka PSL, Pires MRGM. BANFISA and (IN) DICA-SUS na graduação em saúde: o lúdico e a construção de aprendizados. Rev Bras Enferm. 2015;68(1):124-30.,2525 Savi R, Wangenheim CGV, Ulbricht V, Vanzin T. Proposta de um modelo de avaliação de jogos educacionais. Rev Renote Novas Tecnol Educ [Internet]. 2010 [citado 2015 fev. 09];8(3). Disponível em: Disponível em: http://seer.ufrgs.br/renote/article/view/18043
http://seer.ufrgs.br/renote/article/view...
-2626 Keller JM. Motivational design for learning and performance: the ARCS model approach. New York: Springer; 2009.).

The item I learn better when I express my creativity, although it had a successful consensus rating among the judges (CVIA/relevance 0.90 and CVIA/clarity 1.0), was excluded for having a low frequency of responses among the subjects who answered the open questionnaire. In all these cases, the criteria of objectivity, simplicity, clarity, precision and variety of items were prioritized in decisions, as recommended in the literature(2222 Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista ACP, et al. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.). The use of the various ways to calculate the CVI for the validation of items and the method adopted to overcome the limitations of this index has increased the reliability when using this statistical measure, according to published recommendations(2323 Rubio DM, Ber-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003; 27(2):94-111.-2424 Polit DF, Beck TC, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67.). The final version of the instrument, with 53 items distributed in three dimensions that allow for the operationalization of ludic activity/playfulness, includes an adequate amount to measure this construct as a minimum of 20 items is recommended(2222 Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista ACP, et al. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.). The designed instrument can be applied to evaluate the playfulness of other educational games in different health contexts, especially those focused on the critical training of professionals.

Conclusion

The inventive feature of playfulness as a substrate of critical education is little explored in studies on the subject, which gives originality to this study. To assess the playfulness of ludic games/activities, 53 validated items check the degree of involvement, immersion and reinvention of the subjects in the game, along with the dynamics and gameplay. In conjunction with playfulness, the educational context is measured by the perception of learning, the feelings, the thoughts and the degree of cooperation between the players, among other items. Finally, study habits, their affinity for games and sociodemographic characteristics of the participants make up the profile of players measured by the validated instrument. One limitation of this study points to the need to make advancements in the other planned stages for elaborating the scales, and the empirical and analytical procedures necessary for validating the construct and criterion of the instrument.

References

  • 1
    Lu AS, Baranowski T, Thompson D, Buday R. Story immersion of videogames for youth health promotion: a review of literature. Games Health J. 2012;1(3):199-204.
  • 2
    Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, et al. Educational games for health professionals. Cochrane Database Syst Rev. 2013;(1):CD006411.
  • 3
    Irons JY, Petocz P, Kenny DT, Chang AB. Singing as an adjunct therapy for children and adults with cystic fibrosis. Cochrane Database Syst Rev. 2014;(6):CD008036.
  • 4
    Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database of Syst Rev. 2015;(2):CD008349.
  • 5
    Morris BJ, Croker S, Zimmerman Z, Gill D, Romig C. Gaming science: the 'Gamification" of scientific thinking. Front Psychol [Internet]. 2013 [cited 2014 Feb 10];4:607. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
    » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3766824/
  • 6
    Kuss JD. Internet gaming addiction: current perspectives. Psychol Res Behav Manag [Internet]. 2013 [cited 2014 Feb 10];6:125-37. Available from:Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
    » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3832462/
  • 7
    Mellecker R, Lyons EJ, Baranowski T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: a narrative review.. Games Health J 2013;2(3):142-9.
  • 8
    Mishra J, Bavelier D, Gazzaley A. How to assess gaming-induced benefits on attention and working memory.. Games Health J 2012;1(3):192-8.
  • 9
    Alves MA. Interpretação e compreensão: da hermenêutica metodológica à experiência hermenêtica como fundamento do saber filosófico. Princípios (Natal). 2011;18(30)181-98.
  • 10
    Huizinga J. Homo Ludens: o jogo como elemento da cultura. São Paulo: Perspectiva; 2008.
  • 11
    Callois R. Os homens e os jogos: a máscara e a vertigem. Lisboa: Cotovia; 1990.
  • 12
    Haddad AE. Nursing and the national policy of education for health care professional for the Brazilian National Health System. Rev Esc Enferm USP. 2011;45(n.spe2):1803-9.
  • 13
    Mészaros I. A educação para além do capital. São Paulo: Boitempo; 2005.
  • 14
    Pires MRGM, Guilhem D, Gottems LBD. The (IN)DICA-SUS game: a strategy of game-based learning on the Unified Health System. Texto Contexto Enferm. 2013; 22(2):379-88.
  • 15
    Jegers K, Wiberg C. Learning while playing: design implications for edutainment games. In: Ferding RE. Handbook of research on effective electronic gaming in education. London: IGI Global; 2008. p.1449-59.
  • 16
    Teng CI, Huang HC.More than flow: revisiting the theory of four channels of flow. Int J Comp Games Technol [Internet]. 2012 [cited 2015 July 20];2012:724917. Available from: Available from:http://www.hindawi.com/journals/ijcgt/2012/724917/
    » http://www.hindawi.com/journals/ijcgt/2012/724917/
  • 17
    Sampieri RH, Collado CF, Lúcio PB. Metodologia de pesquisa. São Paulo: McGraw-Hill; 2006.
  • 18
    Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2009.
  • 19
    Stemler SE. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Pract Assess Res Eval [Internet]. 2004 [cited 2015 Feb 09];9(4). Available from: Available from: http://pareonline.net/getvn.asp?v=9&n=4
    » http://pareonline.net/getvn.asp?v=9&n=4
  • 20
    Cohen J. Weighted Kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70(4):213-20.
  • 21
    Silva LVS, Tanaka PSL, Pires MRGM. BANFISA and (IN) DICA-SUS na graduação em saúde: o lúdico e a construção de aprendizados. Rev Bras Enferm. 2015;68(1):124-30.
  • 22
    Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista ACP, et al. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.
  • 23
    Rubio DM, Ber-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003; 27(2):94-111.
  • 24
    Polit DF, Beck TC, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67.
  • 25
    Savi R, Wangenheim CGV, Ulbricht V, Vanzin T. Proposta de um modelo de avaliação de jogos educacionais. Rev Renote Novas Tecnol Educ [Internet]. 2010 [citado 2015 fev. 09];8(3). Disponível em: Disponível em: http://seer.ufrgs.br/renote/article/view/18043
    » http://seer.ufrgs.br/renote/article/view/18043
  • 26
    Keller JM. Motivational design for learning and performance: the ARCS model approach. New York: Springer; 2009.
  • 27
    Vigotsky LS. A formação social da mente. São Paulo: Martins Fontes; 1991.

Publication Dates

  • Publication in this collection
    Dec 2015

History

  • Received
    12 June 2015
  • Accepted
    29 Aug 2015
Universidade de São Paulo, Escola de Enfermagem Av. Dr. Enéas de Carvalho Aguiar, 419 , 05403-000 São Paulo - SP/ Brasil, Tel./Fax: (55 11) 3061-7553, - São Paulo - SP - Brazil
E-mail: reeusp@usp.br