Evaluation of semantic equivalence and internal consistency of a Portuguese version of the Internet Addiction Test (IAT)

Abstracts

OBJETIVO: Este estudo teve por objetivo a adaptação transcultural do Internet Addiction Test (IAT) para o idioma português. MÉTODOS: O trabalho consistiu em cinco etapas: (1) tradução; (2) retradução; (3) revisão técnica e avaliação da equivalência semântica por profissionais da área; (4) avaliação do instrumento por uma amostra de estudantes, avaliando-se o seu grau de compreensão; e (5) análise da consistência interna por meio do coeficiente alfa de Cronbach. RESULTADOS: O instrumento foi traduzido e adaptado para o idioma português, demonstrando ser facilmente compreendido e apresentando valor de consistência interna de 0,85. CONCLUSÃO: O instrumento encontra-se traduzido e adaptado para o português e apresenta consistência interna satisfatória. São necessárias análises de equivalência de mensuração e reprodutibilidade.

Adaptação transcultural; escalas; estudos de validação; teste de dependência de internet; Brasil

ORIGINAL ARTICLE

^IPrograma de Transtornos Alimentares do Ambulatório de Bulimia e Transtornos Alimentares do Instituto de Psiquiatria do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (Ambulim-IPq-HCFMUSP)

^IIDepartamento de Medicina Comunitária e Familiar e Departamento de Epidemiologia e Bioestatística da Universidade da Califórnia, São Francisco, Estados Unidos

^IIIAmbulatório do Jogo Patológico (AMJO) do IPq-HCFMUSP

Address correspondence to

ABSTRACT

OBJECTIVE: The aim of this study was to cross-culturally adapt the Internet Addiction Test (IAT) to the Portuguese language.

METHODS: The translation and evaluation process consisted of five steps: (1) translation; (2) back-translation; (3) peer review and evaluation of semantic equivalency by experts; (4) instrument evaluation through a sample of students, by evaluating their understanding level; and (5) analysis of the instrument's internal consistency (Cronbach's alpha-coefficient).

RESULTS: The instrument was translated and adapted to Portuguese. As shown, the Portuguese version of the IAT was easily understood and the internal consistency value was 0.85.

DISCUSSION: The translated and adapted IAT displays a satisfactory internal consistency. In a further step, measurement and reproducibility analyses have to be conducted.

Keywords: Cross-cultural adaptation, scales, validity studies, internet addiction, Brazil.

Introduction

With the great technological advances seen in the last decades, the internet has become more and more popular worldwide, contributing to one of the major technological revolutions of the 20th and 21st centuries. The internet not only favors communication and the search for information, but has also become an important social contact tool by making possible new uses and habits and has created a new virtual life setting for approximately 500 million people. The benefits of using chats (instant electronic communication via "MSN" and others), for example, are considered helpful and an important source of aid by many introverted and shy people. But together with the increase of the worldwide web popularity, reports of individuals who are becoming "addicted" to virtual reality started to appear in the general media and in the scientific literature^1-9.

Internet addiction has gained attention in scientific and legal publications around the world as a result in the rapidly growing popularity of the World Wide Web. We all experience the phenomenon of living with the new Digital Generation (also known as Gen-D) composed of young people born 1990 to 2000 who were raised with constant exposure to virtual webs. These young people, according to some studies, have characteristics different from previous generations and, for this reason, sometimes exhibit unique behavior. Furthermore, the growing use of internet has made the dividing line between recreational and pathologic use more and more tenuous, to the point of constantly testing mental health professionals who do not know how to deal with this dynamic, whether in their patients or in their own personal lives¹⁰.

The first attempts to describe this phenomenon took place in the 1990's. Thomas Hodgkin, for example, is said to have first described the problem. But it was the American psychiatrist, Ivan K. Goldberg, who became best known regarding this new, unrecognized condition that had not yet received clinical attention or even a name. In 1986, he created PsyCom.Net. This was a type of "cyber club" where therapists found information and exchanged experiences regarding internet abuse. To demonstrate the seriousness of the matter, he coined the term "internet addiction disorder" (IAD), the symptoms of which included "abandonment or the reduction of importance of professional or social activities because of the use of internet", "presenting with fantasies or dreams about the internet", and "presenting with voluntary or involuntary typing movements of the fingers", among others¹¹.

In 1995, Mark Griffiths proposed the term "technology dependence" as a result of the non-chemical interaction between man and machine, usually involving characteristics such as the induction and reinforcement of behaviors. According to the author, this dependency constitutes a subset of the behavioral dependencies, presenting a profile including withdrawal, mood change, tolerance, and relapse.

In 1996, the American psychologist, Kimberly Young, presented one of the first research studies about internet abuse at the annual conference of the American Psychological Association in Toronto entitled, "Internet dependence: the emergence of a new disorder". Young conducted an investigation based on a combination of criteria derived from those used on DSM-IV for substance abuse to create a first conceptual outline. This first study included 496 students, of whom 396 described excessive internet use resulting in significant harm to their routines of living. Even though the sample was small compared to the 47 million internet users at that time, this study was considered the first empiric attempt to characterize the problem¹¹.

After further research, she modified her criteria two years later to include 8 of the 10 criteria used in DSM-IV to describe pathologic gambling: 1) Excessive concern about the internet; 2) Need to increase on-line time to obtain the same satisfaction; 3) Repeated efforts to decrease on-line time; 4) Irritability and/or depression; 5) Exhibiting labile emotions when internet access is restricted (the internet as a form of emotional regulation); 6) Staying on-line longer than planned; 7) Excessive use effecting work and social relations; and 8) Lying to others about one's amount of on-line time¹².

Given the magnitude and severity of individuals with this problem, the American Psychiatric Association has considered the inclusion of this diagnosis in the next version of the Diagnostic and Statistical Manual of Mental Disorder (DSM-5)¹³. However, attempts to estimate the number of people presenting with pathological use are variable due to different definitions that, inevitably, create distinct parameters of understanding and measurement, thereby hindering a common interpretation or estimate to define the prevalence of this disorder in the general population¹.

Some evaluation and measurement proposals for internet addiction-related behaviors are described in the literature, including: the Chinese Internet Addiction Inventory (CIAI)¹⁴, the Generalized Problematic Internet Use Scale¹⁵, and the Internet Consequences Scale (ICONS)¹⁶. Although a number of these instruments are used for evaluation, the Internet Addiction Test (IAT)¹⁷ is still the most commonly used, with validated versions being available in several languages. It consists of 20 self-administered items with responses given in a Likert-type scale, ranging from 1 (rarely) to 5 (always). The higher the score, the higher the addiction severity level. It was designed to assess which areas of an individual's life might be affected by their excessive internet use¹⁷. When evaluating the psychometric properties by factor analyses, six domains were identified: salience, excessive use, neglect of work, anticipation, lack of control, and neglect of social life.

The original study¹⁷ describing the IAT reported factorial analysis, internal consistency, and correlations of the six domain subscales with age and internet use. The factorial analysis showed the existence of six domains that explained 62.8% of the variance. These included salience - items 10, 12, 13, 15, 19; excessive use - items 1, 2, 14, 18, 20; neglect of work - items 6, 8, 9; anticipation - items 7, 11; lack of control - items 5, 16, 17; and neglect of social life - items 3, 4. Internal consistency varied from 0.54 to 0.82 for the six domains. There was positive correlation for the salience and excessive use subscales with average internet use (r = 0.32 p < 0.005 and r = 0.27 p < 0.005 respectively) and also of the neglect of social life and internet use (r = 0.22 p < 0.005). There were negative correlations of excessive use with age (r = - 0.27 p < 0.005) and of neglect of social life with duration of use (r = - 0.26 p < 0.005).

Two validation studies have been done of IAT. In the first, done in Switzerland¹⁸, factorial analysis showed good psychometric characteristics, even though a single factor accounted for all of the variance of the scale (45%). An Italian study¹⁹ showed the existence of the six factors found in the original study, explaining 55.6% of the total variance. The IAT is an easily understood and completed instrument, in addition to being self-administered¹⁷.

Experts emphasize the importance of a rigorous translation and transcultural adaptation process to assure that the translated instrument continues to accurately measure the desired construct²⁰. Even though there are studies that show the importance of describing the process of translation and transcultural adaptation of instruments, standard procedures for this process are lacking^20-22. In this sense, cultural differences can interfere with the measurement of constructs, since the process of translation and transcultural adaptation is subject to many conceptual and semantic distortions that can cause an instrument to lose validity²³.

Since 2010, Brazil became the world leader in time spent online at home²⁴ and internet access has been increasingly available, which presents a potential for resulting addiction problems. A translated and adapted version in Portuguese would thus be useful for epidemiologic and clinical research. In this way, the detection of this behavior can be accomplished so as to understand the possible magnitude of the problem. Thus, it is important to have an instrument available and adapted to evaluate internet addiction for the Brazilian population. The aim of this communication is to present the process of transcultural adaptation of the Internet Addiction Test, including translation, evaluation of semantic equivalence, and internal consistency analysis.

Methods

We based the process of transcultural adaptation on procedures suggested by Reichenheim e Moraes²⁰ e Moraes et al.²⁵ and applied by Kachani et al.²⁶, Teixeira et al.²⁷ and Toledo et al.²⁸, that include evaluation of 6 types of equivalence: semantic, conceptual, items, measurement, operational, and functional. Evaluation of conceptual equivalence should be based on literature review and discussion with both experts and members of the target population. Item equivalence should also include both experts and the target population. Semantic equivalence involves translation, back-translation, evaluation of equivalence between the back translation and the original instrument, and input from specialists and the target population for final adjustments before pre-testing. Operational equivalence involves the evaluation of the group completing the instrument regarding its relevance and completeness. Measurement equivalence involves psychometric studies of validity and of factors. Functional equivalence is an overall combination of all of these.

Based on this, we developed a five-step process^20-23. The first step consisted of the translation of the original instrument from English to Portuguese, independently performed by two experienced, English-proficient investigators (CNA, MAC). The second step was the back translation of the initial Portuguese versions (translation 1 and 2) to English by a native English speaker (NH).

The third step was the technical review and semantic equivalence evaluation of these versions, which was carried out by two psychologists specialized in scale adaptation (CNA, MAC) by prioritizing the referential meaning and general meaning²⁰ of the instrument. Adjustments were made and a new version was elaborated and intelligibility (capacity to be correctly understood) and degree of understanding of each question and the instrument as a whole were evaluated. For that, 10 mental health professionals with experience with impulse control disorder (three psychiatrists, three psychologists, three nutritionists, and a physical educator) were asked to evaluate the instrument, question by question, and respond using an adapted verbal-numeric scale^19,20. They were instructed to answer the following question: "Did you understand what was asked?." Answers were given using a Likert-type scale: 0 - I did not understand it at all; 1 - I understood it a little; 2 - I somewhat understood it; 3 - I understood almost everything, but I had some questions; 4 - I understood almost everything; 5- I understood it perfectly and had no questions. We considered answers 0, 1, 2, and 3 to be indicators of poor understanding^20,21. Experts were also asked, in case they did not understand the question or the language used did not seem to be suitable, to suggest changes, justifying their reasoning. Based on this, a new version of the instrument was designed.

In the fourth and fifth steps, the instrument was tested with 115 university students (38 male and 77 female), mean age 23 years (standard deviation: 3.7 years), in the Business Administration Program. These university students were selected through simple randomization by lot. Of 18 classrooms, two were randomly selected for the study. On a single occasion during the school term, all students present were invited to voluntarily participate, receiving instructions from the first author (MAC), assuring the uniformity of data collection. With the consent of the participant, the instrument was self-applied, in group form, in the same classroom. No refusals and/or dropouts occurred. To part of the group (38 subjects), the instrument was applied, along with completion of an adapted verbal-numeric scale to rate comprehension^21,22. To the rest of the group (77 subjects), the instrument was applied to verify the internal consistency level of questions, through Cronbach's alpha coefficient analysis.

Statistical analyses were performed using SPSS version 15.0. Medians and standard deviations were calculated for comprehension scores. Internal consistency for the instrument was calculated using Cronbach's alpha.

Authorization from the originator of the scale to conduct the research was requested and granted. All of the participants signed an Informed Consent Form and the present study is in conformity with the National Health Council, ethical norms nº 196, dated 10/10/1996.

Results

In the translation, some expressions needed to be adapted. The term "on-line" and "off-line" were adapted in all of the questions to the expression "on the internet" and "disconnected", respectively. Stylistic modifications such as omitting the pronoun "you" when feasible were applied in order to keep the flow of text in Portuguese.

We sometimes chose Portuguese words that were not a direct translation of the original English words, but that retained the semantic content and were more colloquial, thus keeping the text accessible. In items 2 and 4, the verbs "neglect" and "form" were replaced with "abandon" and "create", respectively. In item 6, the use of the expression "are impaired" and the use of the verb "spend" were replaced with the terms "suffer" and "spend on line". In item 8, the verb "suffer" was replaced with "fall" and moved to the beginning of the sentence. In question 10, the expression "thoughts of the internet" required special attention. In item 15, the word "fantasize" was replaced with "imagine". In item 13, "snap" was replaced with "explode" and the expression "act annoyed" with "irritated". The word "log-ins", in item 14, was replaced with "connected" and in the expression "to cut down the amount of time" the word "amount", in item 17, was omitted in the Portuguese version. In item 18, the expression "how long" was replaced with "the amount of time" (Table 1).

For the analysis of verbal comprehension, in the experts' opinion, the questions were easy to understand, recording mean values above 4.1 (maximum value, 5.0). The same was true for the sample of students, which recorded a mean above 4.0 (Table 2). It was thus observed that professionals as well as students easily understood the semantic content of the questions that comprise the instrument.

The instrument showed satisfactory internal consistency for all items (0.84) except for numbers 1 and 7 (Table 2). Our sample size was not ideal to evaluate scale domains (best practices for factor analysis requires at least 10 subjects for each item within the scale²⁹). An exploratory factor analysis of our data confirmed a 6-factor structure for the translated version, but the items clustering for each domain coincided only partially between the original and the translated versions. We therefore present Cronbach's alpha only for the whole instrument, leaving evaluation of the IAT's factorial structure and domains for future studies.

Discussion

Studies that describe the process of transcultural adaptation are essential to guarantee the integrity of a translated instrument. Although standardized diagnostic criteria contribute directly to research and clinical psychiatry, by using a common international language, inadequate attention was given to cultural determinants related to mental phenomena, and, because of this, instruments came to be used without appropriate adaptation. This situation has changed, and the need for psychometric evaluation of translated instruments is now a consensus.

The present study describes steps of the transcultural adaptation process of the IAT and presents data that characterize some of the IAT's psychometric properties. We endeavored to take appropriate precautions in this study.

Regarding the translation, in nearly all the questions, verbs, pronouns and verbal conjunctions were adjusted, which ensured that the connotative and denotative meanings were respected when transferring the meaning of words between the two languages¹⁹. Thus, understanding of the instrument is maximized, accomplishing one of the essential and necessary steps in the semantic equivalence evaluation. This can be noted in the good values recorded for the verbal understanding scores.

The transcultural adaptation process consisted of evaluations from experts and students, by means of verbal understanding analysis. The instrument was easy to understand, with no structural changes suggested. Therefore, there was no need for further adaptation, and the same components in the original version were maintained in the translated version.

In the internal consistency analysis, the instrument showed good results (0.85), quite close to the values in the original study (0.54 to 0.82)¹⁷. When calculating Cronbach's alpha, Bland and Altman³¹ suggest that each item is individually tested in relation to all the remaining items of the instrument. In this way an item should be eliminated from the instrument if the scale's final alpha coefficient is higher without it. Thus, on a strictly psychometric basis items 1 and 7 should be dropped, because they do not relate with the remaining items in the scale (low item-total correlation, see Table 2) and they do not contribute to the scales' final alpha (the scale's alpha in fact improve without them - see alpha-if-item-deleted column in Table 2). However, both items do translate behaviors that clinically represent typical behaviors of an impaired relationship with the internet and keeping them in the scale does not represent a major loss in internal coherence since the contribution from their exclusion represents a minimal addition of 0.0165 points in the final Cronbach's alpha. Therefore, our recommendation for now is to keep items 1 and 7 and re-visit their psychometric properties in larger samples to see if these preliminary findings are consistent. If they are, one possible explanation would be that they describe behaviors so common that they are endorsed by both people who have problems with the internet and those who don't, while the remaining items in the scale relate more exclusively to maladaptive use of the internet.

The Internet Addiction Test was translated and adapted to the Portuguese language, and this version is now available. Internal consistency analysis showed satisfactory results, with values very close to those in the original study. More research is needed to complete transcultural adaptation, particularly regarding factors. Further construct validation studies for the confirmatory and exploratory analysis of items will be helpful.

References

• ¹
  Aboujaoude E, Koran LM, Gamel N, Large MD. Potential markers for problematic internet use: a telephone survey of 2513 adults. CNS Spectr. 2006;11:750-5.
• ²
  Abreu CN, Karam RG, Goes DS, Spritzer DT. Dependência de internet e jogos eletrônicos: uma revisão. Rev Bras Psiquiatr. 2008;30:156-67.
• ³
  Stravogiannis A, Abreu CN. Internet addiction: a case report. Rev Bras Psiquiatr. 2009;31:78-9.
• ⁴
  Ko CH, Lin HC, Yang MJ, Jen M. Factors predictive for incidence and remission of internet addiction in young adolescents: a prospective study. Cyberpsychol Behav. 2007;10(4):545-55.
• ⁵
  Kima K, Ryub E, Chonb MY, Yeunb EJ, Choic SY, Seod JS, et al. Internet addiction in Korean adolescents and its relation to depression and suicidal ideation: a questionnaire survey. Int J Nurs Stud. 2006;43:185-92.
• ⁶
  Tao ZL, Liu Y. Is there a relationship between Internet dependence and eating disorders? A comparison study of Internet dependents and non-Internet dependents. Weight Eat Dis. 2009;14:77-83.
• ⁷
  Ha JH, Kim SY, BAe SC, Bae S, Kim H. Depression and internet addiction in adolescents. Psychopathology. 2007;40:424-30.
• ⁸
  Huang Z, Wang M, Qian M, Zhong J, Tao R. Chinese internet addiction inventory: developing a measure of problematic internet use for Chinese college students. Cyberpsychol Behav. 2007;10(6):805-11.
• ⁹
  Dowling NA, Quirk K. Screening for internet dependence: do the proposed diagnostic criteria differentiate normal from dependent internet use? Cyberpsychol Behav. 2009;12(1):21-7.
• ¹⁰
  Young KS, Abreu CN. Internet addiction: a handbook and guide to evaluation and treatment. NY: Wiley; 2010.
• ¹¹
  Abreu CN, Góes DS. Dependência de internet. In: Rangé B (Orgs.). Psicoterapias cognitivo-comportamentais. Porto Alegre: Artmed; 2011. p. 440-58.
• ¹²
  Young KS. Internet addiction: the emergence of a new clinical disorder. Cyberpsychol Behav. 1998;1(3):237-44.
• ¹³
  Block J. Issues for DSM-V: internet addiction. Am J Psychiatry. 2008;165:306-7.
• ¹⁴
  Huang Z, Wang M, Qian M, Zhong J, Tao R. Chinese Internet addiction inventory: developing a measure of problematic Internet use for Chinese college students. Cyberpsychol Behav. 2007;10(6):805-11.
• ¹⁵
  Caplan SE. Problematic Internet use and psychosocial well-being: develop­ment of a theory-based cognitive-behavioral measurement instrument. Comput Hum Behav. 2002;8(5):553-75.
• ¹⁶
  Clark DJ, Frith KH. The development and initial testing of the Internet Consequences Scales (ICONS). Comput Inform Nurs. 2005;23(5):285-91.
• ¹⁷
  Widyanto L, McMurran M. The psychometric properties of the internet addiction test. Cyberpsychology Behav. 2004;7(4):443-50.
• ¹⁸
  Khazaal Y, Billieux J, Thorens G, Khan R, Louati Y, Scarlatti E, et al. French validation of the internet addiction test. Cyberpsychology Behav. 2008;11(6):703-6.
• ¹⁹
  Giovanni Ferraro, Barbara Caci, Antonella D'amico, Marie Di Blasi. Internet addiction disorder: an Italian Study. Cyberpsychology Behav. 2007;10(2):170-5.
• ²⁰
  Reichenheim ME, Moraes CL. Operacionalização de adaptação transcultural de instrumentos de aferição usados em epidemiologia. Rev Saude Publica. 2007;41(4):665-73.
• ²¹
  Conti MA, Slater B, Latorre MRDO. Tradução, validade e reprodutibilidade da EEICA - Escala de Evaluación da Insatisfación Corporal para Adolescentes - no Brasil. Rev Saude Publica. 2009;43(3):515-24.
• ²²
  Conti MA, Latorre MRDO, Hearst N, Segurado A. Cross-cultural adaptation, validation and reliability of the body area scale for Brazilian adolescents. Cad Saude Publica. 2009;25(10):2179-86.
• ²³
  Blaya C, Kipper L, Heldt EIL, Ceitlin LH, Bond M. Versão em português do Defense Style Questionaire (DSQ-40) para avaliação dos mecanismos de defesa: um estudo preliminar. Rev Bras Psiquiatr. 2004;26(4):255-8.
• ²⁴
  Pesquisa Ibope. [citado em 17 maio 2010]. Disponível em: http://www.ibope.com.br/calandraWeb/servlet/CalandraRedirect?temp=2&temp2=3&proj=PortalIBOPE&pub=T&nome=pesquisa_home&db=caldb
  » link
• ²⁵
  Moraes CL, Hasselmann MH, Reichenheim ME. Adaptação transcultural para o português do instrumento "Revised Conflict Tactics Scales (CTS2)" utilizado para identificar violência entre casais. Cad Saude Publica. 2002;18(1):163-76.
• ²⁶
  Kachani A, Barbosa ALR, Brasiliano S, Cordas TA, Hochgraf PB, Conti MA. Tradução, adaptação transcultural para o português (Brasil) e validação de conteúdo da Body Checking Cognitions Scale (BCCS). Rev Psiq Clín. 2011;38(1):13-8. Disponível em: http://dx.doi.org/10.1590/S0101-60832011000100004
• ²⁷
  Teixeira PC, Hearst N, Matsudo SMM, Cordas TA, Conti MA. Adaptação transcultural: tradução e validação de conteúdo da versão brasileira do Commitment Exercise Scale. Rev Psiq Clín. 2011;38(1):24-8. Disponível em: http://dx.doi.org/10.1590/S0101-60832011000100006
• ²⁸
  Toledo E, Taragano R, Cordas TA, Abreu CN, Hearst N, Conti MA. Adaptação transcultural da Massachusetts General Hospital (MGH) Hairpulling Scale para o idioma português (Brasil). Rev Psiq Clín. Disponível em: 2011;38(5):178-83. http://dx.doi.org/10.1590/S0101-60832011000500002
• ²⁹
  Costello AB, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation 2005;11(10). Disponível em: http://pareonline.net/getvn.asp?v=10&n=7
• ³⁰
  Bland JM, Altman DG. Cronbach's alpha. Br Med J. 1997;314-572.

Evaluation of semantic equivalence and internal consistency of a Portuguese version of the Internet Addiction Test (IAT)

Maria Aparecida Conti^I; Adan Pelegrino Jardim^I; Norman Hearst^II; Táki Athanássios Cordás^I; Hermano Tavares^III; Cristiano Nabuco de Abreu^I

Publication Dates

• Publication in this collection
  28 June 2012
• Date of issue
  2012

History

• Received
  29 Nov 2011
• Accepted
  03 Apr 2012