Translation and validation of Convergence Insufficiency Symptom Survey (CISS) to Portuguese - psychometric results

Tradução e validação do CISS para a língua portuguesa - resultados psicométricos

Abstracts

Purpose:

Translate and adapt the Convergence Insuficiency Symptom Survey (CISS) questionnaire to the Portuguese language and culture and assess the psychometric properties of the translated questionnaire (CISSvp).

Methods:

The CISS questionnaire was adapted according to the methodology recommended by some authors. The process involved two translations and back-translations performed by independent evaluators, evaluation of these versions, preparation of a synthesis version and its pre-test. The final version (CISSvp) was applied in 70 patients (21.79 ± 2.42 years) students in higher education, and at two different times, by two observers, to assess its reliability.

Results:

The results showed good internal consistency of the CISSvp (Cronbach's alpha - α=0.893). The test re-test revealed an average of the differences between the first and second evaluation of 0.75 points (SD ± 3.53), which indicates a minimum bias between the two administrations. The interrater reliability assessed by intraclass correlation coefficient ranged from 0.880 to 0.952, revealing that the CISSvp represents an appropriate tool for measuring the visual discomfort associated with near vision tasks with a high level of reproducibility.

Conclusions:

The CISS Portuguese version, showed good psychometric properties and has been sown to be applicable to the Portuguese population, to quantify the visual discomfort associated with near vision, in higher education students.

Translations; Questionnaires; Language; Convergence, ocular; Asthenopia; Psychometrics

Objetivo:

Traduzir e adaptar transculturalmente o questionário Convergence Insuficiency Symptom Survey (CISS) para a língua portuguesa e avaliar as propriedades psicométricas do questionário traduzido (CISSvp).

Métodos:

O questionário CISS foi adaptado segundo metodologia recomendada por alguns autores. O processo envolveu duas traduções e retrotraduções realizadas por avaliadores independentes, avaliação das versões seguida da elaboração de uma versão síntese e pré-teste comentado. A versão final (CISSvp) foi aplicada em 70 indivíduos (21,79 ± 2,42 anos) estudantes do ensino superior, em dois momentos distintos e por dois observadores, para avaliar a confiabilidade do mesmo.

Resultados:

Os resultados evidenciaram boa consistência interna do CISSvp (alfa de Cronbach - α=0,893). O teste re-teste revelou uma media das diferenças entre a primeira e a segunda avaliação de 0,75 pontos (SD=±3,53) o que indica um viés mínimo entre as duas administrações. A confiabilidade inter-examinadores avaliada pelo coeficiente de correlação intraclasse variou de 0,880 e 0,952, revelando que o CISSvp representa uma ferramenta de medida do desconforto visual associado a tarefas em visão próxima com um nível de reprodutibilidade elevado.

Conclusões:

O CISS versão portuguesa apresentou boas propriedades psicométricas e demonstrou ser aplicável na população portuguesa, para quantificar a o desconforto visual associado à visão de perto, em estudantes do ensino superior.

Traduções; Questionários; Linguagem; Convergência ocular; Astenopia; Psicometria

INTRODUCTION

Surveys represent one of the most common types of research in the health, social, and human sciences. Particularly in the health sciences, the prospect of studying the evolution of a specific disorder and monitoring the efficacy of a specific treatment based on assessment surveys has been considered a great advantage, because they are easily accessible and inexpensive⁽¹⁾.

The increasing use of surveys to assess the quality of life, quantify symptoms, and evaluate attitudes and behaviors in studies from distinct countries warrants the translation and adaptation of previously validated questionnaires to different languages and cultures. To facilitate and expedite this task, guidelines on the translation process aiming to ensure the equivalence between the original and the translated versions have been published^(1-3).

The literature has several validated surveys that are useful for measuring the symptoms associated with visual discomfort, such as those developed by Conlon et al.⁽⁴⁾, the Convergence Insufficiency Symptom Survey (CISS)⁽⁵⁾, and the CDQOVL visual quality of life survey⁽⁶⁾.

The CISS survey addresses the most common issues regarding near vision problems and has been subjected to numerous studies to determine its validity, reproducibility, and effectiveness in identifying and monitoring the treatment evolution in individuals with convergence insufficiency^(5,7-9). Initially developed by the CIRS (Convergence Insufficiency and Reading Study) group, it was later modified to include near vision activities for adults (e.g., video games, hobbies, reading, and leisure activities)^(5,7,8). This questionnaire, validated in schoolchildren aged between 9 and 18 years⁽⁵⁾ and in young adults aged between 19 and 30 years⁽⁸⁾, is a reproducible and valid instrument, with high internal consistency and capable of addressing clinical changes during the treatment of convergence insufficiency⁽⁹⁾. In addition, it has proven to be an added value in the diagnosis of other nonstrabismic binocular disorders with symptoms similar to those of convergence insufficiency, particularly the accommodative insufficiency⁽¹⁰⁾. It has also contributed to the quantification of symptoms of visual discomfort in students⁽¹¹⁾, and the identification of individuals susceptible to visual complaints associated with visual discomfort from 3D displays⁽¹²⁾.

In Portugal, no easy-to-apply, self-administered instruments with good reproducibility rates are able to assess the visual discomfort associated with near vision tasks. Therefore, the present study aimed to adapt the CISS survey both transculturally and linguistically into Portuguese.

METHODS

The study was conducted in the Laboratory of Vision Sciences of the University of Beira Interior, with the approval of the Ethics Committee of the Faculty of Health Sciences of the same university (process CE-FCS-2012-027). The research comprised two phases: the phase of translation and transcultural adaptation to the target language and the phase of psychometric validation of the questionnaire.

The translation and transcultural adaptation process was based on the guidelines proposed by the Beaton and Gjersing teams^(2,3).

The psychometric validation of the translated survey was performed via interitem internal consistency analysis using Cronbach's alpha, and via temporal stability and reproducibility assessment using the test-retest analysis. These analyses are able to assess the validity and the reliability of the adapted instrument^(13,14). To achieve the desired results, the translated questionnaire was applied twice by two separate researchers. The interval between the applications of the two questionnaires was 1 week, which was sufficient to ensure that no significant changes would occur in the general behavior of the participants and that they would not remember the answers given in the previous questionnaire. This interval corresponds to the period necessary to analyze the reliability of measurements of the health status using the test-retest reliability test⁽¹⁴⁾.

Participants

The number of participants in the process of translation and adaptation of the survey (pretest) amounted to 20 students. Seventy students aged between 18 and 30 years participated in the psychometric validation phase. Of these, 30 were from the Faculty of Health Sciences and the remaining 40 were from the Department of Sciences and Engineering of the University of Beira Interior. All participants were duly informed about the aim of the study and signed an informed consent form.

Survey

The object of study was the Convergence Insufficiency Symptom Survey (CISS), which comprised 15 items and assessed the presence and frequency of symptoms typical of convergence insufficiency. In each question the subject indicates the frequency of each symptom on a Likert scale with scores ranging from 0 to 4, according to the following frequencies: never (0), infrequently (1), sometimes (2), often (3), and always (4). The points of all 15 items were added to determine the final score, which ranged from 0 to 60⁽¹¹⁾.

Procedures

Before the translation and validation, we requested the questionnaire in digital form and a written authorization from one of the authors of the original survey⁽¹⁾.

After obtaining the authorization and the approval of the ethics committee to conduct the study with students of the University of Beira Interior, the work instrument described below was performed.

Translation and transcultural adaptation process

To obtain a scientifically based translation and transcultural validation of the survey from its original language (English) into the target language (Portuguese), we adopted the methods proposed by Beaton (2000) and Gjersing (2010)^(2,3). The process involved three steps, from the initial translation to the final version of the CISS (designated CISSvp), as summarized in table 1.

Table 1
Summary of CISS (convergence insufficiency symptom survey) questionnaire translation and adaptation process to portuguese

Phase	Action	Result
Initial translation	Independent translation	Translated version T3
	1st meeting
Back-translation	Back-translation	Translated version T4
	2nd meeting
Final review	VersionT4 pre-test	Final version CISSvp
	3rd meeting

First step - Initial translation: two independent translations (designated T1 and T2) were performed by two bilingual translators working in completely different areas of expertise. A first assessment meeting was conducted, when a panel composed of two visual health professionals, one social sciences professor, one neurologist, and two independent translators analyzed T1 and T2 in depth, comparing them with the original questionnaire. This assessment resulted in the establishment of the third translated version of the questionnaire after consensus among all parties involved (T3 version).

Second step - Back-translation: The T3 version of the questionnaire translated into Portuguese was translated back into English by a professional translator, who did not know the original version, resulting in a back-translated version (Tr-t). A second assessment meeting ensued, with the presence of the same panel that attended the first meeting and the professional translator. In this meeting, the original questionnaire, the T3 version, and the Tr-t version, were analyzed, compared, and rated. Small changes were made to the syntax, resulting in the fourth version of the questionnaire (T4 version).

Third step - Final revision: in this phase a pretest was conducted through the assessment of the T4 version of the translated questionnaire by 20 university students to determine inconsistencies and interpret the content of the various items. At this point, the option "I do not understand the question" was added to the answer options for each item. In addition, the assessors were asked to comment on the questions that raised doubts. Following the analysis of the pretest results, the final format of the Portuguese version of the questionnaire was prepared (CISSvp).

Psychometric validation of the scale

Psychometric validation of the scale was performed according to the guidelines proposed in previous studies⁽¹⁵⁾. The following analyses were performed: interitem internal consistency, temporal stability, and interobserver reproducibility.

The analysis of internal consistency aimed to determine whether the questionnaire items were correlated with each other. The internal consistency coefficient provided an estimate of measurement reliability, on the basis of the assumption that items that assess the same constructs must be correlated.

Stability of the instrument, or test-retest reliability, was assessed using Spearman's coefficient to determine the correlation or the association strength between the two instances of the administration of the survey. The intraclass correlation coefficient (ICC) was used to determine the degree of concordance between the measurements taken by different respondents⁽¹⁶⁾.

STATISTICAL ANALYSIS

All statistical analyses were performed using the IBM SPSS Statistics software, version 20.

The reliability of a Likert-type scale was assessed using Cronbach's α coefficient. This coefficient assesses the internal consistency of a set of items, i.e., it evaluates the degree of coherence (relatedness), with an aim to determine whether those items measure the same parameter and whether they can be summed into a single score. The values >0.7 were considered acceptable⁽¹³⁾.

The reliability of the instrument was assessed using the following tests: Spearman's correlation coefficient, to determine the association of CISSvp survey score between the two assessment instances and to determine temporal stability, and the intraclass correlation coefficient (ICC), to evaluate the degree of concordance between the measurements taken by distinct assessors. The degree of concordance of ICC values was determined as proposed by Bland, i.e., acceptable: <0.4, good: 0.41-0.6, very good: 0.61-0.80, and excellent: 0.81-1.0⁽¹⁶⁾.

RESULTS

Survey translation and adaptation

During the first step of the translation and adaptation into Portuguese, difficulties emerged when translating and adapting questions 10 and 11. Question 10 was translated in a similar fashion by the two translators: "sente dor nos olhos." Question 11 was translated differently by the same translators: "sente os olhos doloridos" and "sente os olhos inflamados," and these versions raised controversies among the panel members. Therefore, a separate interview with 10 students was proposed, when they were asked to opine on the interpretation of the two questions. They concluded that the translation of question 11, "sente os olhos inflamados," was the most adequate.

The remaining steps of the process did not raise any controversy. In the third step of the process, the result of the pretest did not indicate any inconsistencies in the interpretation of the items of the translated survey. The overall result of this entire process, which is presented in Annex A, is the draft of the final Portuguese version of the CISS survey.

Psychometric validation of the survey

The answers of the survey administered to both groups of students, one group from the Department of Health Sciences (FCS) and the other from other departments of the University of Beira Interior, were analyzed to validate the translation of the instrument. The mean score differences between the two groups of students were analyzed using the Mann-Whitney test, which indicated that the survey score of FCS students and the score of students from other departments were not significantly different (Table 1).

The internal consistency of the scale was determined for the sample evaluated by FCS students and that evaluated by students from other departments, and for the total sample of respondents. This assessment indicated moderate to high reliability for all groups (Table 2).

Table 2
Descriptive statistics, test of differences and internal consistency of the Portuguese version of the CISS (convergence insufficiency symptom survey) questionnaire (CISSvp) score

Sample (n)	Age mean ± SD	Total score mean ± SD	Mann-Withney p-value	Cronbach's Alpha
Total sample (70)	21.79 ± 2.42	15.56 ± 8.86	-	0.893
CHSS (30)	21.33 ± 1.47	15.03 ± 8.38	0.656	0.901
Other (40)	21.13 ± 2.91	15.95 ± 9.20		0.892

• SD= standard deviation; CHSS= College of Health Sciences Students; Other= college of sciences and college of engineering students.

The analysis of the overall total internal consistency, and the internal consistency when one item was eliminated (Table 3), in both the test and retest, revealed that all items contributed to a higher consistency. However, the exclusion of item 6 neither decreased the overall consistency nor contributed to a significant increase in internal consistency. The Cronbach's alpha obtained in the retest (0.918) was slightly higher than that obtained in the test (0.893). Moreover, as this was a translation and adaptation of an instrument into another language, it would be useful to compare this data with those from other populations and cultures.

Table 3
Reliability: Internal consistency of the portuguese version of CISS (convergence insufficiency symptom survey) questionnaire (CISSvp)

	Correlation total corrected				Cronbach's Alpha if the item is excluded
Item		Test	Re-test		Test	Re-test
1		0.709	0.721		0.880	0.909
2		0.713	0.746		0.879	0.908
3		0.642	0.625		0.883	0.912
4		0.521	0.569		0.888	0.914
5		0.463	0.553		0.890	0.914
6		0.336	0.425		0.895	0.919
7		0.498	0.529		0.889	0.915
8		0.353	0.542		0.894	0.915
9		0.488	0.641		0.889	0.912
10		0.709	0.749		0.880	0.908
11		0.609	0.661		0.884	0.911
12		0.557	0.597		0.886	0.913
13		0.675	0.718		0.881	0.909
14		0.489	0.565		0.889	0.914
15		0.640	0.714		0.883	0.909
Total					0.893	0.918

CISS vp survey reliability analysis

The temporal stability was assessed via the test-retest reliability analysis, by interpreting the Spearman's correlation coefficient between the score of each item in each evaluation. Table 4 shows the Spearman's rho coefficient for each item and for the overall score.

Table 4
Spearman correlation coefficient between test and re-test of the CISSvp questionnaire (convergence insufficiency symptom survey - portuguese version)

Item	1	2	3	4	5	6	7	8
Rho	0.802	0.691	0.840	0.640	0.504	0.668	0.688	0.823
Item	9	10	11	12	13	14	15	Total score
Rho	0.685	0.697	0.782	0.638	0.719	0.565	0.686	0.910

The correlations between the answers from the first and the second survey assessments varied between 0.504 (item 5) and 0.844 (item 3). Using the scores proposed by Maroco, correlations were high for items 1, 3, 8, 11, and 13 (scores between 0.70 and 0.89), and moderate for items 2, 4, 6, 7, 9, 10, 12, 14, and 15 (scores between 0.40 and 0.69)⁽¹⁷⁾. The overall scores obtained on the two instances of the survey assessment exhibited an excellent correlation, which led us to conclude that CISSvp had a good temporal stability.

ICC between the two assessment instances was 0.924 (95% CI: 0.880 and 0.952). The mean difference between the survey's overall score of the first and second assessments was 0.75 ± 3.53 points, indicating minimum bias between the two evaluations (sign test: Z score, -1.162; p, 0.245).

DISCUSSION

The results of the present study indicate that the translation and adaptation of the CISS survey into Portuguese, which resulted in the CISSvp version, is a reliable outcome measurement instrument for university students. The high internal consistency of the answers obtained in the survey (α = 0.893) and the high intraclass correlation (ICC=0.924) indicate that CISSvp exhibited a high degree of reliability.

The translation and adaptation process was scientifically rigorous and followed the guidelines used in similar studies^(2,3). Moreover, the use of two independent translators was very useful, because it allowed the comparison and discussion of the two distinct versions and the preparation of a final translation, after initial discrepancies were resolved. The inclusion of members with distinct areas of expertise in the assessment panel allowed the comparison of different viewpoints from professionals in health and social sciences. The discussions that unfolded to solve those discrepancies and the search for a consensus were essential to the entire process of semantic equivalence. The pretest applied during step 3 of the process indicated that all participants understood the questions from the CISSvp survey.

In the psychometric validation of the scale, the sample of respondents was classified according to the University the students belonged to. The aim of this classification was to investigate whether familiarity with the terms used in the translated survey affected the way the questions were answered. The mean score of the survey was similar for both groups of students, 15.03 and 15.95, and the Mann-Whitney test indicated that the scores obtained for the two student groups were not significantly different. The scores of the interitem internal consistency analysis using Cronbach's alpha coefficient were 0.901 and 0.892, indicating that the internal consistency of CISSvp was high for both groups of students⁽¹³⁾. Using data from all participants, Cronbach's alpha was 0.893, and all items contributed to the increase in the overall internal consistency. These results indicate that the internal consistency of CISSvp was high and similar to the results obtained by Rouse during the validation of the original survey (CISS) using a sample of adult respondents⁽⁸⁾.

ICC was 0.924, which indicates an excellent reproducibility for CISSvp⁽¹⁶⁾. The mean difference (0.75 ± 3.528) and the analysis of variance between the two assessment instances (sign test Z score, -1.162; p, 0.245) suggest minimum bias between the two evaluations applied in a 1-week interval. These data are in line with the results of other studies in which CISS was used^(5,8).

CONCLUSION

The aim of this study was to translate the CISS survey into Portuguese (CISSvp). Both the translation and the psychometric characteristics of the survey application were very satisfactory. The outcome measurement instrument exhibited very good to excellent levels of internal reliability (Cronbach's alpha) and temporal stability (test-retest analysis), indicating that CISSvp is a reliable and reproducible instrument for university students.

• Funding: No specific financial support was available for this study.
• Study carried out at Laboratório de Ciências da Visão, University of Beira Interior.
• Projecto no CE-FCS-2012-027 - Faculdade Ciências da Saúde - UBI.

REFERENCES

• ¹
  Acquadro C, Conway K, Hareendran A, Aaronson N; European Regulatory Issues and Quality of Life Assessment (ERIQA) Group. Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008;11(3):509-21.
• ²
  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of crosscultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186-91.
• ³
  Gjersing L, Caplehorn, J R, Clausen T. Cross-cultural adaptation of research instruments: Language, setting, time and statistical considerations. BMC Med Res Methodol. 2010:10-3.
• ⁴
  Conlon E, Lovegrove W, Chekaluk E, Pattison P. Measuring visual discomfort. Vis Cogn. 1999;6:637-66.
• ⁵
  Borsting E, Rouse MW, Mitchel GL, Scheiman M, Cotter SA, Cooper J, Kulp MT, London R; Coonverge Insufficiency Treatment Trial Group. Validity and reliability of the revised convergence insufficiency symptom survey in children aged 9 to 18 years. Optom Vis Sci. 2003;80(12):832-8. Optom Vis Sci. 2004 Jul; 81(7):489; author reply 489-90. Optom Vis Sci. 2004 Jul; 81(7):489; author reply 489-90. Optom Vis Sci. 2004 Jul; 81(7):489; author reply 489-90. Comment in: Optom Vis Sci. 2004;81(7):489; author reply 489-90.
• ⁶
  Mozlin R. Quality-of-life outcomes assessment. J Optom Vis Dev. 1995;26:194-9.
• ⁷
  Borsting E, Rouse MW, De Land PN, Convergence Insufficiency and Reading Study (CIRS) group. Prospective comparison of convergence insufficiency and normal binocular children on CIRS symptom surveys. Optom Vis Sci. 1999;76(4):221-8.
• ⁸
  Rouse MW, Borsting E, Mitchel GL, Scheiman M, Cotter SA, Cooper J, Kulp MT, London R, Wensveen J; Convergence Insufficiency Treatment Trial Group. Validity and reliability of the revised convergence insufficiency symptom survey in adults. Ophthalmic Physiol Opt. 2004;24(5):384-90.
• ⁹
  The Convergence Insufficiency Treatment Trial (CITT) Study Group. The convergence insufficiency treatment trial: design, methods and baseline data. Ophthalmic Epidemiol. 2008;15(1):24-36.
• ¹⁰
  Marran, LF, De Land PN, Nguyen AL. Accommodative insufficiency is the primary source of symptoms in children diagnosed with convergence insufficiency. Optom Vis Sci. 2006;83(5):281-9. Comment in: Optom Vis Sci. 2006;83(11):857-8; author reply 858-9.
• ¹¹
  Borsting E, Chase C, Tosha C, Ridder WH 3rd. Longitudinal study of visual discomfort symptoms in college students. Optom Vis Sci. 2008;85(10):992-8.
• ¹²
  Lambooij MT, Ijsselsteijn WA, Fortuin M, Evans BJ, Heynderickx I. Susceptibility to visual discomfort of 3-D displays by visual performance measures. IEEE Trans Circuit Syst Video Technol. 2011;21(12):1913-23.
• ¹³
  Cortina JM. What is coefficient alpha? an examination of theory and applications. J Applied Psychol. 1993;78:98-104.
• ¹⁴
  Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. J Clin Epidemiol. 2003;56(8):730-5.
• ¹⁵
  Kimberlin CL, Winsterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm. 2008;65(23):2276-84.
• ¹⁶
  Bland JM, Altman DG. A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Comput Biol Med. 1990;20(5):337-40.
• ¹⁷
  Maroco J. Análise estatística com utilização do SPSS. 3ª ed. Lisboa: Edições Sílabo; 2007.

Publication Dates

• Publication in this collection
  Jan-Feb 2014

History

• Received
  02 Aug 2013
• Accepted
  25 Aug 2013