Recommendations for elaboration, transcultural adaptation and validation process of tests in Speech, Hearing and Language Pathology

Pernambuco, Leandro; Espelt, Albert; Magalhães Junior, Hipólito Virgílio; Lima, Kenio Costa de

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Brief Communication • CoDAS 29 (3) • 2017 • https://doi.org/10.1590/2317-1782/20172016217 copy

Recommendations for elaboration, transcultural adaptation and validation process of tests in Speech, Hearing and Language Pathology

Authorship SCIMAGO INSTITUTIONS RANKINGS

ABSTRACT

Objective

to present a guide with recommendations for translation, adaptation, elaboration and process of validation of tests in Speech and Language Pathology.

Methods

the recommendations were based on international guidelines with a focus on the elaboration, translation, cross-cultural adaptation and validation process of tests.

Results

the recommendations were grouped into two Charts, one of them with procedures for translation and transcultural adaptation and the other for obtaining evidence of validity, reliability and measures of accuracy of the tests.

Conclusion

a guide with norms for the organization and systematization of the process of elaboration, translation, cross-cultural adaptation and validation process of tests in Speech and Language Pathology was created.

Keywords
Validation Studies; Validity of Tests; Speech, Language and Hearing Sciences; Psychometrics; Evaluation

Step

Procedures

a) Initial Guidelines

a1) Obtainment of permission from the authors of the original test.
a2) Formation of a committee of specialists (authors of the new version and other specialists + at least one author of the original version, if possible) to debate the concepts associated with the test to be adapted, considering the characteristics of the population and target culture. Participants should be aware of the test objectives.

b) Development of the test

b1) Translation: Two trained translators, native speakers of the target language and fluent in the source culture and language, independently translate the test, taking into account conceptual equivalence and avoiding literal translation. It is recommended that both translators do not know the test and that they are not specialists in the outcome investigated by the test so that the representativity of the common usage of the target language is maintained.
b2) Synthesis of the translations: undertaken in a collaborative manner^* * In the collaborative evaluations, assessment strategies can be adopted in relation to the equivalencies utilizing, for example, a visual analogical scale or scales of the Likert type. The calculation of the Content Validity Index for Scales (CVI-S) and for Item (CVI-I)(9) helps at this stage as a quantitative indicator of agreement between the evaluators , preferably by the same committee mentioned in procedure a2. The committee will construct a single version from the comparison of the translations and evaluation of semantic, idiomatic, conceptual, linguistic and contextual discrepancies.
b3) Applicability of the synthesis of translations/operational equivalence: the adequacy, structure and application of the items should be verified in a real-life context. There is no explicit recommendation for the sample size at this stage. The formation of representative strata of the target population, with at least ten individuals in each strata, is suggested. A cognitive interview is performed to verify if the population understands the test items. If the test is a questionnaire, the paraphrasing strategy is recommended, in which the interviewer asks the questions and requests the interviewee to immediately repeat them back. The interviewer should also observe possible operational difficulties (for example: long application time; excessive number of questions), non-verbal reactions of the interviewee (for example: boredom; facial expressions) and suggestions from the interviewee to improve the understanding of the item.
The committee mentioned in procedure b2 will meet once again if changes to the test are necessary. The test will be reapplied with new strata and this procedure will be repeated until the test is applied without any difficulty and with comprehension.
b4) Back-translation or reverse translation: the version obtained in procedure b3 will be translated into the source language to evaluate if the items reflect the content of the original version. The version in the target language will be sent to at least two trained translators who do not know the test and who are native speakers of the source language and fluent in the target language and culture.
b5) Synthesis of the back-translated versions: undertaken in a collaborative manner^* * In the collaborative evaluations, assessment strategies can be adopted in relation to the equivalencies utilizing, for example, a visual analogical scale or scales of the Likert type. The calculation of the Content Validity Index for Scales (CVI-S) and for Item (CVI-I)(9) helps at this stage as a quantitative indicator of agreement between the evaluators , preferably by the same committee mentioned in procedure a2. The committee will compare the retranslations to the original test and evaluate the semantic, idiomatic, conceptual, linguistic and contextual discrepancies.
During this step, it is recommended that the authors of the original version give their opinion in terms of the retranslated version.
b6) Final synthesis: undertaken in a collaborative manner^* * In the collaborative evaluations, assessment strategies can be adopted in relation to the equivalencies utilizing, for example, a visual analogical scale or scales of the Likert type. The calculation of the Content Validity Index for Scales (CVI-S) and for Item (CVI-I)(9) helps at this stage as a quantitative indicator of agreement between the evaluators , preferably by the same committee mentioned in procedure a2. The committee will compare the original and final versions in the target language in terms of semantic, idiomatic, conceptual, linguistic, experiential and contextual equivalences.

Step	Procedure
*Evidence of validity based on test content*	Content refers to the themes, writing, formatting, tasks or questions in the test, as well as instructions for the procedures necessary to administer and mark it⁽¹1 AERA: American Educational Research Association, APA: American Psychological Association, NCME: National Council on Measurement in Education. Standards for educational and psychological testing. New York: AERA; 2014.^,³3 Abad FJ, Olea J, Ponsonda V, Garcia C. Measurement in social sciences and health. Madrid: Sintesis; 2011.⁾. Some strategies to elaborate tests are: extensive literature review; empirical experience of the researchers with the outcome of interest; interviews with key informants; holding a panel with specialists and the target population; and consultation with the target population using focus groups. In the elaboration of the items, syntactic and semantic aspects, which contribute to the clarity, pertinence, coherence and scope of the items, as well as operational aspects should be considered. The representativity and relevance of the items in terms of the outcome should be evaluated by a committee of at least ten evaluators with expertise in the theme of the test⁽²2 Streiner DL, Norman GL. Health measurement scales: a practical guide to their development and use. 4th ed. New York: Oxford University Press; 2008.^,³3 Abad FJ, Olea J, Ponsonda V, Garcia C. Measurement in social sciences and health. Madrid: Sintesis; 2011.⁾. The use by the evaluators of a visual analogical scale or scales of the Likert type to evaluate the test items is recommended. The Content Validity Index for Scales (CVI-S) and for Item (CVI-I)⁽⁹9 Polit DF, Beck CT. The content validity index: are you sure you know what’s being reprted? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97. PMid:16977646. http://dx.doi.org/10.1002/nur.20147. http://dx.doi.org/10.1002/nur.20147... ⁾ for the presentation of the quantitative indicator of agreement between the evaluators is calculated.
*Evidence of validity based on the response processes*	The response processes result from the observations or judgments regarding the behavior or performance of different strata of the target population during the application of the test. During this step, we seek to understand which psychological, cognitive and social processes are involved in the application of the test. The procedures are similar to those described in b3 in Chart 1.
*Evidence of validity based on internal consistency*	The extent of the relation between the test items and the outcome, judged from the application of the test in a sample of the target population, is verified. If the internal dimensions of the test are theoretically defined by the authors, a confirmatory factorial analysis (CFA) should be carried out. If the authors need to investigate the possibility of cutting items or the existence of different dimensions of the test, principal component analysis (PCA) or factorial exploratory analysis (FEA), with the posterior CFA to validate the model found, should be undertaken. If the researcher follows the Item Response Theory (IRT) to investigate the latent traces, they can perform this analysis after the CFA. Each one of the analyses mentioned requires a different sample. Other parameters generally adopted are the corrected item-total correlation and the inter-item correlation with values above 0.3⁽²2 Streiner DL, Norman GL. Health measurement scales: a practical guide to their development and use. 4th ed. New York: Oxford University Press; 2008.⁾. Another characteristic investigated during this phase is the Differential Functioning of Items (DFI), which verifies if variables, either individually or in group, place one or more categories at an advantage or disadvantage in relation to the other/s⁽⁴4 Espelt A, Viladrich C, Doval E, Aliaga J, García-Rueda R, Tárrega S. Uso equitativo de tests en ciencias de la salud. Gac Sanit. 2014;28(5):408-10. PMid:24928357. http://dx.doi.org/10.1016/j.gaceta.2014.05.001. http://dx.doi.org/10.1016/j.gaceta.2014.... ⁾.
*Evidence of validity based on the relation to other variables*	The extent to which the relations between the instrument and the other external variables are consistent with the outcome is verified. 1) Convergent Validity: analysis of the relations between the test result and other measures which evaluate the same or similar outcomes; 2) Discriminant Validity: the test is evaluated in terms of its capacity to differentiate distinct groups from the target population or compare it to other measures with different outcomes; 3) Criterion Validity: the criteria are an operationally distinct attribute or result of the outcome of the test and are established by way of hypotheses formulated by the researchers. In the concurrent validity, the result of the test is compared to the outcomes of other tests administered in parallel and, in the predictive validity, the test result in relation to the condition of the evaluated individual in the future or the result of subsequent evaluations is evaluated.
*Reliability/ Precision*	The test should be administered at two distinct times, generally between 7 to 14 days. This period is variable according to the characteristics of the outcome, especially its stability over time. The test should be applied by the same researcher and distinct researchers at both times. The Intraclass Coefficient Correlation (ICC) or Kappa index to verify inter-evaluator and intra-evaluator agreement will be calculated. Internal consistency is obtained using the values of Cronbach’s alpha, measuring error, item-total correlation and inter-item correlation.
*Test equity*	Specialists should assess whether the result previously obtained in the DFI analysis is reproduced in a just manner and guarantee that the results in the test are not influenced by possible disadvantages caused by individual variables.
*Accuracy*	To obtain the indicators for sensitivity, specificity, positive and negative predictive value and positive and negative likelihood ratio, one test result will be compared to the other, taking into account the standard of reference. The cutoff point of the instrument and its discriminatory power in discriminating people with and without the outcome will be obtained using the ROC curve (receiver operator characteristic curve).
*Evidence of validity based on test consequences*	Characterized by the accumulation of evidence related to the use and efficacy of the test. Includes analyses of the responsiveness of the test to certain interventions and takes into account subsequent studies for application and reproduction of the test in practical contexts and related to the outcome.

Sociedade Brasileira de Fonoaudiologia Al. Jaú, 684, 7º andar, 01420-002 São Paulo - SP Brasil, Tel./Fax 55 11 - 3873-4211 - São Paulo - SP - Brazil
E-mail: revista@codas.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Conflict of interests: nothing to declare.