Reproducibility of the Portuguese version of the PEDro Scale Reprodutibilidade da Escala de Qualidade

The objective of this study was to test the interrater reproducibility of the Portuguese version of the PEDro Scale. Seven physiotherapists rated the methodological quality of 50 reports of randomized controlled trials written in Portuguese indexed on the PEDro database. Each report was also rated using the English version of the PEDro Scale. Reproducibility was evaluated by comparing two separate ratings of reports written in Portuguese and comparing the Portuguese PEDro score with the English version of the scale. Kappa coefficients ranged from 0.53 to 1.00 for individual item and an intraclass correlation coefficient (ICC) of 0.82 for the total PEDro score was observed. The standard error of the measurement of the scale was 0.58. The Portuguese version of the scale was comparable with the English version, with an ICC of 0.78. The inter-rater reproducibility of the Brazilian Portuguese PEDro Scale is adequate and similar to the original English version. Evidence-Based Practice; Clinical Trials; Questionnaires The Physiotherapy Evidence Database (PEDro; http://www.pedro.org.au) is a free database of randomized controlled trials (RCTs), systematic reviews and clinical practice guidelines in physiotherapy 1. PEDro is considered to be one of the most comprehensive databases for indexing reports of RCTs that investigate the effects of physiotherapy interventions 2. To be included in PEDro a trial must satisfy five eligibility criteria: (1) the trial must involve comparison of at least two interventions; (2) at least one of the interventions evaluated must be currently part of physiotherapy practice; (3) the interventions should be applied to subjects who are representative of those to whom the intervention might be applied in the course of physiotherapy practice; (4) the trial should involve random allocation or intendedto-be-random allocation of subjects to interventions; (5) the study must be a full paper published in a peer reviewed journal 1. All trials indexed on PEDro are assessed for methodological quality and statistical reporting using the PEDro Scale 1 that considers the following criteria: (1) eligibility criteria and source of participants; (2) random allocation; (3) concealed allocation; (4) baseline comparability; (5) blinding of subjects; (6) blinding of therapists; (7) blinding of assessors; (8) measures of key outcomes from more than 85% of participants; (9) intention-to-treat analysis; (10) betweengroup statistical comparisons; (11) point measures and measures of variability. The scale is NOTA RESEARCH NOTE Shiwa SR et al. 2064 Cad. Saúde Pública, Rio de Janeiro, 27(10):2063-2067, out, 2011 scored out of 10 with one point being awarded for each of the itens (2) to (11) clearly satisfied and reported by the trial. Each trial is assessed by two independent raters. Any disagreements are arbitrated by a third rater. Reports of trials written in languages other than English are assessed by bilingual raters using the English version of the PEDro Scale. More than 4,300 searches of PEDro are performed everyday by physiotherapists worldwide, of which 15% are performed by professionals from Portuguese-speaking countries. Although the PEDro website was recently translated into Portuguese, the search and results pages, including the PEDro Scale score, are only available in English. As only a small proportion of the Brazilian population are fully proficient in English, it is likely that this limitation of the PEDro search function will restrict the use of PEDro by Portuguese-speaking physiotherapists. As a first step to making PEDro more accessible to physiotherapists that are not proficient in English, the PEDro Scale has been cross-culturally adapted into Portuguese by a Portuguese research group 3. This adaptation was performed following the recommendations from current guidelines 4. Some orthographical adjustments were made to the Portuguese version of the PEDro Scale to adapt it to Brazilian Portuguese. No semantic or grammatical modifications were necessary. The scale in Brazilian Portuguese can be accessed at: http://www.pedro.org.au/portuguese/down loads/pedro-scale/. To date, the reproducibility of the Brazilian Portuguese PEDro Scale has not been evaluated. The objective of this study was to test the reproducibility and parallel-form reproducibility of the Brazilian Portuguese version of the PEDro Scale compared to the original English version.

The objective of this study was to test the interrater reproducibility of the Portuguese version of the PEDro Scale.Seven physiotherapists rated the methodological quality of 50 reports of randomized controlled trials written in Portuguese indexed on the PEDro database.Each report was also rated using the English version of the PEDro Scale.Reproducibility was evaluated by comparing two separate ratings of reports written in Portuguese and comparing the Portuguese PEDro score with the English version of the scale.Kappa coefficients ranged from 0.53 to 1.00 for individual item and an intraclass correlation coefficient (ICC) of 0.82 for the total PEDro score was observed.The standard error of the measurement of the scale was 0.58.The Portuguese version of the scale was comparable with the English version, with an ICC of 0.78.The inter-rater reproducibility of the Brazilian Portuguese PEDro Scale is adequate and similar to the original English version.

Evidence-Based Practice; Clinical Trials; Questionnaires
The Physiotherapy Evidence Database (PEDro; http://www.pedro.org.au) is a free database of randomized controlled trials (RCTs), systematic reviews and clinical practice guidelines in physiotherapy 1 .PEDro is considered to be one of the most comprehensive databases for indexing reports of RCTs that investigate the effects of physiotherapy interventions 2 .To be included in PE-Dro a trial must satisfy five eligibility criteria: (1) the trial must involve comparison of at least two interventions ; (2) at least one of the interventions evaluated must be currently part of physiotherapy practice; (3) the interventions should be applied to subjects who are representative of those to whom the intervention might be applied in the course of physiotherapy practice; (4) the trial should involve random allocation or intendedto-be-random allocation of subjects to interventions; (5) the study must be a full paper published in a peer reviewed journal 1 .
All trials indexed on PEDro are assessed for methodological quality and statistical reporting using the PEDro Scale As a first step to making PEDro more accessible to physiotherapists that are not proficient in English, the PEDro Scale has been cross-culturally adapted into Portuguese by a Portuguese research group 3 .This adaptation was performed following the recommendations from current guidelines 4 .Some orthographical adjustments were made to the Portuguese version of the PEDro Scale to adapt it to Brazilian Portuguese.No semantic or grammatical modifications were necessary.The scale in Brazilian Portuguese can be accessed at: http://www.pedro.org.au/portuguese/downloads/pedro-scale/.To date, the reproducibility of the Brazilian Portuguese PEDro Scale has not been evaluated.
The objective of this study was to test the reproducibility and parallel-form reproducibility of the Brazilian Portuguese version of the PEDro Scale compared to the original English version.

Methods
Seven Brazilian physiotherapists participated in this study as independent raters.The raters underwent training to assess the methodological quality of RCTs using the Brazilian Portuguese version of the PEDro Scale through the online PEDro Scale Training Program (http://training.pedro.org.au/portuguese/index.html -this website is password protected).After training, each therapist had to pass an accuracy test to ensure that they were applying the scale to the same standard as other raters.
Fifty reports of RCTs written in Portuguese were used in this study derived from a universe of 65 RCTs indexed on the database.The cita-tion details and PEDro ratings for all Portuguese language trials indexed on PEDro were downloaded on 2 August 2010.Two sets of consensus ratings (individual items and total PEDro score) were generated by the Brazilian raters using the Brazilian Portuguese version of the PEDro Scale (Figure 1).To produce the first set, each trial was independently assessed by two raters and any disagreements were arbitrated by a third rater.This process was then repeated by three different raters to produce a second set of consensus ratings.
The reproducibility of the Brazilian Portuguese version was evaluated based on the definition from the guidelines for health status questionnaires: "the degree to which repeated measurements provide similar answers in stable conditions" 5 (p.36).Both reliability ("the degree to which evaluations could be distinguished from each other, despite measurement error") and agreement ("how close repeated measures are, expressed in the unit of the scale being tested") were investigated.The two sets of the Brazilian-Portuguese version consensus ratings were compared to evaluate test-retest reproducibility.The first set of Brazilian Portuguese consensus ratings was compared to the English version consensus ratings to evaluate parallel-form reproducibility.A sensitivity analysis was also performed to compare the second set of Brazilian Portuguese consensus ratings with those of the English PEDro Scale.
The following statistics were determined to evaluate the test-retest reproducibility of the Brazilian-Portuguese version of the PEDro Scale: 1) kappa coefficients for each item of the PEDro Scale (kappa estimates for an individual item are considered important so that PEDro users can critically appraise the reproducibility of each item from the PEDro Scale); 2) intraclass correlation coefficient type 1,1 (ICC 1,1 ) 6 and its 95% confidence intervals (95%CI) for the total PEDro score and to determine parallel-form reproducibility; 3) standard error of measurement (SEM) was calculated by dividing the standard deviation of the difference in the total PEDro scores by the square root of two 5 .

Results
The mean total PEDro score for the 50 reports of trials was 3.5 (SD = 1.4; range 1-7), indicating that the majority of the reports were of low methodological quality.Five of the PEDro Scale item were clearly satisfied in only 10% (or less) of the reports (Table 1).No PEDro Scale item was clearly satisfied in 90% (or more) of the reports."Moderate" to "almost perfect" 7 test-retest reproducibility was obtained for individual PEDro Scale itens assessed using the Brazilian Portuguese version of the PEDro Scale (Table 1).Kappa values of items 2, 5 and 6 were greater than 0.80 and these criteria were classified as having "almost perfect" 6 reliability.Kappa values of items 3, 7, 9, 10 and 11 ranged between 0.61 and 0.80 and these criteria were classified as having "substantial" reliability.Items 1, 4 and 8 were classified as having "moderate" reliability (kappa values between 0.41 and 0.60).In contrast, in the test-retest reliability of the English version of the PEDro Scale, three items were classified as having "moderate" reliability and eight items as having "substantial" reliability 1 .
The test-retest reproducibility of the total PE-Dro score generated using the Brazilian Portuguese version was classified as "excellent", with an ICC 1,1 of 0.82 (95%CI: 0.70-0.89).This is great-er than the value of 0.68 (95%CI: 0.57-0.76)contained for the English version of the PEDro Scale 1 .The SEM was 0.58 for the total PEDro score.This was classified as "good" agreement.
The parallel-form reproducibility between the Portuguese and English versions of the PEDro Scale was classified as "excellent".The ICC 1,1 of the English version compared to the first Brazilian Portuguese set was 0.78 (95%CI: 0.67-0.86).The sensitivity analysis using the second set of Portuguese ratings yielded similar results with an ICC 1,1 of 0.75 (95%CI: 0.59-0.85).

Discussion
It was observed that the Brazilian Portuguese PEDro Scale has good reproducibility.Parallelform reproducibility between the Portuguese and English versions of the PEDro Scale is excellent.As only orthographical differences between the Portuguese (Portugal) version of the PEDro Scale were made to create the Brazilian Portuguese version, the reproducibility estimates obtained can be applied to both Portuguese versions of the PEDro Scale.
The reliability estimates of individual itens of the Brazilian Portuguese version of the PEDro Scale were found to be similar to the estimates obtained by a previous study that tested the reliability of the English version of the scale 1 .Although it is interesting to note that kappa estimates were slightly higher for the Brazilian scale (Table 1), the factors involved in this comparison are complex and any such conclusion should be treated with caution because different RCTs were evaluated.
The reliability estimates of the total PEDro score of the English PEDro Scale (ICC = 0.68) 1 and Portuguese version (0.82) were similar.The SEM of the Portuguese version of the PEDro Scale reflects a low absolute measurement error.The "excellent" parallel-form of reproducibility between the Portuguese and English versions of the PEDro Scale (ICC = 0.78) suggest that either version may be used to generate PEDro ratings.
An important incidental finding of this study is the low methodological quality and the poor statistical reporting found in the reports that were assessed.The mean total PEDro score of re-

Figure 1 Flow
Figure 1Flow chart of the study process.
Contributors L. O. P. Costa participated in the definition and management of data collection, statistical analysis and in drafting of this article.S. R. Shiwa, A. D. Lopes, L. C. M. Costa, A. Moseley, L. C. Hespanhol Junior e T. O. Sato collaborated in defining the study, data collection and drafting of this article.R. Venâncio e C. Ruggero contributed to defining the study, data collection and final revision of the manuscript.

Table 1
Kappa coeffi cients and the proportion of reports with a yes response (base rate) for each of the individual item for ratings generated using the Portuguese (Brazil) and English versions of the PEDro Scale.*95%confi dence intervals (95%CI) are asymmetric because the upper-bound estimate was outside the range of the scale.In such cases the value was ad-ResumoO objetivo foi testar a reprodutibilidade da versão em português da Escala de Qualidade PEDro.Sete fisioterapeutas avaliaram a qualidade metodológica de 50 estudos controlados aleatorizados em português, indexados na base de dados PEDro.Cada artigo já possuía sua respectiva avaliação nessa base de dados, utilizando a versão em inglês da escala PEDro.Foi calculada a confiabilidade da escala, assim como foi comparada a pontuação total de consenso com a pontuação das avaliações utilizando a escala em inglês.Os coeficientes kappa variaram entre 0,53 e 1,00 para itens individuais, e um coeficiente de correlação intraclasse (CCI) de 0,82 foi obtido para a pontuação total.O erro-padrão de medida foi de 0,58 ponto.A versão em português da escala foi comparada com a versão em inglês e foi observado um CCI de 0,78.A reprodutibilidade da versão em língua portuguesa da Escala de Qualidade PEDro foi adequada e similar à versão em inglês.