SciELO - Scientific Electronic Library Online

 
vol.22Congenital anomalies in Rio Grande do Sul State: a time series analysisSleep bruxism: population based study in people with 18 years or more in the city of Rio Grande, Brazil author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Revista Brasileira de Epidemiologia

Print version ISSN 1415-790XOn-line version ISSN 1980-5497

Rev. bras. epidemiol. vol.22  São Paulo  2019  Epub Apr 29, 2019

http://dx.doi.org/10.1590/1980-549720190037 

ORIGINAL ARTICLE

Validity and reliability of the Brazilian version of the Johns Hopkins Fall Risk Assessment Tool to assess the risk of falls

Maria Carmen MartinezI 
http://orcid.org/0000-0002-4427-5976

Viviane Ernesto IwamotoII 

Maria do Rosário Dias de Oliveira LatorreIII 
http://orcid.org/0000-0002-5189-3457

Adriana Moreira Noronha SimõesII 

Ana Paula de Sousa OliveiraIV 

Carlos Eduardo Alves CardosoII 

Ifigênia Augusta Braga MarquesII 

Patrícia VendramimV 

Paula Cristina LopesVI 

Thais Helena Saes de Sant’AnaVII 

IWAF Informática - São Paulo (SP), Brazil.

IINursing Services Management, Hospital Samaritano - São Paulo (SP), Brazil.

IIIDepartment of Epidemiology, School of Public Health, Universidade de São Paulo - São Paulo (SP), Brazil.

IVEpidemiology Center, Hospital Samaritano - São Paulo (SP), Brazil.

VDivision of Pulmonology, Heart Institute of the Hospital das Clínicas, School of Medicine, Universidade de São Paulo - São Paulo (SP), Brazil.

VIFlourishing, Love, Optimism & Wellness - São Paulo (SP), Brazil.

VIIDiagnostic Services, 9 de Julho Endoscopia Ltda. - São Paulo (SP), Brazil.

ABSTRACT:

Objective:

To evaluate the validity and reliability of the Johns Hopkins Fall Risk Assessment Tool (JH-FRAT), which assesses the risk of falls in hospitalized inpatients.

Method:

Study with 297 patients at a hospital in São Paulo, using retrospective data form 2014. Validity was assessed by accuracy (sensitivity, specificity, positive predictive value - PPV and negative predictive value - NPV) and discriminant analysis (comparison of patients with and without falls in relation to the scale items and comparison of previous risk situations in relation to the injury). The χ2 test and Fisher’s exact test were used. Reliability was assessed by reproducibility between methods and interobserver test-retest comparison in a subsample of 60 patients. We used the Kappa, quadratic weighted Kappa and PABAK statistics.

Results:

Sensitivity was 97.0%, specificity was 6%, PPV was 36.2% and NPV was 90.6%. Five of the eight items of the scale and the overall classification showed risk discrimination capability (p < 0.050). The risk of previous situations did not discriminate the injury resulting from the falls (p = 0.557). Reproducibility between methods was substantial (PABAK = 0.71). The interobserver reproducibility ranged between items (PABAK 0.25 to 1.00) and was substantial to the overall risk classification (PABAK = 0.71).

Conclusion:

JH-FRAT showed validity and reliability expected of a screening tool for risk of falls, and it can contribute to the implementation of fall management strategies in hospitals.

Keywords: Data accuracy; Reproducibility of results; Reliability and validity; Nursing methodology research; Accidental falls; Quality of health care

INTRODUCTION

Falls that occur with hospitalized patients are adverse events with negative repercussions for patients and institutions1,2,3,4,5, whose prevention should adopt evidence-based practices such as having the call bell within the patient’s reach, hospital bed in low position and treatment of the causes of mental confusion2,3,6,7,8. Among the recommended practices is the use of scales that identify risk conditions for the occurrence of falls, either in the form of specific risks or in risk stratification, directing interventions compatible with the identified needs2,3,4,7,8.

Among the instruments available in the international literature, two scales went through the cross-cultural adaptation process for use in Brazil, the Morse Fall Scale9 and the Johns Hopkins Fall Risk Assessment Tool (JH-FRAT)5; both did not report the measurement properties of the Brazilian version. This study chose to evaluate the measurement properties of the JH-FRAT scale as an instrument whose content’s validity was already demonstrated in the Brazilian version, showing good applicability in systematized protocols and simple operationalization5.

JH-FRAT was structured by professionals from Johns Hopkins Hospital and the Johns Hopkins University School of Nursing for integration into the institutional fall management program for hospitalized adult patients6,10,11. This scale was structured based on risk factors identified in the literature - providing a risk classification for directing preventive measures -, tested in patients and adjusted by a consensus group10. After two years of use in the institution, the acceptability and content of the scale were evaluated, focusing on the relationship between items, the scoring model and the clarity and relevance of the items, resulting in the definition of its final form and guaranteeing its content’s validity11.

The scale’s measurement properties were evaluated in patients hospitalized in South Korea12 and in the elderly in American nursing homes13, showing acceptable sensitivity and specificity12,13. These studies, however, present methodological limitations that make it difficult to generalize the results to other populations.

Considering that previous studies have shown that JH-FRAT has acceptable measurement properties10,11,12,13 and that there are no assessments of risk scales for falls in Brazil, this study aimed to analyze the measurement properties of JH-FRAT for assessing the risk of falls in hospitalized adult patients.

METHOD

The study was approved by the Research Ethics Committee of Hospital Samaritano and followed the principles of the Declaration of Helsinki. Due to the use of secondary data, the Informed Consent was not necessary. Authorization was obtained from the authors of JH-FRAT and of the Brazilian version of the Morse scale for use in this study.

The target population was composed of adult patients hospitalized in a high complexity private hospital in the city of São Paulo, which uses a protocol for the management of falls since 2005, with the implementation of JH-FRAT at the end of 2013. The protocol provides for the daily assessment of individual risks through JH-FRAT, with the adoption of preventive measures according to each patient’s risks and of universal precautionary measures, such as maintaining night lights activated, signaling risks and freeing the environment of obstacles. The evaluation was performed by nurses and filed in an electronic medical record. This study used data from 2014, after a period of training and routine consolidation by the nursing team.

This is a case-control study including patients from the adult hospitalization areas (clinical, surgical, oncology, gynecology and obstetrics and intensive care), with a wide variety of clinical conditions. We selected 100.0% of patients with reported falls in 2014 (99 cases) and 198 patients without reported falls (two controls for each case), resulting in a sample of 297 individuals. The controls selected were patients admitted to the room next to a patient with a reported fall event, in the same hospitalization sector and on the same date of the fall, in order to homogenize the characteristics of the care received. No other pairing criteria were used in order to preserve differences regarding other potential risk factors that are part of JH-FRAT’s operational structure. Some patients had more than one reported fall event, with each being assessed independently. For patients with falls, the last evaluation was considered within the 24 hours before the fall. For patients without falls, the same date for patients with falls was used. Individuals aged less than 18 years and/or whose hospitalization lasted for less than 24 hours were excluded.

Five researcher nurses collected the data retroactively from the electronic medical record, using a form composed of identification fields (chart number, attendance number, age, date of evaluation, hospitalization sector), for items from the JH-FRAT scale and Morse scale. Data were also obtained from the fall notification forms, filled out by nursing assistants.

The JH-FRAT scale is composed of eight aspects of fall risk11:

  1. previous risk-defining situations: complete paralysis/immobilization; history of two or more falls in the six months prior to admission; history of one fall during hospitalization; and patient considered as high-risk according to specific protocols. If the patient has complete paralysis/immobilization, they are classified as low-risk; if they present one of the other conditions, they are classified as high-risk. In these cases, the assessment is terminated and the rest of the scale is applied only to patients who are not classified under these conditions;

  2. age at admission: 0-3 points;

  3. history of one fall in the six months prior to admission: 0-5 points;

  4. intestinal and urinary eliminations (increased frequency, urgency, incontinence): 0-4 points;

  5. use of drugs with high risk for falls: 0-7 points;

  6. presence of care equipment that may compromise patient mobility: 0-3 points;

  7. impairment of mobility: 0-6 points;

  8. impairment in cognition: 0-7 points.

The sum of points from items 2 to 8 provides a score of 0 to 35 points, which is classified as low risk (0-5 points), moderate risk (6-13 points) and high risk (≥ 14 points)11. Item 1 of the scale is not included in the computation, only in the risk classification. Further details on the scale are described in the literature5,6,11.

The Morse scale14 showed satisfactory accuracy in the identification of patients at high risk of falls in international studies15,16. It is composed of six aspects (history of falls, secondary diagnoses, walking aid, intravenous therapy/devices, gait, and mental state), with scores varying from 0 to 30, providing a score ranging from 0 to 125 points categorized as low risk (0-24 points), medium risk (25-44 points) and high risk (≥ 45 points)9.

In this study, the validity was evaluated through the analysis of accuracy, criterion (concurrent analysis) and discriminant, and reliability was evaluated through the analysis of reproducibility.

Scales for the assessment of fall risk are considered diagnostic tests for the event risk (fall)17. The validity of the diagnostic tests is analyzed by means of accuracy studies, in which the results of a test applied to a population are compared to those obtained by a gold standard17,18,19. In this case, the test is the JH-FRAT and the gold standard is the occurrence of fall. Accuracy analysis verified sensitivity, specificity, positive (PPV) and negative (NPV) predictive values and positive (PLR) and negative (NLR) likelihood ratios Sensitivity evaluates the probability of a test being positive when the disease is present and consists of calculating the prevalence of exposure among events; specificity evaluates the probability of a test being negative when there is no disease and consists in calculating the prevalence of non-exposed among non-events; the PPV evaluates the probability of having the disease when the test was positive and consists of calculating the prevalence of events among those exposed; and the NPV evaluates the probability of not having the disease when the test was negative and consists of calculating the prevalence of non-events among non-exposed individuals. PLR informs the probability of a positive result occurring in a person with the disease in relation to the probability of occurrence in a person without the disease. NLR informs the likelihood of a negative result occurring in a person with the disease in relation to the probability of occurrence in a person without the disease3,20,21. The fall risk was considered as exposure (JH-FRAT scale), and the analyzes considered two scenarios:

  1. cutoff point > 6 points (moderate/high risk versus low risk);

  2. cutoff point > 13 points (high risk versus moderate/low risk) in JH-FRAT.

The criterion validity compares the analyzed scale to a valid standard that predicts the same performance22,23, being concurrent when the instrument and criterion are applied simultaneously22. The concurrent criterion validity investigated the associations between JH-FRAT and Morse risk classification using the χ2 test.

The discriminant analysis verifies whether a test is capable of identifying differences in a characteristic of subjects22. It was verified whether each of the eight items of the scale was able to discriminate the presence of the specific risk among patients with and without falls using the χ2 test.

Also in the discriminant analysis, it was verified whether previous risk-defining situations (item 1 of the scale) were associated with the severity of the injuries between the fall events, through the χ2 test. The severity of the injuries was classified, from an adaptation of Schwendimann et al.1, in:

  1. no injury;

  2. mild injury: minor cuts, mild bleeding, skin excoriation, pain, edema, mild traumatic brain injury (TBI), hyperemia and hematoma;

  3. moderate injury: excessive bleeding, laceration requiring suturing, moderate TBI, dislocation and sprain;

  4. severe injury: fracture, subdural hematoma, severe TBI, death.

Reproducibility analysis was performed using the interobserver test-retest comparison, with a random sub-sample of 60 patients. Two researchers independently assessed the same patient using the JH-FRAT scale, based on the same reference date, from data recorded in an electronic medical record. Quadratic weighted Kappa (PABAK) and prevalence-adjusted and bias-adjusted Kappa statistics were used. For the interpretation of the results, the criteria of Landis & Koch were used24.

A significance level of 5% was considered in all analyzes. The software SPSS (version 23), STATA (version 14) and WinPepi (version 11.65) were used.

RESULTS

Table 1 shows that the study population consisted of 297 patients, 50.8% females and 49.2% males; 55.9% aged 60 years or older; 63.0% married/had a partner; 46.1% had complete superior education; 14.1% in intensive care units; and 85.9% in open units. Patients with and without falls did not present differences as to gender (p = 0.870), age range (p = 0.200), marital status (0,159), educational level (0,189) and reason for hospitalization (p = 0.677). It was observed that 10.8% of the patients had low risk of falls, being 3.0% among those with falls and 14.6% among those without falls. Patients with high risk of falls had a higher percentage of occurrence of events (62.6%) than the others (44.9%) (p = 0.002).

Table 1. Descriptive statistics of the study population according to the occurrence of falls, private hospital, São Paulo, 2014. 

Variables With falls Without falls Total p-value*
% % %
Sex
Female 51 51.5 100 50.5 151 50.8 0.870
Male 48 48.5 98 49.5 146 49.2
Age group
< 50 20 20.2 66 33.3 86 29.0 0.200
50 to 59 17 17.2 28 14.1 45 15.2
60 to 69 17 17.2 30 15.2 47 15.8
70 to 79 26 26.3 47 23.7 73 24.6
80 and over 19 19.2 27 13.6 46 15.5
Marital status
Married/With partner 66 66.7 121 61.1 187 63.0 0.159
Single 13 13.1 36 18.2 49 16.5
Divorced/Separated/Widowed 20 20.2 34 17.2 54 18.2
Others/Not informed 0 0.0 7 3.5 7 2.4
Schooling
Complete Higher Education 44 44.4 93 47.0 137 46.1 0.189
Complete secondary/technical education 34 34.3 73 36.9 107 36.0
Up until primary education 18 18.2 20 10.1 38 12.8
Not informed 3 3.0 12 6.1 15 5.1
Referral from
Others 33 33.3 71 35.9 104 35.0 0.667
Emergency room 66 66.7 127 64.1 193 65.0
Sector
Low risk: 0-5 points 3 3.0 29 14.6 32 10.8 0.002
Moderate risk: 6-13 points 34 34.3 80 40.4 114 38.4
High risk: > 13 points 62 62.6 89 44.9 151 50.8
Total 99 100.0 198 100.0 297 100.0

*Association test using χ2.

The accuracy analysis of scenario 1 showed sensitivity = 97.0%; specificity = 14.6%; PPV = 36.2%; NPV = 90.6%; PLR = 1.14; and NLR = 4.83. Scenario 2 showed sensitivity = 62.6%; specificity = 55.1%; PPV = 41.1%; NPV = 74.7%; PLR = 1.39; and NLR = 1.47.

Table 2 shows that, in the concurrent criterion validity, there was a statistically significant association between the two scales (p < 0.001). The highest proportion of patients classified as low risk by JH-FRAT was classified as low risk by Morse (46.9%); the highest proportion of patients classified as moderate risk by JH-FRAT was classified as moderate risk by Morse (53.5%); and the highest proportion of patients classified as high risk by JH-FRAT was classified as high risk by Morse (57.6%). When moderate and high risk are grouped, the similarity in results between the two scales rises to 90.2%.

Table 2. Comparative analysis of the fall risk classification according to the Johns Hopkins Fall Risk Assessment Tool and Morse, private hospital, São Paulo, Brazil, 2014. 

Risk classification Low risk Medium risk High risk Total p-value*
% % % %
JH-FRAT scale Morse scale
Low risk: 0-5 points 15 46.9 14 43.8 3 9.4 32 100.0 < 0.001
Moderate risk: 6-13 points 12 10.5 61 53.5 41 36.0 114 100.0
High risk: > 13 points 14 9.3 50 33.1 87 57.6 151 100.0
Total 41 13.8 125 42.1 131 44.1 297 100.0

JH-FRAT: Johns Hopkins Fall Risk Assessment Tool; *association test using χ2.

The discriminant analysis showed which items of the scale had a statistically significant difference in the presence of the specific risk (Table 3). In the item “history of falls”, 18.2% of the patients with events had a history of one fall in the six months prior to hospitalization against 2.5% of those without a fall (p < 0.001). In the item “use of drugs with high risk for falls”, the category “2 or more drugs” was observed in 70.7% of patients with events and in 53.5% of those without events, and sedation in the last 24 hours occurred in 3.0% of patients with falls and in 9.1% of those without falls (p = 0.027). The results of the category “sedation in the last 24 hours” are different than expected, but the small number of cases of falls (3 patients) does not have statistical stability. The item “equipment impairing mobility” showed that ≥ 3 devices were present in 5.1% of patients with events and in 11.1% of those without events (p = 0.040). Again, the results may suffer bias from the small number of subjects with falls (five patients). In the item “impaired mobility”, patients with events had higher percentages than those without events in all the categories evaluated (p = 0.003). The same occurred in the item “cognition” (p < 0.001).

Table 3. Descriptive statistics between patients with and without fall event according to the presence of fall risks identified by the Johns Hopkins Fall Risk Assessment Tool, private hospital, São Paulo, Brazil, 2014. 

Component item of JH-FRAT With falls Without falls Total p-value*
% % %
Previous risk situations
No score/no classification 59 59.6 122 61.6 181 60.9 0.601
Low risk 1 1.0 5 2.5 6 2.0
High risk 39 39.4 71 35.9 110 37.0
Age group (years)
No score (< 60) 36 36.4 88 44.4 124 41.8 0.270
60-69 18 18.2 34 17.2 52 17.5
70-79 22 22.2 47 23.7 69 23.2
80 and over 23 23.2 29 14.6 52 17.5
History of falls
No score (no history) 81 81.8 193 97.5 274 92.3 < 0.001
One fall in the previous six months 18 18.2 5 2.5 23 7.7
Eliminations
No score (no problems) 53 53.5 109 55.1 162 54.5 0.970
Urgency/increased frequency or incontinence 45 45.5 87 43.9 132 44.4
Urgency/increased frequency and incontinence 1 1.0 2 1.0 3 1.0
Use of drugs with high risk for falls
No score 8 8.1 25 12.6 33 11.1 0.027
1 high-risk drug 18 18.2 49 24.7 67 22.6
2 or more high-risk drugs 70 70.7 106 53.5 176 59.3
Sedation in the last 24 hours 3 3.0 18 9.1 21 7.1
Equipment compromising mobility
No score (no equipment) 12 12.1 14 7.1 26 8.8 0.040
1 device 55 55.6 127 64.1 182 61.3
2 devices 27 27.3 35 17.7 62 20.9
3 or more devices 5 5.1 22 11.1 27 9.1
Compromised mobility**
No score (no alterations) 12 12.1 57 28.8 69 23.2 0.003
1 item scored 55 55.6 101 51.0 156 52.5
2 items scored 28 28.3 38 19.2 66 22.2
3 items scored 4 4.0 2 1.0 6 2.0
Cognition***
No score (no alterations) 53 53.5 152 76.8 205 69.0 < 0.001
1 or 2 points 19 19.2 30 15.2 49 16.5
3 to 5 points 15 15.2 14 7.1 29 9.8
6 or 7 points 12 12.1 2 1.0 14 4.7
Final risk classification
Low risk: 0-5 points 3 3.0 29 14.6 32 10.8 < 0.001
Moderate risk: 6-13 points 34 34.3 80 40.4 114 38.4
High risk: > 13 points 62 62.6 89 44.9 151 50.8
Total 99 100.0 198 100.0 297 100.0

JH-FRAT: Johns Hopkins Fall Risk Assessment Tool; *association test using χ2; **multiple choice question: needs assistance/supervision and/or gait and/or visual/auditory impairment; ***multiple choice question: altered perceptions and/or impulsivity and/or lack of understanding of limitations.

Also in the discriminant validity analysis, the previous situations that defined the risk (item 1 of the scale) were not associated with the injury(ies) due to the fall (p = 0.557).

Table 4 shows the results of the reliability analysis through interobserver reproducibility. Quadratic weighted Kappa values ranged from 0.15 to 1.00. There was improvement after adjustment for prevalence and bias, with PABAK ranging from 0.25 to 1.00. The item “prior risk situations” presented reasonable reproducibility (PABAK = 0.24). For the other items, substantial or almost perfect reproducibility (PABAK of 0.60 to 1.00) was observed, and the overall risk classification presented substantial reproducibility (PABAK = 0.71).

Table 4. Test-retest agreement for each item and for the final risk classification of the Johns Hopkins Fall Risk Assessment Tool, private hospital, São Paulo, Brazil, 2014 (N = 60). 

Item Quadratic Weighted Kappa 95%CI Quadratic Weighted PABAK
Previous risk situations 0.21 -0.04 - 0.46 0.24
Age at time of fall 1.00 -- 1.00
History of falls 0.20 -0.21 - 0.61 0.80
Eliminations 0.15 -0.09 - 0.39 0.77
Use of drugs with high risk for falls 0.23 -0.07 - 0.53 0.84
Equipment compromising mobility 0.37 0.07 - 0.67 0.84
Mobility 0.36 0.01 - 0.39 0.74
Cognition 0.34 0.03 - 0.65 0.87
Overall risk classification (total scale) 0.27 -0.01 - 0.56 0.71

95%CI: 95% confidence interval; PABAK: prevalence-adjusted and bias-adjusted Kappa.

DISCUSSION

The study analyzed whether the JH-FRAT scale is valid and reliable for assessing the risk of falls in adult inpatients. The results suggest that JH-FRAT has a better sensitivity at the score’s cutoff point > 6, grouping moderate and high-risk patients in the same category. An explanation for this is the existence of a managed protocol for falls in the study institution, where patients under certain conditions (risk of bleeding, bone fragility, among others) are classified as high risk according to item 1 of the scale (previous defining situations of risk). A modified version of JH-FRAT evaluating 107 outpatients aged 65 years and older in nursing homes in Southern California showed sensitivity of 100.0% at cutoff point > 6 and 86.2% at cutoff point > 1313. A JH-FRAT study with 356 patients older than 18 years in five general school hospitals in South Korea, with cutoff point > 13 points, showed a sensitivity of 62.0%12.

JH-FRAT showed better specificity at the cutoff point > 13, separating high-risk patients from the others. Sensitivity and specificity are inversely proportional19,21 and sensitivity was high. Considering that the systematized actions prevented the occurrence of falls, there was an increase in the number of false positives (patients at risk, but without falls), with consequent decrease in specificity. The diagnosis of a condition and the exclusion of false positives are defined by confirmatory tests that must have high specificity, but are subsequent to screening exams21. The study in southern California showed specificity of 0.0% at the cutoff point > 6 and 49.3% at the cutoff point > 13 points13. The study in South Korea, with cutoff point > 13 points, showed specificity of 69.5%12.

The PPVs show that 63.8% (in the cutoff point > 6) and 58.9% (in cutoff point > 13) of patients with a positive risk test received unnecessary preventive care for falls. This interpretation must be cautious. The increased prevalence of the event determines the increase in the PPV and the decrease in the VPN, and both depend on the context in which the test is applied16,18,20. Considering that the effect of the “treated” population reduces the occurrence of falls among those at risk, bias can occur with decreasing PPV and increasing NPV. The study in southern California showed PPV of 39.7% and NPV of 92.9% at cutoff point > 13 points13. The study in South Korea, with cutoff point > 13 points, showed PPV of 33.6% and NPV of 86.0%12.

LR offers an odds ratio measure, combining sensitivity and specificity in the same indicator20. The results were greater than 1.00 and thus contribute to the identification of the risk20, especially the NLR of scenario 1, in which patients without falls had 4.83 times the chance of obtaining a low risk classification when compared to the others.

For a screening test, the most important feature is sensitivity, with a high rate of true-positive and a low false-negative rate20,21. JH-FRAT, which an instrument for risk screening, showed high sensitivity.

The analysis of the validity of concurrent criteria showed equivalences between the risk classification of the two scales (JH-FRAT and Morse), with better performance when the moderate and high risks are grouped, suggesting that JH-FRAT has greater validity when separating the low-risk patients from the others.

Five of the eight JH-FRAT items and the overall risk score discriminated risk, with higher prevalence among patients with falls than among those without falls, as expected.

The previous situations that defined the risk (item 1 of the scale) were not able to discriminate the type of injury resulting from the fall. These results may be biased due to the institutional protocol, reducing both the occurrence and severity of falls.

The prevalence of patients classified as moderate/high risk, by both JH-FRAT and Morse, was 90.2%. A paradox in the Kappa reproducibility analyzes can be induced by a high prevalence of the analyzed condition and can be adjusted by the PABAK statistic25. Reproducibility between tests showed substantial agreement after adjustment for this statistic.

The interobserver test-retest measures showed substantial or near-perfect agreement after the use of the PABAK statistic on all items, except in the “prior risk situations”. The results obtained by quadratic weighted Kappa showed improvement after correction by the PABAK statistic, pointing to the already mentioned paradox. The study in southern California also showed substantial or near-perfect agreement in most items and in the overall classification (cutoff point ≥ 14)13.

This study has some limitations. The most important is the fact that it included a “treated” population, since preventive interventions reduce the occurrence and severity of events, and may alter the outcomes and accuracy of a risk assessment scale26,27. Another limitation refers to the use of data collected retrospectively, a type of collection that is affected by the quality of the available data28. Lastly, the relatively small number of falls assessed may have generated some bias.

Falls are events that are sensitive to care practices and can be prevented through systematized approaches in institutional programs2,3,4,5,15,26. The best results are obtained by combining multiple interventions, including individualized risk assessment and universal precautions4.8. The use of multifaceted interventions can reduce the occurrence of falls by up to 18.0%, but this reduction may vary depending on the quantity and type of actions2. The results showed JH-FRAT as valid and reliable in terms of accuracy, discriminant capacity and agreement on reproducibility, confirming it as a tool for risk assessment of falls and bringing contributions to professionals and institutions interested in the use of evidence-based strategies for the management of falls in hospitals.

There are few studies on the validity and reliability of JH-FRAT, and the present study is the first in Brazil. It is advisable that new studies be carried out with larger groups of hospitalized patients, preferably with a prospective design. In the absence of ethical conflicts, it is desirable to carry out studies on “untreated” populations and to compare results before and after the implementation of preventive protocols.

CONCLUSIONS

The study suggests that the Brazilian version of the JH-FRAT scale has satisfactory validity and reliability in terms of accuracy, criterion, discriminant capacity and reproducibility. It can be used in the evaluation of the fall risk in hospitalized patients, directing preventive actions. This is the first study in Brazil to evaluate JH-FRAT and has limitations, and new studies are desirable.

ACKNOWLEDGEMENTS

The authors would like to thank Hospital Samaritano de São Paulo for their support in the development of the study through the provision of equipment, materials and human resources, without which research would not be possible.

REFERENCES

1. Schwendimann R, Bühler H, De Geest S, Milisen K. Falls and consequent injuries in hospitalized patients: effects of an interdisciplinary falls prevention program. BMC Health Serv Res 2006; 6: 69. https://dx.doi.org/10.1186%2F1472-6963-6-69Links ]

2. National Health Services. National Patient Safety Agency. Slips, trips and falls in hospital: The third report from the Patient Safety Observatory. London: National Health Services; 2007. [ Links ]

3. Oliver D, Healy F. Falls risk prediction tools for hospital inpatients: do they work? Nurs Times 2009; 105(7): 18-21. [ Links ]

4. Cumbler EU, Simpson JR, Rosenthal LD, Likosky DJ. Inpatient falls: defining the problem and identifying possible solutions. Part I: An evidence-based review. Neurohospitalist 2013; 3(3): 135-43. https://doi.org/10.1177/1941874412470665Links ]

5. Martinez MC, Iwamoto VE, Latorre MRDO, Noronha AM, Oliveira APS, Cardoso CEA, et al. Transcultural adaptation of the Johns Hopkins Fall Risk Assessment Tool. Rev Latino-Am Enfermagem 2016; 24: e2783. http://dx.doi.org/10.1590/1518-8345.1158.2783Links ]

6. Johns Hopkins Hospital. Fall risk assessment, prevention and management, adult. In: Johns Hopkins Hospital, editor. Nursing practice and organization manual: clinical protocols and procedures. Baltimore: Johns Hopkins Hospital; 2011. v. 2. [ Links ]

7. Martinez MC, Iwamoto VE, Noronha AM, Oliveira APS, Sant'Ana THS, Cardoso CEA. Avaliação do risco de quedas em pacientes internados: por que realizar e como conduzir? Revista ACRED 2016; 6(11): 136-45. [ Links ]

8. Agency for Healthcare Research and Quality. Preventing falls in hospitals: a toolkit for improving quality of care. Rockvilee: Agency for Healthcare Research and Quality; 2013. [ Links ]

9. Urbanetto JS, Creutzberg M, Franz F, Ojeda BS, Gustavo AS, Bittencourt HR, et al. Morse fall scale: translation and transcultural adaptation for the Portuguese language. Rev Esc Enferm USP 2013; 47(3): 569-75. http://dx.doi.org/10.1590/S0080-623420130000300007Links ]

10. Poe SS, Cvach MM, Gartrell DG, Radzik BR, Joy TL. An Evidence-based Approach to Fall Risk Assessment, Prevention and Management: Lessons Learned. J Nurs Care Qual 2005; 20(2): 107-16. [ Links ]

11. Poe SS, Cvach MM, Dawson PB, Straus H, Hill EE. The Johns Hopkins Fall Risk assessment Tool: Postimplementation Evaluation. J Nurs Care Qual 2007; 22(4): 293-8. https://doi.org/10.1097/01.NCQ.0000290408.74027.39Links ]

12. Kim KS, Kim JA, Choi YK, Kim YJ, Park MH, Kim HY, et al. A Comparative Study on the Validity of Fall Risk assessment Scales in Korean Hospitals. Asian Nurs Res 2011; 5(1): 28-37. https://doi.org/10.1016/S1976-1317(11)60011-XLinks ]

13. Hnizdo S, Archuleta RA, Taylor B, Kim SC. Validity and reliability of the modified John Hopkins Fall Risk Assessment Tool for elderly patients in home health care. Geriatr Nurs 2013; 34(5): 423-7. https://doi.org/10.1016/j.gerinurse.2013.05.011Links ]

14. Morse JM. Preventing patient falls. California: Thousand Oaks; 1997. [ Links ]

15. Baek S, Piao J, Jin Y, Lee SM. Validity of the Morse Fall Scale implemented in an electronic medical record system. J Clin Nurs 2014; 23(17-18): 2434-40. https://doi.org/10.1111/jocn.12359Links ]

16. Costa-Dias MJM, Martins T, Araújo F. Study of the cut-off point of the Morse Fall Scale (MFS). Rev Enf Ref 2014; serIV(1): 65-74. http://dx.doi.org/10.12707/RIII13101Links ]

17. Aranda-Gallardo M, Morales-Asencio JM, Canca-Sanchez JC, Barrero-Sojo S, Perez-Jimenez C, Morales-Fernandez A, et al. Instruments for assessing the risk of falls in acute hospitalized patients: a systematic review and meta-analysis. BMC Health Serv Res 2013; 13: 122. https://dx.doi.org/10.1186%2F1472-6963-13-122Links ]

18. Glaros AG, Kline RB. Understanding the accuracy of tests with cutting scores: the sensitivity, specificity, and predictive value model. J Clin Psychol 1988; 44(6): 1013-23. [ Links ]

19. Parikh R, Mathai A, Parikh S, Sekhar GC, Thomas R. Understanding and using sensitivity, specificity and predictive values. Indian J Ophthalmol 2008; 56(1): 45-50. [ Links ]

20. Streiner DL. Diagnosing Tests: Using and Misusing Diagnostic and Screening Tests. J Personal Assess 2003; 81(3): 209-9. https://doi.org/10.1207/S15327752JPA8103_03Links ]

21. Brasil. Ministério da Saúde. Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Parte I - Introdução a conceitos teóricos. In: Brasil. Ministério da Saúde. Secretaria de Atenção à Saúde. Departamento de Atenção Básica, editor. Série A. Normas e Manuais Técnicos: Cadernos de Atenção Primária: Rastreamento. Brasília: Ministério da Saúde; 2010. n. 29. p. 17-23. [ Links ]

22. Pasquali L. Validade dos testes psicológicos: será possível reencontrar o caminho? Psic Teor e Pesq 2007; 23(n. esp.):99-107. http://dx.doi.org/10.1590/S0102-37722007000500019Links ]

23. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. COSMIN checklist manual. COSMIN. Amsterdam: University Medical Center, Department of Epidemiology and Biostatistics, Institute for Health and Care Research; 2012. [ Links ]

24. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1): 159-74. [ Links ]

25. Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85(3): 257-68. [ Links ]

26. Oliver D, Daly F, Martin FC, McMurdo MET. Risck factors and risk assessment tools for falls in hospital in-patients: a systematic review. Age Ageing 2004; 33(2): 122-30. https://doi.org/10.1093/ageing/afh017Links ]

27. Milisen K, Staelens N, Schwendimann R, De Paepe L, Verhaeghe J, Braes T, et al. Fall Prediction in Inpatients by Bedside Nurses Using the St. Thomas's Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) Instrument: A Multicenter Study. J Am Geriatr Soc 2007; 55(5): 725-33. https://doi.org/10.1111/j.1532-5415.2007.01151.xLinks ]

28. Dawson-Saunder B, Trapp RG. Reading the medical literature. In: Dawson-Saunder B, Trapp RG, editores. Basic & Clinical Biostatistics. Connecticut: Appleton & Lange; 1994. p. 268-88. [ Links ]

Financial support: none.

7Identification/approval number from the Research Ethics Committee: 678.566 from 05/27/2014 from the Research Ethics Committee of the Sociedade Hospital Samaritano de São Paulo.

Received: May 25, 2017; Revised: February 24, 2018; Accepted: May 15, 2018

Correspondence author: Viviane Ernesto Iwamoto. Gerência de Serviços de Enfermagem. Hospital Samaritano. Rua Conselheiro Brotero, 1.486, Higienópolis, CEP: 01232-010, São Paulo, SP, Brasil. E-mail: viviane.iwamoto@amaritano.org.br

Conflict of interests: nothing to declare

Author’s Contributions: Maria Carmen Martinez: coordination of the research project, training of the work team, statistical analysis of the data, discussion of the results and drafting of the text. Viviane Ernesto Iwamoto: coordination of the research project, training of the work team, supervision of data collection, discussion of results and drafting of the text. Maria do Rosário Dias de Oliveira Latorre: participated in the elaboration of the research project, the definition of statistical methods, the discussion of the results and in the drafting of the text. Ana Paula de Sousa Oliveira: participated in the elaboration of the research project, preparation and consolidation of databases, data collection and analysis, discussion of the results and drafting of the text. Adriana Moreira Noronha, Carlos Eduardo Alves Cardoso, Ifigênia Augusta Braga Marques, Patrícia Vendramim, Paula Cristina Lopes, Thais Helena Saes de Sant’ Ana: participated in the elaboration of the research project, data collection, discussion of the results and drafting of the text.

Creative Commons License Este é um artigo publicado em acesso aberto sob uma licença Creative Commons