Reliability between Cardiovascular Risk Assessment Tools: A Pilot Study

Cardiovascular disease (CVD) is a major cause of morbidity and mortality and cause of 17.1 million deaths worldwide, which corresponds to 45% of deaths for chronic noncommunicable diseases.1 In Brazil, CVD is responsible for approximately 20% of deaths in people older than 30 years, and in 2015 it represented an estimated total cost of BR 37.1 billion.2,3 Therefore, CVD prevention is crucial in clinical practice, and identifying asymptomatic subjects at high risk is essential for an effective prevention.4,5 To meet this demand, cardiovascular risk assessment tools, and risk scores, including the Framingham risk score (FRS), have been the most widely used worldwide.6 However, it is known that these tools have limitations and may overestimate the risk in certain populations, which prompted the development of other scores.7,8 For example, the Systematic Coronary Risk Evaluation (SCORE), created based on the results of 12 European cohort studies, has been recommended since 2003 by the European CVD Prevention Directive.5 This score estimates the 10-year risk of fatal CVD relying on a model that encompasses countries with high and low incidence of CVDs (SCORE-High and SCORE-Low, respectively).8 618

household, classified as 'low' and 'high' in relation to the average of the population of the state of Santa Catarina, Brazil, in 2014 (BRL1,245 11 ) -and education level, categorized into 'low' (from no education to elementary school) and 'high' (from some high school to college graduate).
The following clinical characteristics were evaluated -presence of systemic arterial hypertension (previous diagnosis and/or use of antihypertensive medication), type 1 and 2 diabetes mellitus (DM) (previous diagnosis and/or treatment), smoking habit ('non-smoker' and 'smoker', i.e., current smokers or those who had stopped smoking less than two years before); 12 and lipid profile -total cholesterol (TC) (mg/dL) and HDL cholesterol (HDL-c) (mg/dL) during the last 12 months (nine of the 51 participants had no recent lipid profile). In addition, weight (kg) and height were determined using an anthropometric scale, and systolic blood pressure (SBP) (mmHg) was measured in the upper limbs after five minutes of rest, in supine position, using an automatic oscillometric sphygmomanometer; the highest measure between both arms was considered for analysis. 13 Body mass index (BMI) was calculated, and a BMI ≥ 25 kg/m²and > 30 kg/m² considered overweight and obesity, respectively. 14

Application of CVD risk scores
The FRS uses the variables gender, age, SBP, hypertension treatment, smoking habit, DM, HDL-c and TC for calculating the global 10-year CVD risk, using the online calculator available on the Framingham Heart Study website. 15 According to this tool, subjects were classified as having low (<10%), moderate (10-20%) or high risk (> 20%). 6 The SCORE, in turn, classifies individuals at low (<1%), moderate (≥1% and <5%), and high risk (≥5%) of having fatal CVD in 10 years, using the variables gender, age, SBP, TC, HDL-c and smoking status for its calculation. 5 The SCORE was calculated using the online calculator available on the HeartScore website, and both versions of the SCORE for high-and low-risk European countries were applied to the participants of our study. 16 For participants with no recent lipid profile, risk calculation was performed using models in which lipid variables are replaced by BMI in both FRS and SCORE (Table 1).
In Brazil, the Brazilian Cardiovascular Prevention Guideline recommends the use of the 2008 FRS, which estimates the 10-year risk of global CVD. 4,7 However, some studies indicate that there may be differences in risk stratification between the FRS and the SCORE, which could lead to different therapeutic approaches for the same patient, especially with regard to beginning treatment with hypolipidemic drugs. 6,9,10 Given the need to identify asymptomatic subjects at high risk of developing CVD, and effects of using different scores on treatment decision making, the aim of this study was to assess the degree of agreement between the FRS and the SCORE in cardiovascular disease risk stratification of a disease-free population at a teaching hospital.

Study design
This was a cross-sectional, observational, descriptive and analytical pilot study.

Population and sample
We interviewed 121 patients attending the internal medicine outpatient clinic of a university hospital in Southern Brazil, from January 2014 to January 2015. From this convenience sample, 51 patients of both sexes aged between 40 and 65 years (the widest age range common to both scores), without a diagnosis of CVD met the inclusion criteria. Patients with CVD and patients with incomplete data for risk score application were excluded. The study was approved by the human research ethics committee (project number 1973.8713.8.0000.0121) (Annex IV) and conducted after the consent form was signed (Brazilian National Health Council Resolution 196/96/MS).

Study variables
Using a standard form, trained medical students collected participants' sociodemographic and clinical data by interview and by review of medical records.
The sociodemographic variables were age, gender, self-reported race (white and non-white), per capita family income -self-declared income in Brazilian reais divided by the number of residents of the same

Statistical analysis
Normality of the data was visually verified by analysis of histograms and no specific statistic test was needed for such evaluation. Continuous variables were described as mean and standard deviation, and categorical variables as proportion and absolute frequency. Risk strata were divided into low/moderate risk and high risk for comparison of sociodemographic and clinical variables. The proportion of individuals in each stratum was calculated with a 95% confidence interval (CI). Parametric and non-parametric statistics were used to complement the descriptive analysis and to identify the associations between the variables included in the scores and the high-risk stratum of the different tools studied. Unpaired Student's t test was used for analysis of continuous variables and Chi-square test or Fisher's test, when appropriate, for categorical variables, and a p value <0.05 was considered statistically significant. The agreement rate between risk scores was assessed by kappa statistics, a correlation statistic used to test for interrater reliability. It ranges from -1 to 1 and is interpreted as follows: 17

Results
A total of 51 patients met the inclusion criteria, and Table 2  A moderate agreement was observed between the FRS and SCORE-Low stratification (K = 0.516); however, when the same analysis was performed comparing the subgroups "low/ moderate risk" vs. "high risk", these two scores showed poor agreement (K = 0.352). On the other hand, there was a high agreement between the FRS and the SCORE-High (K = 0.638), with similar results in the comparisons between the subgroups (K = 0.628). There was an excellent agreement between the two SCORE models (K = 0.807); however, this value  decreased when the comparison was made between the low/ moderate risk" vs. "high risk" subgroups, although a high agreement was maintained (K = 0.682).
Most patients in the high-risk category was men, white, hypertensive, non-smokers, overweight or obese in all scores used; there was a higher prevalence of diabetic patients in the FRS compared with the SCORE. In all tools, there was a statistically significant relationship between the high-risk stratum and the variables age and SBP. Participants in this category was older and had higher average SBP compared with individuals at low/moderate risk. In addition, SCORE-High and FRS also showed a significant relationship between the high-risk group and male participants, but only the FRS showed a significant relationship between this group and the presence of DM and SAH (Table 3).
When comparing the distribution of variables in the high-risk stratum of FRS versus the SCORE-High (which showed higher agreement according to kappa statistics), the p value ranged from 0.28 to >0.99 (Table 4).

Discussion
The FRS classified a higher proportion of subjects as at high risk for CVD compared with the SCORE (35.3% versus 23.5% by SCORE-High, and 13.7% by SCORE-Low), suggesting the tendency of this score to overestimate the risk in certain populations. 7,8 with possible implications in therapeutic decisions. When assessing the degree of agreement between the 10-year cardiovascular risk stratification using these three tools, a high degree of agreement was observed between the FRS and the SCORE-High, both when comparing the three risk groups (low, moderate and high) and in the two groups "low/ moderate risk" vs. "high risk" (K = 0.638 and K = 0.628, respectively). When FRS was compared with the SCORE-Low, the degree of agreement was moderate (K = 0.516), but the result of the dichotomized (low/ moderate risk vs. high risk) analysis was poor (K = 0.352).
The literature indicates poor to high degree of agreement between FRS and SCORE, and this variation is observed depending on where the comparison was performed and/or on the methodology used. 7,9,10,19,20 In a Spanish study, there was poor agreement between the FRS version recommended by the Adult Treatment Panel III (ATP III) and the SCORE-Low, 10, 21 while a research in Germany showed moderate agreement between an older version of the FRS (1991) and both SCORE models (High and Low). 7 In an Iranian study using a different methodology for assessing the degree of agreement, the result was similar (high agreement between FRS vs. SCORE-High) when compared to the present series. 9 Ten-year cardiovascular risk % In Brazil, in a population of HIV-positive patients, there was poor to moderate agreement between the models, and poor agreement in women who survived breast cancer. However, no studies comparing the 2008 FRS with the SCORE model was found. 19,20 Given the findings from this series, in accordance with previous studies, 7,9,10 it can be inferred that we may use both the FRS 2008 and the SCORE-High to stratify cardiovascular risk, without this meaning the need to adopt different therapeutic measures.

Table 3 -Distribution of variables for cardiovascular disease risk stratification according to the Framingham risk score and the SCORE-High/Low in 51 patients attending the internal medicine outpatient clinic of a university hospital in
The comparison of the FRS' and SCORE-High's highrisk groups (Table 4) corroborates that these two models are similar in terms risk stratification, since there was no statistically significant difference in the distribution of the variables analyzed between the tools, indicating that both groups had a similar composition.
The degree of agreement between SCORE-High and SCORE-Low ranged from excellent (K = 0.807) to high (K = 0.682) -the latter being obtained from the dichotomized risk classification -as these two models were derived from the same cohorts. 8 It is important to mention that although the FRS and the SCORE assess different cardiovascular outcomes -risk of global CVD and fatal CVD, respectively -we believe that the comparison between both instruments is valid, since they both stratify patients at low, moderate or high risk. 4,6,8 In addition, the equivalence of risk estimates is mentioned in the literature; the values obtained by the SCORE stratification, when multiplied by three for men and by four for women, are equivalent to the FRS stratification. 5,9 In many European countries, cardiovascular mortality data are easy to obtain, allowing SCORE calculators to be calibrated according to the cardiovascular mortality in each country, regardless of existing cohort studies to validate the risk stratification tools. 8 When comparing the results of the FRS and the SCORE in the studied sample, we observed a risk pattern similar to that of European countries with high cardiovascular risk. Therefore, it is possible to adjust these calculators to the Brazilian population, since there are no calibrated scores for this population so far, despite indicators of cardiovascular mortality comparable to those of countries of the SCORE-High group. 2,22  However, the comparison of the high risk versus low/ moderate risk stratum showed that when using the FRS assessment, a greater number of traditional cardiovascular risk factors had a statistically significant relationship with the high-risk stratum (Table 3). Higher mean age and SBP, and male sex were related to the SCORE-High, and in the FRS, in addition to these variables, hypertension and DM were also present in the high-risk group. This suggests that the FRS is a more appropriate risk stratification score to the population studied, since a statistically significant relationship was indeed expected between the traditional cardiovascular risk variables (age, male sex, hypertension, DM, dyslipidemia, and smoking) and the high-risk group. 6 However, this may be a result of the non-inclusion of DM in the SCORE models and also of the sample's low power.
The prevalence of individuals classified as at high cardiovascular risk by both FRS and SCORE was higher in our study (35.3% and 25.3%, respectively) compared to the literature, which reported a prevalence ranging from 1.9 to 15.1%, depending on the instrument used and the group of patients analyzed. 10,19,23 This result can be explained by the characteristics of the sample, composed of patients attending a tertiary university hospital, which treats patients with more complex needs. 24 In addition, the high prevalence of the cardiovascular risk factors 25-27hypertension (47.1%), DM (27.5%) -as well as overweight and obesity (78.5%) which was higher than that observed in the Brazilian (18.9%) and North American (33.8%) populations, 28,29 in the sample may also have influenced this result.
The inclusion of diabetic patients in the study can be considered a limitation, since according to the Brazilian and European guidelines, these patients are already considered at high cardiovascular risk. 4,5 The main objective of this study, however, was to compare other risk stratification models than those proposed by the guidelines; this motivated the inclusion of diabetic participants, since the FRS includes diabetes in its regression model, although the SCORE does not consider this variable (as this information was not consistently collected in the cohorts used in its development). 6,8 In addition, the sample size and composition can be seen as limitations, as a small number of participants and the convenience sampling make it difficult to extrapolate these results to the general population.
Considering the characteristics of the cardiovascular risk assessment tools studied, we understand the repercussions of the use of these tools in clinical practice. On the other hand, further studies are needed to validate cardiovascular risk scores in the Brazilian population.

Conclusion
In the study population, a higher number of patients were classified in the high cardiovascular risk group according to the FRS compared with European models. However, there was a high agreement between the FRS and the SCORE-high regarding risk stratification, although the agreement between the FRS and the SCORE-low ranged from moderate to poor.

Potential Conflict of Interest
No potential conflict of interest relevant to this article was reported.

Sources of Funding
There were no external funding sources for this study.

Study Association
This study is not associated with any thesis or dissertation work.

Ethics approval and consent to participate
This study was approved by the Ethics Committee of the UFSC under the protocol number CAAE 1973.8713.8.0000. 0121. All the procedures in this study were in accordance with the 1975 Helsinki Declaration, updated in 2013. Informed consent was obtained from all participants included in the study.