Performance of Diagnostic Tests for Intermediate Probabilities of Coronary Heart Disease: A Decision Making Analysis

Abstract

Patients with intermediate probability of coronary disease are a diagnostic challenge and it is precisely in this population where the degree of uncertainty is greater that the diagnostic tests have their greater applicability. However, according to the current definition, subjecting to tests a population with a disease probability between 10 and 90% can generate unnecessary tests and misleading results. Knowing the characteristics of each test, as well as risks and benefits of drug treatment for coronary disease and combining this information through diagnostic thresholds brings a new perspective to decision making. To review the origin of the currently recommended concepts of intermediate probability and to determine the thresholds for diagnosis and treatment of noninvasive tests and, based on them, propose a new concept of intermediate probability of coronary disease. Through the bibliographic review, meta-analyses were extracted in which data of sensitivity, specificity, positive and negative likelihood ratio, risks and benefits of the tests and treatment were provided. Using an algorithm developed by Pauker et al. it was possible to obtain the diagnostic and treatment thresholds adjusted for each tests in question. The concept of intermediate probability of coronary disease is quite broad, ranging, according to the authors, between 10 and 90%, 1 and 92%, 15 and 85%, with different rationale. Considering the discriminatory power of each test, risks and treatment benefits, the diagnostic and treatment thresholds were defined for exercise testing (22-58%), eco-stress (10-72%), myocardial scintigraphy (12-80%), nuclear magnetic resonance (16-80%) and coronary angiotomography (6.7-81%). The decision to submit to diagnostic tests should be individualized, taking into account the diagnostic and treatment thresholds of each method in question.

Keywords:
Coronary Artery Disease; Probability; Clinical Decision-Making; Diagnosis; Meta-Analysis

Resumo

Pacientes com probabilidade intermediária de doença coronariana são um desafio diagnóstico e é justamente nessa população onde o grau de incerteza é maior que os testes diagnósticos têm sua maior aplicabilidade. Entretanto, de acordo com a definição vigente, submeter uma população com probabilidade de doença entre 10 e 90% pode gerar exames desnecessários e resultados equivocados. Conhecer as características de cada teste, assim como riscos e benefícios do tratamento medicamentoso para doença coronariana e conjugar essas informações através dos limiares de diagnóstico trazem uma nova perspectiva à tomada de decisão. Objetivos: Revisar a origem dos conceitos atualmente preconizados de probabilidade intermediária e determinar os limiares de diagnóstico e tratamento dos testes não invasivos e, com base neles, propor um novo conceito de probabilidade intermediária de doença coronariana. Através da revisão bibliográfica foram extraídas metanálises nas quais dados de sensibilidade, especificidade, razão de verossimilhança positiva e negativa, riscos e benefícios dos testes e tratamento foram fornecidos. Utilizando-se algoritmo desenvolvido por Pauker e colaboradores foi possível obter os limiares de diagnóstico e tratamento ajustados para cada exame em questão. O conceito de probabilidade intermediária de doença coronariana é bastante amplo, variando, conforme os autores, entre 10 e 90%, 1 e 92%, 15 e 85%, com racionalidade distinta. Contemplando-se o poder discriminatório de cada exame, riscos dos testes, riscos e benefícios do tratamento, os limiares de diagnóstico e tratamento foram definidos para teste ergométrico (22-58%), eco-stress (10-72), cintilografia miocárdica (12-80%), ressonância nuclear magnética (16-80%) e angiotomografia de coronárias (6,7-81%). A decisão quanto à submissão aos testes diagnósticos deve ser individualizada, levando-se em consideração os limiares de diagnóstico e tratamento de cada método em questão.

Palavras-chave
da Artéria Coronariana; Probabilidade; Tomada de Decisão Clínica; Diagnóstico; Metanalise

Introduction

Estimation of the probability that a given event occurs is a constant challenge to decision making in virtually all areas of knowledge. Particularly in the medical field, the first step in the search for a specific diagnosis arises from a clinical suspicion, based on anamnesis and physical examination findings and on the prevalence of a given pathology in a population of interest. However, the determination of the patient's probability of presenting the pathology in question may still express a degree of uncertainty considered too large for decision-making - by both physicians and patients. Thus, often the investigation continues to increase the information pertinent to the case.

This knowledge assists in deciding on the best course of action: only observing a patient with a suspected pathology, submitting him to diagnostic tests or treating him without the need for further investigation. Few tests are able to completely determine whether a patient is healthy or sick, since we do not have diagnostic tests with sensitivity and specificity of 100%, but it is precisely in patients with intermediate probability that their greater applicability is found. Adequate knowledge of the characteristics and limitations of the test in question will allow us to determine the diagnostic and treatment thresholds that should guide the clinical decision. Bayesian analysis consists in adjusting a given initial probability given new information - whether it is a finding at the clinical examination or the result of a diagnostic test. It is possible to obtain the post-test probability by multiplying the likelihood ratio of a test by the a priori probability. In view of these values, it will be possible to evaluate the real usefulness in testing a patient and how much a particular result will be able to promote changes in the pre-test probability of disease obtained during the clinical evaluation.¹

It is when the degree of uncertainty is larger that usually a given test is capable of promoting greater changes between the pre- and post-test probabilities. However, the definition of these limits is often arbitrary, reproducing in diagnostic studies ranges as wide as 10 and 90%.

Two thresholds are extremely important in the decision to observe, test, or treat without testing. When the pre-test probability of a particular disease is low, even if a particular test is accurate enough, little change will be observed in the post-test probability of the patient presenting the disease. This threshold, below which no diagnostic investigation or even specific treatment is justified is referred to as the diagnostic threshold. Certain patients are so likely to present a pathology that even a negative result from a diagnostic test will hardly reduce the patient's chance of developing disease (post-test probability). The value above which further investigation is dispensable and the patient should be conducted to the treatment in question is considered the therapeutic threshold. The establishment of such thresholds can be intuitively performed by the attending physician, considering the cost-benefit relationship of the treatment, using life expectancy data to calculate this relationship or even assigning value to the patient's preferences in relation to the possibilities of outcomes with the disease and its treatment.¹

Determining the limits of the intermediate probability and its rationality is the focus of the present article, since the adequacy of these values is questionable in a scenario where the diagnostic tests present different performances.

Objectives

This review article looks for the origins of the concepts commonly used as an intermediate probability definition and questions the adequacy of the values most frequently presented in relation to the variations in the diagnostic performance of the different tests used in clinical practice, as well as to determine the diagnostic and treatment thresholds for each test.

Methods

Through PUBMED/MEDLINE, a bibliographic research was conducted focusing on articles that considered the diagnostic performance and risks of the most frequently used tests for the diagnosis of coronary artery disease - ergometric test, myocardial perfusion scintigraphy, echo-stress, nuclear magnetic resonance and Angiotomography of Coronary Artery Disease; Risks and benefits of the treatment most frequently advocated in coronary artery disease - ASA, statin and beta-blocker; as well as references of intermediate probability of coronary disease and medical decision - more specifically the Bayesian analysis.

The search contained the following terms mesh: diagnostic test/ accuracy/ sensitivity/ specificity/ intermediate probability/ coronary disease/ exercise test/ myocardial perfusion imaging / magnetic resonance imaging/ coronary angiography/ medical decision/ Bayesian analysis/ pre-test probability/ post-test probability/adverse effects/ improvement.

Statistical analysis

For the calculation of the diagnostic and therapeutic thresholds, the formulas described below were used, based on the following data: sensitivity, specificity, risks and benefits of the test, risks and benefits of the treatment.²

Test Threshold or Diagnostic Threshold (Tt):

Tt = (Ppos/nd) x (Rrx) + Rt

(Ppos/nd) x (Rrx) + (Ppos/d) x (Brx)

Therapeutic Threshold (Ttx):

Ttrx = (Pneg/nd) x (Rrx) - Rt

(Pneg/nd) x (Rrx) + (Pneg/d) x (Brx)

Tt = Diagnostic threshold

Ttrx = Therapeutic threshold

Ppos/nd = Probability of a positive test, when the patient is healthy (False Positive or 1-Specificity)

Ppos/d = Probability of a positive test, given that the patient is sick (True Positive or Sensitivity)

Pneg/nd = Probability of a negative test, given that the patient is not ill (True Negative or Specificity)

Pneg/d = Probability of a negative test, when the patient is sick (False Negative or 1-Sensitivity)

Rrx = Treatment risks

Rt = Test risks

Brx = Treatment benefit

Results

Among the studies that considered the delimitation of the intermediate probability, it is of particular importance the one realized by Diamond et al. that, in 1980, studied 43 patients submitted to Coronariography, later stratified with noninvasive tests. Of these, 8 out of 12 patients with normal coronary angiography had a posttest probability of less than 10%; While 26 of 31 patients with obstructive coronary artery disease had a posttest probability greater than 90%. This was the first work in which the limits of 10 and 90% were established for intermediate probability.³

In a study conducted by Goldman et al.⁴ in 1982, a pre-test probability of between 1 and 92% was considered intermediate, considering the probability on which the ergometric test was able to produce a post-test value above or below 50%. Also in this same article were calculated thresholds of pre-test probability on which non-invasive stratification would be able to generate a shift to a probability that would allow decision making.⁴

The Brazilian Guidelines for Stable Coronary Disease considers as intermediate probability values between 10% and 90%, and Montalescot et al.⁶ at the European Consensus of Stable Angina, admit as intermediate probability values between 15% and 85%, considering that the majority of diagnostic tests show sensitivity and specificity around 85%.^5-6 According to these data, when applied to a healthy population, 15 out of 100 tests will give false results and, therefore, it will be more appropriate not to submit patients with low disease prevalence (less than 15%) to the methods in question. In patients with high disease probability the test will also not bring benefits.

The characteristics of the following diagnostic tests were then analyzed: ergometric (or stress test), dipyridamole myocardial scintigraphy, myocardial scintigraphy with ET, dobutamine echo-stress, magnetic nuclear resonance and coronary angiotomography, where sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic and therapeutic threshold were considered and expressed in Table 1.^7-9 A particularly noteworthy fact is the possibility of establishing thresholds for diagnosis and treatment for coronary angiography - until now considered a golben standard with which all the others are compared when comparing their performance with the data provided by the reserve flow fraction (RFF).¹⁰

Table 1
Characteristics of the diagnostic tests: sensitivity, specificity, positive and negative likelihood ratio

Diagnostic Tests	Sensitivity (%)	Specificity (%)	LR+	LR-
Ergometric Testing4	67	71	2.31	0.46
Myocardial Scintigraphy7	87	81	4.57	0.16
Eco-stress8	79	87	6.07	0.24
NMR9	89	76	3.7	0.14
CCTA9	87	91	9.6	0.14

• NMR: nuclear magnetic resonance; CCTA: Coronary computed tomography angiography; LR+: Positive likelihood ratio; LR: Negative likelihood ratio.

To determine the risks related to the tests analysed only events with significant repercussion were considered as those capable of interrupting the test or generating clinical or hemodynamic repercussions - described in table 2,^11-15 such as claustrophobia, hypotension, arrhythmias, infarction, stroke, cardiac arrest and death.

Table 2
Risks and adverse reactions in diagnostic tests

Diagnostic test	Adverse reactions	Incidence (%)
Ergometric Testing4	arrhythmia, AMI, death	0.11
Myocardial Scintigraphy11	Hypotension, arrhythmia, AMI, death	50
Dobutamine Eco-stress13	Hypotension, hypertension, arrhythmia, AMI, death	5.6
Magnetic Nuclear Resonance12	Claustrophobia	15
CCTA14	Serious reactions, Hypotension by Isordil	4.1
Coronary Cineangiography15	Dissection, Vessel occlusion, AMI, Bleeding, Stroke	11.5

Considering ASA, statin - more specifically simvastatin and beta-blocker, the drugs most frequently used in the treatment of coronary artery disease, the risks and the benefit of the joint use of the 3 drugs were obtained for 1 year, considering the therapeutic adherence and the probability of each user benefitting from the use of the drugs in question, values are expressed in table 3.¹⁶,¹⁷

Table 3
Risks and benefits associated with treatment

Medicines	Benefit (in 1 year)	Risks (in 1 year)
ASA	2.47%	0.4 %	(Bleeding)
Statin	6.85%	4.84%	(Rhabdomyolysis, Elevation of transaminases)
Beta blocker	4.61%	4.2%	(Atrioventricular block, syncope)
Set	13.34%	8.91%

Based on the information about the performance characteristics of the test, its associated risks, as well as the benefit of the joint treatment and the inherent risks, it was possible to establish the diagnostic and therapeutic thresholds of the five methods in question - ET, MPS Study, Eco-stress with dobutamine, MRI and coronary angiotomography, expressed in Table 4.

Table 4
Diagnostic and therapeutic thresholds of the different diagnostic methods

Diagnostic Tests	Diagnostic Threshold (%)	Therapeutic Threshold (%)
Ergometric Testing	22	58
Myocardial Scintigraphy	12	80
Dobutamine Eco-stress	10	72
Magnetic Nuclear Resonance	16	80
CCTA	6.7	81

Discussion

The diagnostic process is configured in a complex procedure of association of ideas, with inclusion of positive and negative data. The estimation of the probability of coronary artery disease begins in a subjective way already in anamnesis and clinical examination, where today we search for information about the main known risk factors for coronary disease - hypertension, DM, dyslipidemia, smoking and family history - to know its characteristics and limitations is crucial. However, the physician often faces a level of certainty considered insufficient for decision making, especially when the patient has an "intermediate probability" of disease.

It is precisely in this larger range where researchers focus their efforts on a better risk stratification - using, among other strategies, submission to diagnostic tests. Effective tests do not guarantee, by themselves, the best clinical outcome, but help find the best course of action, with patient benefits and savings for health services when used properly.

In 1980, Pauker et al.² already advised how to quantitatively establish thresholds for diagnosis and treatment based on test performance data - sensitivity, specificity, risks and benefits, and risks and benefits of the treatment to be instituted if certain pathologies are confirmed.²

Thus, taking the ergometric test and its diagnostic and therapeutic thresholds of 22 and 58%, respectively, the adequacy of the "static" values from 10 to 90% can be questioned, since some patients within this range, should be submitted to the test in question. Thus, we would observe that patients between 10 and 22% pre-test probability of CAD would be unnecessarily submitted to the ergometric test, burdening the health system and without benefits to the patient. Still within this same perspective, patients with a probability of 58-90% of pre-test probability would also be submitted to the test, when the analysis of these thresholds suggests that rationality exists in treating these patients without testing them.

This phenomenon can also be observed in relation to the other tests, enough to observe the discrepancy between the interval between the diagnostic and therapeutic thresholds of each test and the fixed values adopted in the literature, be them between 10-90% or 20-70%.

In all the studies analysed to obtain the data regarding the performance of the diagnostic tests, the methods were compared with the coronary angiography, considered gold standard. However, with the advent of the flow reserve fraction, the coronary angiography was questioned as the best parameter against which all the others could be compared, since the functional information showed a better correlation with adverse events, reflecting a pathological behavior that extrapolates the phenomenon of luminal obstruction when considering also the component of endothelial dysfunction in its evaluation.

Specifically analysing the coronary angiography when compared to the RFF in a study performed by Sant'Anna et al.,¹⁰ the results showed a 57% and 96% accuracy for moderate and severe lesions, respectively, with a sensitivity of 85.7% and a specificity of 36.7%. Extrapolating these data to the calculation of the thresholds, we highlight the narrow range of probability where its use would be justified, between 34% (diagnostic threshold) and 61% of pre-test probability (therapeutic threshold). This fact is of particular importance considering that the Brazilian Consensus of Stable Angina classifies as class II A, level of evidence C the stratification of high-risk patients with cineangiocoronariography. However, data in the Brazilian literature point to a rate of complications that cannot be neglected, capable of reaching values as expressive as 11.5% during catheterization followed by angioplasty.¹⁵

The submission to an invasive test in patients whose high probability of CAD would justify the start of treatment may add an unnecessary risk to the propaedeutic of these patients. Emblematic works, such as that carried out by Patel et al.¹⁷ in 2010, point to a rate of 39.2% of negative tests for obstructive coronary disease (lesions lower than 20% of luminal obstruction) in 398,978 patients referred to coronary angiography in 663 hospitals, reinforcing the need for a better diagnostic stratification prior to the indication of invasive examination.¹⁸ Hence, it would be more appropriate to relocate the indication of the procedure to those in whom an intervention is planned, may be it angioplasty or surgery.

Data analysis shows that the risk of the test is the main determinant of the diagnostic threshold, while the risk of the treatment influences the therapeutic threshold. The greater the risk of the test, the greater the pre-test probability required to justify its use, with a higher diagnostic threshold and lower therapeutic threshold. Likewise, the greater the risk of the treatment, the greater the degree of certainty to start the treatment should be, raising the therapeutic threshold values.

Limitations

In the present study only the diagnostic role of the main tests was contemplated - which diverges from the current clinical practice, where the diagnosis of coronary artery disease can be obtained by clinical criteria and the tests play a prognostic role greater than the diagnostic one, and the catheterization is reserved to those patients in whom an intervention is planned. However, the methodology applied reinforces that clinical treatment can and should be offered and restricts the circumstances in which the tests are actually needed to obtain the diagnosis and institute treatment. Although changes in test performance are observed according to the source consulted, there is little impact on the observed outcome.

Conclusion

Considering the performance of the majority of the diagnostic tests, with sensitivity and specificity around 80%, it is understood that values between 30 and 70% contemplate the great majority of the intervals between thresholds of the different diagnostic methods. However, there is still a significant portion that is inadequately evaluated, where the risk of being submitted to a diagnostic test is higher than the risk of non-treatment and another where the risk of treatment is lower than the risks related to the test. Thus, it is necessary that the intermediate probability be stipulated for each test, including updating it in the light of the recent advances that make possible a greater accuracy of the method in question, minimizing the uncertainties inherent in the decision-making process.

• Sources of Funding
  There were no external funding sources for this study.
• Study Association
  This article is part of the thesis of Doctoral submitted by Clarissa Antunes Thiers, Universidade Federal do Rio de Janeiro (UFRJ).

References

• ¹
  Fagan TJ. Letter: Nomogram for Bayes theorem. N Engl J Med. 1975;293(5):257. Doi 10.1056/NEJMi97507312930513.
  » https://doi.org/10.1056/NEJMi97507312930513
• ²
  Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302(20):1109-17.doi:10.1056/NEJMi98005153022003.
  » https://doi.org/10.1056/NEJMi98005153022003
• ³
  Diamond GA, Forrester JS, Hirsch M, Staniloff HM, Vas R, Berman DS, et al. Application of conditional probability analysis to the clinical diagnosis of coronary artery disease. J Clin Invest. 1980;65(5):1210-21.PMID:6767741.
• ⁴
  Goldman L, Cook EF, Mitchell N, Flatley M, Sherman H, Rosati R, et al. Incremental value of the exercise test for diagnosing the presence or absence of coronary artery disease. Circulation. 1982;66(5):945-53.PMID:7127706.
• ⁵
  Cesar LA, Ferreira JF, Armaganijan D, Gowdak LH, Mansur AP, Bodanese LC, et al. Guideline for stable coronary artery disease. Arq Bras Cardiol. 2014;103(2 Suppl 2):1-56.PMID:254100816.
• ⁶
  Montalescot G, Sechtem U, Achenbach S, Andreotti F, Arden C, Budaj A, et al. 2013 ESC guidelines on the management of stable coronary artery disease: the Task Force on the management of stable coronary artery disease of the European Society of Cardiology. Eur Heart J. 2013;34(38):2949-3003.doi:10.1093/eurheartj/eht296.
  » https://doi.org/10.1093/eurheartj/eht296
• ⁷
  Jaarsma C, Leiner T, Bekkers SC, Crijns HJ, Wildberger JE, Nagel E, et al. Diagnostic performance of noninvasive myocardial perfusion imaging using single-photon emission computed tomography, cardiac magnetic resonance, and positron emission tomography imaging for the detection of obstructive coronary artery disease: a meta-analysis. J Am Coll Cardiol. 2012;59(19):1719-28. doi:10.1016/j.jaac.2011.12.040.
  » https://doi.org/10.1016/j.jaac.2011.12.040
• ⁸
  Heijenbrok-Kal MH, Fleischmann KE, Hunink MG. Stress echocardiography, stress single-photon-emission computed tomography and electron beam computed tomography for the assessment of coronary artery disease: a meta-analysis of diagnostic performance. Am Heart J. 2007;154(3):415-23. PMID:17719283.
• ⁹
  Budoff MJ, Achenbach S, Duerinckx A. Clinical utility of computed tomography and magnetic resonance techniques for noninvasive coronary angiography. J Am Coll Cardiol. 2003;42(11):1867-78. PMID:14662244.
• ¹⁰
  Sant'Anna FM, da Silva ER, Batista LA, Brito MB, Ventura FM, Ferraz HA, et al. What is the angiography error when defining myocardial ischemia during percutaneous coronary interventions? Arq Bras Cardiol. 2008;91(3):162-7, 79-84.
• ¹¹
  Ranhosky A, Kempthorne-Rawson J. The safety of intravenous dipyridamole thallium myocardial perfusion imaging. Intravenous Dipyridamole Thallium Imaging Study Group. Circulation. 1990;81(4):1205-9. PMID:2107985.
• ¹²
  Enders J, Zimmermann E, Rief M, Martus P, Klingebiel R, Asbach P, et al. Reduction of claustrophobia during magnetic resonance imaging: methods and design of the "CLAUSTRO" randomized controlled trial. BMC medical imaging. 2011;11:4. doi:10.1186/1471-2342-11-4.
  » https://doi.org/10.1186/1471-2342-11-4
• ¹³
  Geleijnse ML, Krenning BJ, Nemes A, van Dalen BM, Soliman OI, Ten Cate FJ, et al. Incidence, pathophysiology, and treatment of complications during dobutamine-atropine stress echocardiography. Circulation. 2010;121(15):1756-67. doi:10,116/CIRCULATIONAHA.109.859264.
  » https://doi.org/10,116/CIRCULATIONAHA.109.859264
• ¹⁴
  Heidenreich PA, McDonald KM, Hastie T, Fadel B, Hagan V, Lee BK, et al. Meta-analysis of trials comparing beta-blockers, calcium antagonists, and nitrates for stable angina. JAMA. 1999;281(20):1927-36. PMID:10349897
• ¹⁵
  Mallet ALR, Oliveira GMMd, Klein CH, carvalho MRMd, Silva NASe. Letalidade e complicações de angioplastias em hospitais públicos no Rio de Janeiro, RJ. Rev Saúde Pública [online]. 2009;43(6):917-27.PMID:20027504.
• ¹⁶
  Hayden M, Pignone M, Phillips C, Mulrow C. Aspirin for the primary prevention of cardiovascular events: a summary of the evidence for the U.S. Preventive Services Task Force. Annals of internal medicine. 2002;136(2):161-72. PMID:11790072.
• ¹⁷
  Deanfield JE, Sellier P, Thaulow E, Bultas J, Yunis C, Shi H, et al. Potent anti-ischaemic effects of statins in chronic stable angina: incremental benefit beyond lipid lowering? Eur Heart J. 2010;31(21):2650-9. doi:10.1093/eurheartj/eh9133.
  » https://doi.org/10.1093/eurheartj/eh9133
• ¹⁸
  Patel MR, Peterson ED, Dai D, Brennan JM, Redberg RF, Anderson HV, et al. Low diagnostic yield of elective coronary angiography. N Engl J Med. 2010;362(10):886-95. PMID:20220183.

Publication Dates

• Publication in this collection
  21 Sept 2017
• Date of issue
  Nov-Dec 2017

History

• Received
  06 Oct 2016
• Reviewed
  11 May 2017
• Accepted
  07 July 2017