DIAGNOSTIC VALIDITY OF SCREENING QUESTIONNAIRE AMERICAN COLLEGE OF SPORTS MEDICINE/AMERICAN HEART ASSOCIATION

VALIDADE DIAGNÓSTICA DO QUESTIONÁRIO DE TRIAGEM DO AMERICAN COLLEGE OF SPORTS MEDICINE/AMERICAN HEART ASSOCIATION

ABSTRACT

The prevention of cardiovascular events is one of the main objectives of a preexercise screening questionnaire. The aim of this study was to evaluate the diagnostic validity of ACSM/AHA questionnaire (AAPQ), through the Sensitivity (Se), Specificity (Sp) and AUC (area under the curve). The study included 69 subjects (40-81 years). Comparisons between sex and age categories were performed. There was no significant difference. However, the variation in positive responses seemed to increase in percentage terms, especially after 60 years (from 78.6% to 100%). The AAPQ Se values were high (88.2%) suggesting that the majority of individuals in whom the instrument was positive showed some contraindication. However, the Sp value was only 13.5%, indicating that the questionnaire was not able to identify those with no contra indication to start an exercise program. Thus, the instrument would have a low diagnostic capacity (AUC = 0.51). The results suggest that the questionnaire presents a low capacity to identify individuals suitable for inclusion in an exercise program.

Keywords:
Validity of tests; Risk factors; Exercise; Questionnaire

RESUMO

A prevenção de eventos cardiovasculares é um dos principais objetivos de um questionário de triagem para a prática de exercícios. O objetivo do estudo foi verificar a validade diagnóstica do questionário do ACSM/AHA (AAPQ), através da sensibilidade (SE) e especificidade (ES), e AUC (área sob a curva). Participaram do estudo 69 indivíduos (40 a 81 anos). Comparações entre sexo e categorias de idade foram realizadas. Não foi verificada diferença significativa. Contudo, a variação nas respostas positivas pareceu aumentar percentualmente, principalmente a partir dos 60 anos (de 78,6% a 100%). Os valores de SE do AAPQ foram altos (88,2%), sugerindo que grande parte dos indivíduos em que o resultado do instrumento foi positivo apresentava alguma contraindicação. Contudo, o valor da ES foi de apenas 13,5%, indicando que o questionário não foi capaz de identificar aqueles indivíduos sem contraindicação ao início de um programa de exercícios. Com isso, o instrumento teria uma baixa capacidade diagnóstica (AUC=0,51). Os resultados sugerem que o questionário apresenta baixa capacidade em identificar indivíduos aptos a iniciar um programa de exercícios.

Palavras-chave:
Validade dos testes; Fator de risco; Exercício; Questionário

Introduction

Physical inactivity has been identified as a risk factor responsible for 3.2 million (2.7-3.7 million) deaths and 2.8% (2.4-3.2) lost years of healthy living in 2010¹. Higher levels of physical activity reduce the risk of coronary artery disease, stroke, diabetes, hypertension, colon and breast cancer, and depression. In addition, increased caloric expenditure contributes to weight control^2-4.

Paradoxically, especially considering sedentary adults over 40 years of age, the risk of a cardiovascular event appears to increase during a session involving moderate to high intensity exercises, which are characteristics recommended for achieving significant health benefits⁵. Preventing these events is one of the main purposes of a pre-participation screening, which can be performed through clinical examinations or self-administered questionnaires⁶. The alternative of the questionnaires is important not only to identify individuals with greater need for clinical examinations, but also to recognize those who do not need immediate tests. Two questionnaires deserve special attention, that is, the Physical Activity Readiness (PAR-Q)^7,8, and the American Heart Association (AHA)/American College of Sports Medicine (ACSM) Pre-participation Questionnaire (AAPQ)⁹.

These instruments are strongly recommended by the literature^6,9, however, their quality has been questioned by recent studies, especially that of the PAR-Q^10,11 with regard to the identification of the individuals who would be at risk due to physical exercise practice. The current literature still lacks studies regarding the AAPQ, which is recommended as a screening strategy, but whose Portuguese version, translated and adapted by Santos¹³, has been included in clinical publications¹².

Methods

Subjetcs

Sixty-nine 40-81-year-old subjects (41 men and 28 women) were randomly selected from 2012 to 2013 at a cardiology clinic. All of them had been referred for a cardiac exercise stress testing (cardiovascular disease investigation, monitoring of an existing cardiovascular disease and physical evaluation to start regular physical activity). In order to previously calculate the sample size, the proposal by Arkin and Wachtel was used for evaluating the diagnostic tests¹⁷, thus, establishing a SE at 90% and SP at 70% with an alpha of 5% and study power of 90%. Such a proposal would require a sample of 67 individuals. The study protocol was approved by the ethics committee of the university so-called Universidade Salgado de Oliveira under Opinion number 24895114.1.0000.5289, and all participants signed the Free Informed Consent Form.

Procedures

All the subjects answered the AAPQ (Figure 1); then, they underwent clinical and anthropometric examinations. The respondents did not know that their answers would be compared to clinical trials. The application of the questionnaire and the conduction of the ergometric tests were always performed by the same evaluator. The AAPQ consists of 32 items divided into three sections. In the first section, the respondent should indicate whether he/she has or had already had any health condition or symptom that might contraindicate the practice of exercises. If any of the items are marked, the individual is advised to consult a physician or another health care professional before beginning any physical activities. The second section lists common cardiovascular risk factors; if two or more items are marked, searching a physician or another health care professional is recommended. If there is no medical indication in any of the previous sections, the third section recommends the exercise program to be unrestricted started¹⁴.

Figure 1
Portuguese version of the AAPQ

The body mass and height were measured by using the Filizola stamometer (2 kg at 180 kg and zero at 210 cm). The Body Mass Index (BMI) was calculated based on the body mass and height measured.

A maximal exercise testing (ET) was performed on a treadmill (Inbramed®) by using Bruce Protocol. Continuous 12-lead electrocardiogram (ECG) monitoring was performed with the ERGO-PC software (Micromed®). Blood pressure was measured at rest and every minute during the test by using a Welch-Allyn® mobile aneroid sphygmomanometer. The following clinical criteria for discontinuation were established during the test: effort-limiting progressive angina; limiting pain or discomfort in the lower limbs that determined impossibility of ambulation; a blood pressure drop over 10mm Hg compared to that measured in the immediately preceding stage; increased systolic blood pressure ≥ 240 mm Hg or diastolic blood pressure ≥ 120 mmHg; complex ventricular arrhythmias¹⁸.

The ET was considered as an electrocardiographic criterion for effort-induced ischemia when angina occurred; the ST-segment depression with either horizontal or descending morphology (≥ 1mm, measured at point J), and an ascending morphology when (≥ 1.5mm, measured at point Y). The drugs with a negative chronotropic effect were suspended 48h before the test, except when requested by the physician, thus, being maintained¹⁶.

Statistical analysis

The proportion of AAPQ respondents who would be referred for cardiac examination was calculated for men and women. Comparisons between sex and age categories were carried out by using the chi-squared test and Fisher's exact test.

Regarding SE and SP, the AAPQ would be considered ‘true’ if there was agreement between its results and the ischemic answer seen in the stress testing (positive test). In contrast, it would be considered ‘false’ if there was no agreement. The validity of the instrument was assessed by calculating SE and SP. SE is defined as the proportion of individuals with contraindications to practice physical exercises who showed positive AAPQ. On the other hand, SP is the proportion of the individuals without exercise contraindications who had a negative AAPQ. The area under the curve (AUC) is a general coefficient for the interaction between SE and SP, and it can be considered as a measurement of the questionnaire diagnostic power.

Continuous variables were expressed as mean and standard deviation, and the inferential analysis by using the t test, and categorical variables as percentages. For all analyzes, the significance level was set at p ≤ 0.05 and STATA® Statistical Software 12.0 Standard Edition for Windows (StataCorp, College Station, TX) was used.

Results

Table 1 shows the descriptive statistics regarding the characteristics of men and women in the sample assessed. The men were significantly taller and had a higher body mass, in addition to showing a better performance in the stress testing evidenced by the maximum Resting Metabolic Equivalent (METs) values ​​and testing time. The other variables showed no difference.

Table 1
Sample Characteristics

	Women	Men	p
Age (years)	58,5 ± 10,6	61,9 ± 9,8	0,18
Body Mass (kg)	73,1 ± 17,5	82,8 ± 12,2	0,009
Height (cm)	161,7 ± 6,5	173,5 ± 7,7	<0,001
BMI (k/m 2 )	27,8 ± 5,6	27,4 ± 3,0	0,71
Resting SBP (mmHg)	128,0 ± 10,1	129,1 ± 13,0	0,78
Resting DBP (mmHg)	82,7 ± 5,9	80,9 ± 7,9	0,46
Maximum METs	7,3 ± 2,4	9,1 ± 2,5	0,005
Exercise Testing Time (min.)	7,3 ± 2,6	8,7 ± 2,3	0,03
Maximum SBP (mmHg)	168,5 ± 24,7	181,2 ± 22,0	0,09
Maximum DBP (mmHg)	81,7 ± 8,4	80,2 ± 6,1	0,54
Maximum HR (bpm)	152,6 ± 13,7	151,0 ± 21,0	0,73
ECG changes (%)	17,7	29,3	0,28
Hypertension(%)	10,7	2,4	0,30
Dyslipidemia (%)	7,1	2,4	0,56
Diabetes (%)	3,6	2,4	1,00

• Note: BMI = Body Mass Index, SBP = Systolic Blood Pressure, DBP = Diastolic Blood Pressure, MET = Resting Metabolic Equivalent, HR = Heart Rate, ECG = Electrocardiogram,
• Source: the authors

Table 2 shows the percentage of individuals who positively answered the AAPQ and who should, thus, be referred for cardiac examination. No significant difference was seen between sexes or the different age categories. However, the variation in positive responses appeared to increase in percentage, mainly from the age of 60 years on (from 78.6% to 100%).

Table 2
Percentage of positive respondents according to the questions on age and sex

Categories in years
		40-49	50-59	60-69	≥70
Risck factors (1 st Section)	Women	71,4	71,4	90,0	100,0
	Men	100,0	91,7	78,6	100,0
	Both Sexes	83,3	84,2	83,3	100,0
Risck factors(2 nd Section)	Women	85,7	71,4	100,0	75,0
	Men	60,0	83,3	92,9	100,0
	Both Sexes	75,0	78,9	95,8	92,9

• Source: the authors

The SE values of AAPQ for both the first and second sections were high (88.2%), which suggests that several individuals who had a positive result of the instrument showed indicative of ischemic electrocardiographic response during the stress testing. However, the SP value was only 13.5%, suggesting that the AAPQ would fail to accurately identify the individuals with no ischemic response at the beginning of a physical exercise program. The AUC value was of 0.51, which would classify the instrument as having a poor diagnostic capacity¹⁹.

Discussion

The application of questionnaires such as the AAPQ has been strongly recommended in the literature as a screening method before starting physical exercise programs²⁰. However, despite the extensive application of these questionnaires, there is still no evidence to justify why it is so widely used. In the present study, for example, a high proportion of positive responses was seen in the AAPQ, which suggests that the questionnaire would unnecessarily exclude a great number of individuals. This finding is reinforced when assessing the SE and SP values, since several positive answers were obtained, which made the instrument correctly identify the individuals with possible contraindications to the exercise (88.2%), but mistakenly classify those without contraindications (SP = 13.5%). A pre-exercise screening questionnaire with such a limited identification capacity (AUC = 0.51) would prove not to be accurate, and above all, the high number of false positives would lead to cost increases due to unnecessary clinical examinations.

On the other hand, other indications suggested a good sensitivity of the AAPQ. Lopes et al.²¹ applied the questionnaire to a group of individuals with previously diagnosed peripheral arterial disease and all of them were identified as having a contraindication according to the results of the AAPQ application. However, in case healthy individuals were included in the sample, it is not certain that the questionnaire would also accurately identify them.

When comparing the findings of the present study with the values ​​shown by Whitfield et al.¹⁴ who used the original version of the instrument, similarities can been seen between the results. For example, the positive responses obtained for both sexes were as follows: 85.0% (40-44 years of age); 93.9% (45-49 years of age); 96% (50-54 years of age); 96.5% (55-59 years of age); 98.7% (60-64 years of age); 97.4% (65-69 years of age) and 99.1% (≥70 years of age). These answers suggest equivalence between the Portuguese version herein used and the original one, mainly because they were samples of similar age (mean of 56.6 years of age in the study by Whitfield et al.¹⁴). In spite of the fact that a more appropriate study on measurement equivalence requires comparing validity coefficients, this is not possible because there is no study that has previously validated the instrument (either in its original or translated version).

Recent findings suggest that other popular screening instruments, such as PAR-Q, would not be an appropriate diagnostic tool, especially when identifying individuals without contraindications (SE and SP values: 77.8% and 19.8%, respectively, and AUC: 0.49)¹¹. Any pre-participation screening tool should accurately identify the individual with both, a high risk (SE) and a low risk (SP) for practicing physical exercises. Based on the results of the present study, the AAPQ seems to fail in the second option. In an attempt to minimize the excessive amount of positive answers, Whitfield et al.¹⁴ suggested the exclusion of questions regarding drug use and age group. When applying this suggestion to the results of the present study, a 15% reduction of positive responses was really seen, however, the SE and SP values were 76.5% and 26.9%, respectively, generating an AUC of 0.52, which would also suggest a low diagnostic capacity.

It is worth mentioning some limitations of the present study. For example, the fact of including subjects who had previously been referred to stress testing might have generated a greater proportion of positive responses in the questionnaire. Another consideration should be made with regard to both, the ‘outcomes’ and the criteria used herein to classify the questionnaire answers as true or false. It is understood that the stress testing has limitations, since it takes only the ECG into consideration²². However, it is noteworthy that even with its limitations, the use of such an exam has been advocated as a criterion for validating pre-participation screening questionnaires²³. Banerjee et al.²⁴ found it inappropriate to establish standard values regarding the diagnostic capacity of the stress testing, since a variation would occur according to some variables, such as age, sex, clinical characteristics, prevalence of cardiovascular disease and type of ergometer used. This variation also made the authors of the present study not to include SE and SP values of the ergometric test in the sample calculation.

Conclusions

The analysis of the diagnostic validity of the AAPQ suggests that the questionnaire does not accurately assess the individuals able to start doing physical exercises. Therefore, its use as a screening pre-participation instrument with regard to a physical exercise programs does not seem to be justifiable.

Aknowledgements:

To Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro -FAPERJ, "Jovem Cientista do Nosso Estado" scholarship (no. E-26/203.237/2016)

References

• ¹
  Lim SS, Vos T, Flaxman AD, Danaei G, Shibuya K, Adair-Rohani H, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; 380(9859):2224-60. Doi: 10.1016/S0140-6736(12)61766-8
  » https://doi.org/10.1016/S0140-6736(12)61766-8
• ²
  Lee I-M, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT, et al. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet. 2012; 380 (9838):219-229. Doi: 10.1016/S0140-6736(12)61031-9
  » https://doi.org/10.1016/S0140-6736(12)61031-9
• ³
  World Health Organization. Global recommendations on physical activity for health. Geneva, Switzerland: WHO; 201 A guide for population-based approaches to increasing levels of physical activity: implementation of the WHO Global Strategy on Diet, Physical Activity and Health. Geneva, World Health Organization, 2007.
• ⁴
  World Health Organization. Global status report on noncommunicable diseases 2010. Geneva, Switzerland: WHO Press; 2011
• ⁵
  Eijsvogels TM, Molossi S, Lee DC, Emery MS, Thompson PD. Exercise at the extremes: The amount of exercise to reduce cardiovascular events. J Am Coll Cardiol 2016;67(3):316-29. Doi: 10.1016/j.jacc.2015.11.034
  » https://doi.org/10.1016/j.jacc.2015.11.034
• ⁶
  Warburton DE, Gledhill N, Jamnik VK, Bredin SS, McKenzie DC, Stone J, et al. Evidence-based risk assessment and recommendations for physical activity clearance: consensus document 2011. Appl Physiol Nutr Metab 2011;36(Suppl 1):S266-98. Doi: 10.1139/h11-062
  » https://doi.org/10.1139/h11-062
• ⁷
  Shephard RJ, Bailey DA, Mirwald RL. Development of the Canadian Home Fitness Test. Can Med Assoc J 1976; 114: 675-679.
• ⁸
  Shephard RJ, Cox MH, Simper K. An analysis of "Par-Q" responses in an office population. Can J Public Health. 1981; 72:37-40.
• ⁹
  American College of Sports Medicine. ACSM's guidelines for exercise testing and prescription. Philadelphia: Wolters Kluwer/Lippincott Williams & Wilkins Health; 2014.
• ¹⁰
  Oliveira Luz LG, Maranhao Neto GA, Farinatti PTV. Validade do questionário de prontidão para a atividade física (par-q) em idosos. Rev. Bras Cineantropom Desempenho Hum 2007;9:366-371. Doi: 10.5007/%x
  » https://doi.org/10.5007/%x
• ¹¹
  Maranhao Neto GA, Luz LG, Farinatti PT. Diagnostic accuracy of pre-exercise screening questionnaire: emphasis on educational level and cognitive status. Arch Gerontol Geriatr 2013;57:211-214. Doi: 10.1016/j.archger.2013.03.008
  » https://doi.org/10.1016/j.archger.2013.03.008
• ¹²
  Atestado médico - prática e ética. São Paulo: Conselho Regional de Medicina do Estado de São Paulo, 2013.
• ¹³
  Santos TM. [Internet]. Questionário do AHA/ACSM para estratificação pré-participação em programas de atividades físicas; 2008 (acesso em 21 jan 2015). Disponível em: http://www.prohealth.com.br/ugf/lato_files/Screening-AHA-2008_Jul.pdf
  » http://www.prohealth.com.br/ugf/lato_files/Screening-AHA-2008_Jul.pdf
• ¹⁴
  Whitfield GP, Pettee Gabriel KK, Rahbar MH, Kohl HW 3rd. Application of the american heart association/american college of sports medicine adult preparticipation screening checklist to a nationally representative sample of US adults aged >=40 years from the National Health and Nutrition Examination Survey 2001 to 2004. Circulation 2014;129(10):1113-1120. Doi: 10.1161/CIRCULATIONAHA.113.004160
  » https://doi.org/10.1161/CIRCULATIONAHA.113.004160
• ¹⁵
  Riebe D, Franklin BA, Thompson PD, Garber CE, Whitfield GP, Magal M et al. Updating ACSM's recommendations for exercise preparticipation health screening. Med Sci Sports Exerc 2015;47(11):2473-9. Doi: 10.1249/MSS.0000000000000664
  » https://doi.org/10.1249/MSS.0000000000000664
• ¹⁶
  Whitfield GP, Riebe D, Magal M, Liguori G. Applying the ACSM preparticipation screening algorithm to U.S. Adults: National Health and Nutrition Examination Survey 2001-2004. Med Sci Sports Exerc 2017;49(10):2056-2063. Doi: 10.1249/MSS.0000000000001331
  » https://doi.org/10.1249/MSS.0000000000001331
• ¹⁷
  Arkin CF, Wachtel MS. How Many Patients Are Necessary to Assess Test Performance? JAMA 1990; 263:275-278.Doi: 10.1001/jama.1990.03440020109043
  » https://doi.org/10.1001/jama.1990.03440020109043
• ¹⁸
  Meneghelo RS, Araújo CGS, Stein R, Mastrocolla LE, Albuquerque PF, Serra SM et al. Sociedade Brasileira de Cardiologia. III Diretrizes da Sociedade Brasileira de Cardiologia sobre Teste Ergométrico. Arq Bras Cardiol 2010; 95(5 supl.1): 1-26. Doi: 10.1590/S0066-782X2010000800001
  » https://doi.org/10.1590/S0066-782X2010000800001
• ¹⁹
  Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1978;8:283-298.Doi: 10.1016/S0001-2998(78)80014-2
  » https://doi.org/10.1016/S0001-2998(78)80014-2
• ²⁰
  Ohman EM, Granger CB, Harington RA, Lee KL. Risk strati?cation and therapeutic decision making in acute coronary syndromes. JAMA 2000; 284(7): 876-878. Doi: 10.1001/jama.284.7.876
  » https://doi.org/10.1001/jama.284.7.876
• ²¹
  Lopes PR, Barbosa JPAS, Lima AHRA, Miranda AS, Rodrigues LBCC, Rodrigues SLC, et al. Triagem pré-participação em exercício físico em pacientes com doença arterial periférica. J Vasc Bras 2012;11(3):194-198. Doi: 10.1590/S1677-54492012000300005
  » https://doi.org/10.1590/S1677-54492012000300005
• ²²
  Fletcher GF, Ades PA, Kligfield P, Arena R, Balady BJ, Bittner VA, et al. Exercise standards for testing and training: a scientific statement from the American Heart Association. Circulation 2013;128(8):873-934. Doi: 10.1161/CIR.0b013e31829b5b44
  » https://doi.org/10.1161/CIR.0b013e31829b5b44
• ²³
  Humphrey R A, Lakomy J. An evaluation of pre-exercise screening questionnaires used within the health and fitness industry in the United Kingdom. Phys Ther Sport 2003;4(4):187-191. Doi: 10.1016/j.ptsp.2003.08.006
  » https://doi.org/10.1016/j.ptsp.2003.08.006
• ²⁴
  Banerjee A, Newman DR, Bruel A, Heneghan C. Diagnostic accuracy of exercise stress testing for coronary artery disease: a systematic review and meta-analysis of prospective studies. Int J Clin Pract 2012;66(5):477-492. Doi: 10.1111/j.1742-1241.2012.02900.x
  » https://doi.org/10.1111/j.1742-1241.2012.02900.x

Publication Dates

• Publication in this collection
  22 July 2019
• Date of issue
  2019

History

• Received
  27 Sept 2017
• Reviewed
  28 July 2018
• Accepted
  02 Feb 2019