Analysis of agreement between Visual Analogue Scales (VAS) and numerical questions to assess perception of teratogenic risks in treatment with drugs and radiotherapy in women

Análise de concordância entre as escalas visual analógica (VAS) e questões numéricas para avaliar a percepção de riscos teratogênicos em tratamento com medicamentos e radioterapia em mulheres

Abstracts

Objectives:

to evaluate the agreement between Visual Analogue Scales (VAS) and numerical questions as a way of assessing the perception of teratogenic risk of treatment with drugs and radiotherapy.

Methods:

the sample comprised 144 pregnant and 143 non-pregnant women consecutively recruited at public health centers in Porto Alegre, in the State of Rio Grande do Sul, from February to August 2011. The perception of risk for congenital malformations in the general population and the perceptions of teratogenic risk for exposure to acetaminophen, meto-clopramide, misoprostol and radiotherapy were measured using VAS and numerical questions. The agreement between the results of the two techniques was tested using a Bland-Altman plot.

Results:

the medians for the perceptions measured using VAS were higher than those obtained using numerical questions for all variables. The perception of risk for acetaminophen showed the lower bias of the two techniques (bias=13.17; p<0.001) and exposure to radiotherapy, the higher (bias=25.02; p<0.001).

Conclusion:

there was no agreement between the measurements obtained using the two techniques for any of the risk perceptions under study. Risk perceptions were higher using VAS, for all kinds of exposure. Studies should be conducted to assess whether there is overestimation in other situations and social contexts owing to the use of VAS.

Risk; Visual analogue scale; Pharmaceutical preparations; Teratogens

Objectivos:

avaliar a concordância entre Escalas Visuais Analógicas (EVA) e perguntas numéricas para aferir a percepção de risco teratogênico de medicamentos e radioterapia.

Métodos:

a amostra foi constituída por 144 gestantes e 143 não gestantes recrutadas consecutivamente em centros públicos de saúde de Porto Alegre, RS, entre fevereiro e agosto de 2011. A percepção de risco de malformações congênitas na população geral e as percepções de risco teratogênico das exposições a paracetamol, metoclo-pramida, misoprostol e radioterapia na gestação foram aferidas por EVA e perguntas numéricas. A concordância entre as duas técnicas foi avaliada pela análise gráfica de Bland-Altman

Resultados:

as medianas das percepções de risco teratogênico medidas por EVA foram superiores às obtidas através da pergunta numérica, para todas as variáveis. A percepção de risco ao paracetamol apresentou o menor viés entre as duas técnicas de aferição (viés=13,17; p<0,001) e a exposição à radioterapia, o maior (viés=25,02;p<0,001).

Conclusões:

não houve concordância entre as duas técnicas, para nenhuma das percepções de risco estudadas. As percepções de risco foram maiores para EVA, para todas as exposições. Sugerimos a realização de estudos que avaliem se também ocorre superestimação em outras situações e contextos sociais, em função do uso de EVA.

Risco; Escala visual analógica; Preparações farmacêuticas; Teratogênios

Introduction

Several studies have focused on the investigation of variables that influence the perception and assessment of risk by individuals.¹ These studies have demonstrated that the way in which individuals perceive risk is influenced by cultural, religious and individual variables, as well as by the perception of their own health. Additionally, risk perceptions may be distorted in people who have poor numerical skills.²

Although it is estimated that drug therapy is responsible for only 1% of all congenital defects of known etiology,³ studies have demonstrated that the perception of teratogenic risk related to medications is overestimated by expectant mothers, women and healthcare providers.^4-10 In a survey conducted in 295 Dutch women, 5% stated that pregnant women should not use any drugs at all.¹¹ Erroneous risk perception may prevent the pregnant woman from using safe medications and influence compliance with pharmacological treatment.^6,8

Most studies of the perception of teratogenic risk have used Visual Analogue Scales (VAS).⁴>⁶>¹²>¹³ However, the estimation of risk perception using VAS has limitations and disadvantages, since people tend to mark values located at the center of the scales and it is difficult to mark low values with accu-racy.^6,14 Alternatively, Nordeng et al.⁸ have proposed Numerical Rating Scales to measure the perception of teratogenic risk related to medications. No study that used other tools besides VAS and Numerical Rating Scales to assess perception of teratogenic risk of drug therapy. Studies that have assessed perception of risk of adverse drug reactions have also used VAS.^15-18

The present study aims to evaluate the agreement between VAS and numerical scales in assessing perception of the teratogenic risk related to medications and exposure to radiotherapy in women.

Methods

The study population comprised 287 women (144 pregnant and 143 non-pregnant) up to 49 years of age recruited consecutively at the prenatal and gynecological care services of three public health centers located in the city of Porto Alegre, Brazil.

The sample size was calculated for a larger study which intended to compare the perception of terato-genic risk related to medications and exposures between pregnant and non-pregnant women. The sample was estimated using data available in the literature. In a Norwegian study,⁸ 87.5% of the women correctly estimated the risk of congenital malformations in the general population (that is, the probability of a pregnant woman giving birth to a child with any major malformation). Thus, for purposes of calculation, the current study estimated that 70% of non-pregnant and 50% of pregnant participants would correctly assess the risk of congenital malformations in the general population. For 5% significance and 80% power, the estimated sample size was 103 women for each group.

The interviews were conducted from February to August 2011 by two trained interviewers with a degree in Pharmacy.

The data collection used two techniques to measure risk perception: the VAS and a question with a numerical answer. In addition to risk perceptions, socio-demographic data were also collected (age, self-reported color/race, marital status, schooling, occupation, and family income), along with data related to the respondents' obstetric history and history of family congenital anomalies. Assessment of perceived risk using both techniques was carried out in the same interview, which was conducted prior to prenatal or gynecological visits.

The VAS is a continuous, horizontal, 10-cm horizontal line with the leftmost extremity defined as 0 (no risk) and the rightmost extremity defined as 10 (maximum risk). The respondent is asked to mark the point on the line that corresponds to the perceived risk for the question "In a population of healthy pregnant women, what do you think the risk of a child being born with any physical or mental disability is on a scale from zero to ten?" Likewise, the perception of teratogenic risk was also measured for medications that are commonly used in pregnancy (acetaminophen and metoclopramide), miso-prostol (clandestinely used as an abortive agent, since abortion is illegal in Brazil) and exposure to radiotherapy during pregnancy. The question "In a population of healthy pregnant women, for every 100 babies that are born, how many do you think are born with some sort of physical or mental disability?" required a numerical response from the respondent for the general risk of congenital malformations. In an analogical manner, the perception of teratogenic risk for the above listed medications was also measured using the question: "Of every 100 babies that are born, how many do you think are born with some sort of physical or mental disability if the mother is taking_______(acetaminophen, metoclopramide or misoprostol). The perception of teratogenic risk for the exposure to radiotherapy during pregnancy was measured using the question: "Of every 100 babies that are born, how many do you think are born with some sort of physical or mental disability if the mother is being exposed to radiotherapy during pregnancy?". If the study participant did not know the answer, she could respond "don't know".

To enable comparison of the two measures, the value obtained using VAS was multiplied by ten. The data were digitized and checked using Teleform v.10.5. For analysis, the PASW Statistics v.18 software package was used. The normality of the frequency distribution of the variables was tested using the Shapiro-Wilk test. The asymmetrical variables were represented by the median and interquartile range. The categorical variables were represented by the absolute and relative frequency. The medians were compared using a signed ranked Wilcoxon test.

The agreement between the two techniques for measuring the perception of teratogenic risk was evaluated using a Bland-Altman plot.¹⁹ The bias (mean difference) between the two techniques was calculated using a paired t-test, and the agreement limits (AL) were obtained by the following equation:

AL = bias ± (1.96*SD),

where SD is the standard deviation of the difference between techniques.

The significance level adopted was 0.05. The degree of association between techniques was measured using Spearman's correlation coefficient.

The project was approved by the Research Ethics Committee of the Municipal Health Department of Porto Alegre (no. 001.039589.10.2). All respondents were informed about the study and signed an Informed Consent Form. If the participant was a minor, the consent of a legal guardian aged over 18 years was requested.

Results

The main socio-demographic characteristics of the sample are described in Table 1. Most respondents were white, married or living with their partner, had 9 to 11 years of schooling and had paid work.

Table 1
Sociodemographic characteristics of the interviewed women (15 to 49 years) (N=287)^a.

Characteristic	N	%
Age (years)b	28	(23 - 34)
Color/race
White	196	68.3
Non-white	91	31.7
Marital Status
Lives with partner	190	68.1
Does not live with partner	89	31.9
Schooling (years)
Up to 8	64	22.3
9 - 11	168	58.5
Over 11	55	19.2
Occupation
Paid job	179	62.4
Housewife	55	19.2
Unemployed	53	18.5
Parity
0 or 1 child	205	71.4
2 or more children	82	28.6
Family income (BRL)
≤ 1200.00	114	40.9
1201.00 - 2000.00	81	29.0
≥ 2001.00	84	30.1

• ^a
  The variation in the total number of participants in the categories results from absent data for each respective variable;
• ^b
  Median and interquartile range.

The medians and interquartile ranges for the perception of teratogenic risk, sorted by measuring technique, and the Spearman correlations between the techniques are shown in Table 2. The medians for the perception of teratogenic risk measured using VAS were higher than those obtained using the numerical question for all variables, with statistically significant differences (p<0.001). The lowest median for perception of teratogenic risk was for acetaminophen, while the highest was for misoprostol. The lowest Spearman correlation coefficient was for the risk of congenital malformations in the general population, while the highest was for metoclo-pramide.

Table 2
Median values and interquartile ranges of perceived teratogenic risk per measuring technique and Spearman's correlation between the techniques.

Medications	Visual Analogue Scale (VAS)			Numeric question			pc	VAS x Numeric question
	N	Mda	Q1- Q3b	n	Mda	Q1-Q3b		Spearman's Correlation Coefficient	p
General riskd	287	22.0	10 - 44	287	8.0	2 - 20	<0.001	0.334	<0.001
Acetaminophen	287	15.0	3 - 32	285	3.0	0 - 10	<0.001	0.640	<0.001
Metoclopramide	286	16.5	4 - 34.2	284	3.5	0 - 10	<0.001	0.699	<0.001
Misoprostol	281	84.0	57.5-96	280	50.0	10 - 90	<0.001	0.629	<0.001
Radiation therapy	285	78.0	53 - 92	285	42.0	10 - 80	<0.001	0.482	<0.001

• ^a
  Md= median;
• ^b
  Q1= first quartile; Q3= third quartile;
• ^c
  Comparison VAS versus numeric question by Wilcoxon test; ^dBaseline risk perception.

The Bland-Altman graphs show the degree of agreement of the perception of teratogenic risk between the techniques (Figure 1). The lowest bias between the techniques was for acetaminophen (bias = 13.17; p<0.001) and the highest for the exposure to radiotherapy (bias = 25.02; p<0.001).

Figure 1
The Bland-Altman plots comparing the two techniques of measuring the perception of teratogenic risk (Visual Analogue Scale x numeric question) for general risk (n=287).^a

U: Upper Agreement Limit; L: Lower Agreement Limit; *p<0.001 obtained from a paired t-test; ^aThe removal of extreme values did not substantially change the results.

Discussion

This is to our knowledge the first study that assesses the agreement between two measurement techniques for perception of teratogenic risk related to medications and exposure to radiation therapy. We did not find agreement between the risk perception measurements obtained by VAS and by numerical questions for any of the risk perceptions under study.

The results of the analysis of correlation between the techniques indicate a direct association between the VAS and the numerical question for all variables under study. It should be noted that the association between the techniques for risk perception related to acetaminophen, metoclopramide and misoprostol is strong, while for general risk and radiotherapy exposure it is moderate. It is also noteworthy that, although some studies still use correlation coefficients to assess the agreement between methods, these coefficients are not an agreement measure, but rather a measure of association.^19,20 Correlation coefficients assess the degree of relationship between variables, providing a figure that indicates how the variables vary from one another. ²⁰

Regarding the agreement between measurement techniques, the positive and statistically significant biases for all risk perceptions under study indicate, in addition to the lack of agreement, that the risk perceptions obtained using VAS were higher than those obtained using numerical questions.

The analysis of the perceptions of teratogenic risk of misoprostol and radiotherapy showed the highest ranges between the agreement limits and greatest biases, which indicated greater heterogeneity of responses to exposure to these treatments compared with others. Such heterogeneity can also be seen in the graphs, where two distinct patterns can be noted. For the perception of general risk, for acetaminophen and metoclopramide there is a greater concentration of responses next to the intersections between the axes, which is not observed for the exposure to radiotherapy or to misoprostol, in which there is greater result variability. The large concentration of answers close to the intersection between the axes may mean greater knowledge on the part of interviewees regarding the low teratogenic risk of acetaminophen, metoclopramide and the general risk for congenital malformations. Such knowledge can also be inferred from the proximity of medians to the correct value, established by the literature as 3%, ⁷,²¹ especially when the risk perception was assessed using a numerical question. The heterogeneity of the answers regarding risk perception related to misoprostol and radiotherapy may in turn indicate greater lack of knowledge about the actual teratogenic effects of these exposures, or a difficulty in accurately assessing this risk. Another possible explanation is that latter could be less well-known than the former, since acetaminophen and metoclopramide are widely used by the general population, including pregnant women. The high values of the medians for these exposures also provide evidence that the interviewees found it difficult to evaluate the teratogenic effects of exposure.

The fact that VAS leads to higher perceptions of teratogenic risk than the numerical question makes the means of the techniques for assessing general risk related to acetaminophen and metoclopramide a little higher than the true value of perception of teratogenic risk (estimated at 3%) for the points concentrated close to the intersections between the axes, that is, for those interviewees who were apparently aware of the low risk of these exposures. This higher risk perceptions using VAS in comparison to the numerical question does not have a great impact on exposure to radiation therapy and misoprostol, in which the high inter-subject variability demonstrates a lack of knowledge regarding the risk or difficulty in evaluating it. When the perception of teratogenic risk by the interviewee is correct and low, the VAS ends up overestimating the answer, as it is based on a spatial evaluation. This may lead investigators to make a false interpretation of risk overestimation. This may be one of the reasons why most studies of this subject have found overestimated perceptions of teratogenic risk.^4-6.⁸,¹³

In the setting under study, the bias values found have a substantial impact on the assessment of risk perception. For example, considering the general risk, the bias of which was 15.7, an interviewee who has given, for the answer, the figure of two malformed children in every hundred births (therefore, a perception within the range expected to be correct), will register a risk of approximately 17.7 on the VAS. Thus, the correct perception of terato-genic risk for one of the techniques would be higher for the other.

The lack of agreement between the two measurement techniques may indicate the difficulty the population under study experienced in understanding numerical concepts. VAS entails two mathematical abstractions--risk estimation and the indication of this value on a metric scale. For the numerical question, however, only one estimation skill is required. Other studies have pointed out that numeracy, one of the dimensions of literacy related to the ability to understand numbers,²²,²³ interferes with the understanding and appropriation of health information.² This difficulty understanding mathematical concepts is evident from the variability of the measurement techniques used.

The present study has a number of limitations. The decision to use different ranges for both measurement techniques may influence the results. The choice of using the endpoints 0 and 10 on the VAS seemed to be more adequate for populations in which school grades are traditionally measured in the decimal range. We therefore believe that providing a value between 0 and 10 for risk is closer to the daily life of respondents than assigning values between 0 and 100. On the other hand, the decision to use the range between 0 and 100 for the numerical question was due to the low prevalence of the event under study. If the question required the number of malformed newborns in a population of 10 pregnant women, we would probably obtain decimal responses to the question; otherwise it would not be properly understood. We did not randomize the order of the VAS and numerical questions, a procedure used to ensure that any effect found is not merely due to some influence of the first question on the way the participant answers the second.

Although the present study assessed only the agreement between the VAS and one other technique for measuring the perception of teratogenic risk, this scale has been widely used for the assessment of other health-related outcomes.^14-18,24 In this respect, it is important to conduct other studies to ascertain whether the differences found in this study also occur, as a result of the measurement technique used, in other situations and social contexts. Finally, we underscore the importance of using suitable analytical methods - such as the Bland-Altman graphic analysis - to assess the agreement between techniques, replacing correlation coefficients, which measure only the association.

Given that Visual Analogue Scales are broadly used for measuring health outcomes, we suggest that studies should be conducted to assess whether the differences found in this study also occur in other situations and social contexts as a result of the use of these scales.

References

• ¹
  Polifka JE, Faustman EM, Neil N. Weighing the risks and the benefits: a call for the empirical assessment of perceived teratogenic risk. Reprod Toxicol. 1997;11(4):633-40.
• ²
  Conover EA, Polifka JE. The art and science of teratogen risk communication. Am J Med Genet C Semin Med Genet. 2011;157(3):227-33.
• ³
  De Santis M, Straface G, Carducci B, Cavaliere AF, De Santis L, Lucchese A, Merola AM, Caruso A. Risk of drug-induced congenital defects. Eur J Obstet Gynecol Reprod Biol. 2004;117(1):10-9.
• ⁴
  Koren G, Bologa M, Long D, Feldman Y, Shear NH. Perception of teratogenic risk by pregnant women exposed to drugs and chemicals during the first trimester. Am J Obstet Gynecol. 1989;160(5 Pt 1):1190-4.
• ⁵
  Mazzotta P, Magee LA, Maltepe C, Lifshitz A, Navioz Y, Koren G. The perception of teratogenic risk by women with nausea and vomiting of pregnancy. Reprod Toxicol. 1999;13(4):313-9.
• ⁶
  Sanz E, Gómez-López T, Martínez-Quintas MJ. Perception of teratogenic risk of common medicines. Eur J Obstet Gynecol Reprod Biol. 2001;95(1):127-31.
• ⁷
  Bonari L, Koren G, Einarson TR, Jasper JD, Taddio A, Einarson A. Use of antidepressants by pregnant women: evaluation of perception of risk, efficacy of evidence based counseling and determinants of decision making. Arch Womens Ment Health. 2005;8(4):214-20.
• ⁸
  Nordeng H, Ystrøm E, Einarson A. Perception of risk regarding the use of medications and other exposures during pregnancy. Eur J Clin Pharmacol. 2010;66(2):207-14.
• ⁹
  Csajka C, Jaquet A, Winterfeld U, Meyer Y, Einarson A, Panchaud A. Risk perception by healthcare professionals related to drug use during pregnancy: a Swiss survey. Swiss Med Wkly. 2014;144:w13936.
• ¹⁰
  Cantilino A, Lorenzo L, de Paula JoA, Einarson A. Use of psychotropic medications during pregnancy: perception of teratogenic risk among physicians in two Latin American countries. Rev Bras Psiquiatr. 2014;36(2):106-10.
• ¹¹
  van Trigt AM, Waardenburg CM, Haaijer-Ruskamp FM, de Jong-van den Berg LT. Questions about drugs: how do pregnant women solve them? Pharm World Sci. 1994;16(6):254-9.
• ¹²
  Pole M, Einarson A, Pairaudeau N, Einarson T, Koren G. Drug labeling and risk perceptions of teratogenicity: a survey of pregnant Canadian women and their health professionals. J Clin Pharmacol. 2000;40(6):573-7.
• ¹³
  Koren G, Levichek Z. The teratogenicity of drugs for nausea and vomiting of pregnancy: perceived versus true risk. Am J Obstet Gynecol. 2002;186(5 Suppl Understanding):S248-52.
• ¹⁴
  Caughey AB, Washington AE, Kuppermann M. Perceived risk of prenatal diagnostic procedure-related miscarriage and Down syndrome among pregnant women. Am J Obstet Gynecol. 2008;198(3):333.e1-8.
• ¹⁵
  Montastuc JL, Bongard V, Lapeyre-Mestre M. Perception of the risk of gastrointestinal adverse drug reactions with non-steroidal anti-inflammatory drugs (including coxibs): differences among general practitioners, gastroenterologists and rheumatologists. Eur J Clin Pharmacol. 2003;59(8-9):685-8.
• ¹⁶
  Cullen G, Kelly E, Murray FE. Patients' knowledge of adverse reactions to current medications. Br J Clin Pharmacol. 2006;62(2):232-6.
• ¹⁷
  Durrieu G, Hurault C, Bongard V, Damase-Michel C, Montastruc JL. Perception of risk of adverse drug reactions by medical students: influence of a 1 year pharmacological course. Br J Clin Pharmacol. 2007;64(2):233-6.
• ¹⁸
  Durrieu G, Hurault C, Damase-Michel C, Montastruc JL. Perception of risk of adverse drug reactions: a 3-year follow-up of a cohort of medical students. Fundam Clin Pharmacol. 2010;24(4):423-7.
• ¹⁹
  Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307-10.
• ²⁰
  Hirakata VN, Camey AS. Análise de concordância entre métodos de Bland-Altman. Rev HCPA. 2009;29(3):261-8.
• ²¹
  Kalter H, Warkany J. Medical progress. Congenital malformations: etiologic factors and their role in prevention (first of two parts). N Engl J Med. 1983;308(8):424-31.
• ²²
  Peters E, Hibbard J, Slovic P, Dieckmann N. Numeracy skill and the communication, comprehension, and use of risk-benefit information. Health Aff (Millwood). 2007;26(3):741-8.
• ²³
  Valverde G, Nâslund-Hadle E. The State of Numeracy Education in Latin America and the Caribbean Inter-American Development Bank Technical Notes [Internet]; 2010. 47 p.
• ²⁴
  Abdel-Aziz E, Hassan IM, Al-Taher H. Assessment of women's satisfaction with medical termination of pregnancy. J Obstet Gynaecol.2004;24(4):429-33.

Publication Dates

• Publication in this collection
  Oct-Dec 2014

History

• Received
  26 Feb 2014
• Reviewed
  29 Aug 2014
• Accepted
  30 Sept 2014