Predictors of influenza vaccine uptake: translation into Portuguese and validation of a questionnaire

Abstract:

The difficulty in achieving ideal coverage rates with the influenza vaccine in Brazil and the growing wave of antivaccine movements in the world point to the need for a more in-depth understanding of the individual determinants of to this vaccine uptake. The Health Belief Model, a theoretical model that aims to explain and predict health-related behaviors, suggests that individual beliefs influence the adoption of health-related behaviors. The objective of this study was a cross-cultural adaptation of an instrument to assess predictors of influenza vaccine uptake in Brazilian adults. The authors conducted translation, back-translation, face validity, and a survey for construct validity. They also analyzed the factors associated with influenza vaccine uptake in 2017. An instrument originally with seven domains was identified and selected. In the factor analysis, four of the model’s seven constructs were validated: Susceptibility, Barriers, Cues to action, and Self-efficacy. In the survey with 396 persons, 59.3% reported having received the influenza vaccine in the last campaign in 2017. Female sex, age > 50 years, pregnancy, vaccination in private healthcare services, hepatitis B vaccination, and influenza vaccination prior to 2017 were associated with vaccination in 2017. In the final logistic regression model, perceived Barriers appeared as a strong factor for non-vaccination, while Cues to action increased the odds of vaccination.

Keywords:
Vaccination; Human Influenza; Patient Acceptance of Health Care; Psychological Models; Questionnaires

Resumo:

A dificuldade de alcançar coberturas ideais para a vacina contra influenza no Brasil e a crescente onda do movimento antivacina no mundo apontam a necessidade de aprofundar a compreensão dos determinantes individuais de adesão a essa vacina. O Modelo de Crenças em Saúde, um modelo teórico que objetiva explicar e predizer o comportamento em relação à saúde, sugere que crenças individuais influenciam a adoção de comportamentos relacionados a essa área. Este trabalho teve como objetivo a adaptação transcultural de um instrumento para avaliar os preditores de aceitação da vacina da influenza em adultos no Brasil. Realizaram-se a tradução, a retrotradução, a validade de face e um inquérito para validade de construto. Também foram analisados os fatores associados à adesão à vacina da influenza em 2017. Um instrumento, originalmente com sete domínios, foi identificado e selecionado. Na análise fatorial, quatro dos sete construtos do modelo teórico foram validados: Suscetibilidade, Barreiras, Estímulos para a ação e Motivação para a saúde. No inquérito das 396 pessoas, 59,3% relataram vacinação contra influenza na última campanha de 2017. Sexo feminino, idade > 50 anos, gestante, vacina na rede privada, vacinação contra a hepatite B e influenza antes de 2017 mostraram-se como fatores associados à vacinação em 2017. No modelo logístico final, a percepção de Barreiras apresentou-se como um forte Estímulo para não vacinação, ao passo que Estímulos para a ação atuou aumentando a chance de vacinação.

Palavras-chave:
Vacinação; Influenza Humana; Aceitação pelo Paciente de Cuidados de Saúde; Modelos Psicológicos; Questionários

Resumen:

La dificultad de alcanzar coberturas ideales para la vacuna contra la gripe en Brasil y la creciente ola del movimiento antivacunas en el mundo apunta la necesidad de profundizar la comprensión de los determinantes individuales de adhesión a esa vacuna. El Modelo de Creencias en Salud, un modelo teórico que tiene como objetivo explicar y predecir el comportamiento en relación con la salud, sugiere que las creencias individuales influencian la adopción de comportamientos relacionados con esa área. El objetivo de este trabajo es la adaptación transcultural de un instrumento para evaluar los predictores de aceptación de la vacuna de la gripe en adultos en Brasil. Se realizó la traducción, retrotraducción, validez de la presentación, así como una encuesta para la validez del constructo. También se analizaron los factores asociados con la adhesión a la vacuna de la gripe en 2017. Un instrumento, originalmente con siete dominios, fue identificado y seleccionado. En el análisis factorial, cuatro de los siete constructos del modelo teórico fueron evaluados: Susceptibilidad, Barreras, Estímulos para la acción y Motivación para la salud. En el cuestionario a las 396 personas, 59,3% informaron haber sido vacunadas contra la gripe en la última campaña de 2017. Sexo femenino, edad > 50 años, embarazada, vacunada en la red privada, vacunación contra la hepatitis B y gripe antes de 2017 se mostraron como factores asociados a la vacunación en 2017. En el modelo logístico final, la percepción de Barreras se presentó como un fuerte Estímulo para la no vacunación, al paso que Estímulos para la acción actuó aumentando la oportunidad de vacunación.

Palabras-clave:
Vacunación; Gripe Humana; Aceptación de la Atención de Salud; Modelos Psicológicos; Cuestionarios

Introduction

Brazil has invested heavily in the implementation of annual influenza vaccination campaigns since 1999. According to data from the National Immunization Program (PNI), the coverage reached by these campaigns is on target, but heterogeneities need to be investigated, since prevalence studies report unsatisfactory adherence levels ^1,2 while others have identified a trend towards the growth of antivaccine movements in the country ³.

The Health Belief Model (HBM) ^4,5, used in studies to assess behaviors involving adherence to the influenza vaccine ^6,7,8,9,10, suggests that individual beliefs can influence the adoption of health-related behaviors ⁵.

Following a review of the international literature, the instrument developed initially by Blue & Valley ¹⁰ and later used by Shahrabani et al. ⁸, hereinafter “BVS”, was chosen to measure adherence behavior to the influenza vaccine using the HBM in an adult population. Blue & Valley based their work on studies by Champion ^11,12, who developed an instrument using HBM to assess uptake of breast self-examination and mammography. Blue & Valley selected the relevant items from the literature ^13,14,15 for their questionnaire based on the HBM and included additional predictors, “knowledge” and “self-efficacy for health”, based on Strecher & Rosenstock ¹⁵. Although some instruments cited above were assessed for their psychometric characteristics ¹⁴, a detailed psychometric assessment of the instrument chosen by us has not been performed to date.

The choice of the BVS was based on some observations, such as: its use in at least four previous publications ^10,16,17,18, closed questions, and a target population consisting of healthy adult workers (similar to the current study’s target population). In addition, the original questionnaire by Blue & Valley ¹⁰ that gave rise to the BVS presents reasonable test-retest reliability in its dimensions according to Pearson’s correlation.

The BVS questionnaire contains 46 items and was authorized for our use by Shahrabani et al. ⁸. The questionnaire has the following dimensions of HBM: Susceptibility, Severity, Benefits, Barriers, Cues to action, Knowledge, and Self-efficacy for health ^4,8,10. The responses to the items are measured by a 5-point Likert scale (1 = strongly agree; 2 = agree; 3 = neither agree or disagree; 4 = disagree; 5 = strongly disagree).

Little is known about the individual barriers and facilitators associated with influenza vaccine uptake by the Brazilian population, perhaps due partly to the lack of an instrument in Portuguese that would allow investigating these aspects. This main objective of this study was a cross-cultural adaptation of the BVS instrument to study the association between adherence to the vaccine and the dimensions of the HBM. The study describes the stages of translation, back-translation, face validity, and construct validity of the BVS questionnaire, in addition to analysis of the factors associated with the influenza vaccine uptake in 2017.

Methods

Cross-cultural adaptation

The BVS questionnaire underwent a semantic cross-cultural adaptation for use in Brazil, according to the stages recommended by Beaton et al. ¹⁹. Cross-cultural adaptation is necessary to achieve equivalence between the original version and the translated version of the questionnaire and guarantee comparability between international studies. The stages were: translation, face validity, and construct validity. For face validity and construct validity, the questionnaire was applied in digital format to a sample of Brazilian adults recruited via social networks, and correlation and exploratory factor analyses were performed, in addition to analysis of internal consistency of the questionnaire adapted to Brazilian Portuguese.

Translation and back-translation

The instrument’s translation involved the following stages:

(1) Independent translation by three translators (T1, T2, and T3) with proficiency in English and Portuguese, two of whom were members of the research team (C.T.C. and P.M.L.) with knowledge of the concepts and theme examined in the questionnaire and one (K.M.) with no technical knowledge in the health field;

(2) Synthesis of the translations (version T-123) with a face-to-face meeting to reach a consensus with the translators among the three translations, C.R.N. (responsible for conducting the study), and L.M.T.G., with experience in elaborating questionnaires and with HBM, who played the role of methodological expert (a researcher with experience in validations);

(3) Independent back-translation by two translators without knowledge of the original version or of the concepts involved, whose mother tongue was English and who were fluent in Portuguese and knowledgeable of Brazilian culture, one of whom was an undergraduate medical student at the University of California in Los Angeles (United States) who had been a research exchange student at Oswaldo Cruz Foundation (Fiocruz; Rio de Janeiro, Brazil) and another who was a graduate student in Literature from the University of Cambridge (United Kingdom), working as a translator in Brazil and also as press director for a British school in São Paulo;

(4) Meeting of an expert committee including four of the research team members (C.R.N., C.T.C., P.M.L., and L.M.T.G.), in addition to three outside participants: K.M., D.B., and A.D. (A.D. is the manager of a vaccination unit who had more than 10 years’ experience in immunizations). Before the meeting, participants received files with the original questionnaire, the translated versions, consensus version T-123, the back-translations, and instructions for the evaluation of each item’s equivalence, comparing the original version and the back-translated version, assigning scores from 1 to 3 (1 = not equivalent, 2 = more or less equivalent, 3 = equivalent);

(5) Consolidation of the questionnaire in Portuguese, aimed to develop the pre-final version of the questionnaire to be tested in the field. This stage was documented in detail concerning the questions and reasons for consensus decisions.

Face validity

To verify clarity of the items in the questionnaire, the pre-final version was elaborated in Google Forms (https://docs.google.com./forms/) digital format and sent electronically through an electronic mailing list to 43 health professionals. Recruitment was done through social networks and aimed at maximum regional and occupational diversity to contemplate Brazil’s regional linguistic differences. The participants thus comprised a convenience sample with a minimum of one year of work in training-related activity. All participants received a free and informed consent form and confirmed their agreement to participate through the Google Forms digital platform. Health professionals were asked to assess each item on the questionnaire on a scale of 1 to 10, considering the categories: confusing (1 to 3), unclear (4 to 7), and clear (8 to 10), with space reserved for suggestions and criticisms. This phase of the study was approved by the Institutional Review Board of the Fiocruz under protocol n. 1.807.327 and CAAE n. 56087116.9.0000.5240. Adjustments were made to items that were considered unclear, resulting in the questionnaire’s final version, hereinafter “BVSb” (Brazilian version of the BVS).

Construct validity

For validation of the constructs in the BVSb questionnaire, the latter was used in a survey conducted from September to December 2017 after the annual influenza vaccination campaign that had been held from April 17 to May 26. The target population was Brazilian adults 18 years and older. A Facebook page was created to explain the project’s objectives, and push notification was used to expand its visibility. Recruitment was done broadly through non-directional Facebook push notification. The instrument used in the survey contained the 45 items from the BVSb questionnaire, in addition to questions related to the history of influenza vaccination and other vaccines, sociodemographic questions, and health-related questions. The choice of social networks to send the questionnaire rather than using hard copies (as done by Shahrabani et al. ⁸) aimed to facilitate recruitment of the study’s target population.

The survey’s data analysis began with tabulation of the study participants’ sociodemographic and health characteristics, stratifying them between those vaccinated versus not vaccinated in 2017. Chi-square test was used to test the associations between the characteristics and the flu vaccine’s uptake in 2017, considering statistical significance at 5% as an indicator of association between variables.

Next, as a further step in exploratory analysis, a correlation matrix was calculated among the 45 items answered in the BVSb questionnaire, using Pearson’s correlation. This first required converting the Likert scale into a numerical scale from 1 to 5. Items belonging to the same construct are expected to be correlated with each other, but weakly correlated or not correlated with items belonging to other constructs. The next stage consisted of an exploratory factor analysis aimed at describing the items’ variance and covariance according to the seven factors as proposed in the theoretical model. We used two methods to evaluate the validity of the exploratory factor analysis: the Kaiser Meyer-Olkin (KMO) index and Bartlett’s sphericity test ²⁰. The KMO index, also known as the measure of sampling adequacy, informs on the proportion of the items’ variance that can be explained by a latent variable. Bartlett’s sphericity text assesses the degree to which a covariance matrix is similar to an identity matrix ²¹. The analysis used function fa() from the psych library available in the R environment (http://www.r-project.org), which uses least-squares methods to find the solution with the least residuals, and orthogonal rotation was assumed (varimax; oblique rotation was also considered, with very similar results) along with principal axis analysis ²². Interpretation of the results considered that the construct had acceptable evidence of validity when at least three items from the theoretical model presented loadings above the threshold of 0.50, assumed as the criterion for pertinence. In some situations, factor analysis suggested that an item should be moved from one construct to another. In others, the construct was not represented well by the items, suggesting its inadequacy as part of the instrument. The model’s fit was assessed with the comparative fit index, defined as the ratio between the difference between the 7-factor model’s chi-square statistic and degrees of freedom and the null model’s chi-square statistic and degrees of freedom. After determining each construct and its respective items based on factor analysis, each construct’s internal validity was calculated using Cronbach’s alpha ²².

Logistic regression

To analyze the association between the explanatory variables and uptake of the vaccine in 2017 in the set of participants in the validation survey, a logistic regression model was adjusted using the vaccination status reported in 2017 as the outcome. For each validated construct, scores were built based on the mean of the responses to the items belonging to each construct, considering the pertinence of items proposed by factor analysis. Descriptive variables that were associated with vaccination according to the chi-square test were also included in the bivariate regression model. The adjusted model only considered the dimensions of HBM, the demographic variables (age and sex), and formal recommendation for vaccination (health professionals and pregnant women). The logistic regression model’s fit was evaluated according to the Akaike information criterion (AIC). All the analyses were performed in the R environment using the plyr, stringr, epiDisplay, corrplot, psy, and psych libraries.

Results

Translation and back-translation

All the intermediate translation versions and the comments from each expert meeting and the participants in face validity are available in Neves ²³. The translations were generally quite similar, only varying in the verb tenses and in the use of more versus less formal wording. For example, “getting the flu” was translated as “catching the flu” or “contracting influenza”. The general preference was for informal wording that would not lead to two interpretations, such as “catching the flu”. The items that generated more debate were those in which the original version suggested a very strong effect of flu on the person’s life. For example: “If I get the flu, my job would be in serious danger.” In this case, translating “danger” as “perigo” (in Portuguese) was considered an exaggeration, and the option was “my work could be jeopardized”. Other decisions involved adapting items to the current Brazilian scenario, for example, the item “I got the flu vaccine because my doctor or nurse said it was good for me” was translated as “I got the flu vaccine because a health professional recommended it”, generalizing to all the health professions. During this stage of the translation, doubts on the interpretation of some items were also resolved in consultation with the original article’s lead author ⁸.

Face validity

Forty-three health professionals participated, of whom 83.7% were females, 51.2% were 30 to 39 years of age, 53.5% were born in the state of Rio de Janeiro, and the rest were born in other states of Southeast and South Brazil. Place of residence for 70% of the participants was Rio de Janeiro, and the vast majority had specialization courses (51.2%) or Master’s or PhDs (30.2%).

In this evaluation, 35 of the 45 items (77.7%) were considered clear and 10 (22.2%) unclear. No item was considered confusing. Some adjustments were made to the unclear items, based on suggestions from the participants, with preference for words and terms that were better at capturing the general population’s daily living experience. In the item “Getting the flu vaccine would keep me from missing work”, the expression keep from was replaced with the expression decrease the odds, and in the item “Getting the flu vaccine is not convenient for me,” the word convenient was replaced with the more colloquial easy. After these changes, made by the research team (C.R.N., C.T.C., and P.M.L.), the translated version of the BVS questionnaire was considered ready for use. Table 1 shows this final version of the questionnaire (BVSb).

Table 1
Items in the BVSb questionnaire.

	Distribution of items according to theoretical model	Label	F1	F2	F3	F4	F5	F6	F7	Items after factor analysis
Susceptibility	Working with many people every day increases my odds of catching the flu.	susc1	-0.084	0.550	-0.014	-0.083	0.014	0.051	0.307	Maintain	>Susceptibility: 0.77
	Only people over 60 catch the flu.	susc2	0.001	-0.009	0.056	0.049	0.093	0.146	-0.189	Exclude
	I have high odds of catching the flu.	susc3	0.001	0.741	0.114	-0.045	0.148	-0.017	0.071	Maintain
	Healthy people can catch the flu.	susc4	-0.076	0.426	0.075	-0.013	-0.005	0.081	0.271	Exclude
	I think the odds are high of my catching the flu in the near future.	susc5	-0.013	0.823	0.100	-0.055	0.118	0.056	-0.034	Maintain
	I worry a lot about the possibility of catching the flu.	susc6	-0.133	0.109	-0.042	0.122	0.657	0.052	0.152	Exclude
	I’m going to catch the flu in the next year.	susc7	0.073	0.437	-0.071	0.161	0.083	-0.029	0.011	Exclude
Severity	Thinking I can catch the flu scares me.	seve1	-0.016	0.105	0.028	0.104	0.733	0.008	0.091	Exclude	Exclude
	If I caught the flu it could hurt my job.	seve2	0.079	-0.069	-0.175	0.013	0.345	0.090	-0.151	Exclude
	If I caught the flu it could hurt my family.	seve3	-0.010	0.143	0.001	0.073	0.440	0.154	0.226	Exclude
	Catching the flu would make daily activities more difficult.	seve4	-0.025	0.161	0.113	0.057	0.101	0.074	0.648	Exclude
	If I caught the flu it would be more serious than other diseases.	seve5	0.071	0.018	0.071	0.125	0.449	0.141	0.011	Exclude
	The flu can be a serious disease.	seve6	-0.162	0.091	0.083	-0.005	0.302	0.158	0.508	Exclude
Benefits	Getting the flu vaccine will keep me from catching the flu.	bene1	-0.042	-0.082	0.057	0.211	0.107	0.460	-0.087	Exclude	Exclude
	Getting the flu vaccine will protect the people living with me from catching the flu.	bene2	-0.030	0.087	0.049	0.118	0.136	0.410	0.220	Exclude
	Getting the flu vaccine will decrease the odds of missing work.	bene3	-0.144	0.104	-0.030	0.122	0.107	0.493	0.079	Exclude
	I have much to gain from getting the flu vaccine.	bene4	-0.514	0.046	0.050	0.204	0.266	0.283	0.307	Move
	I’m not very afraid of catching the flu if I get the flu vaccine.	bene5	-0.260	-0.031	0.027	0.219	0.092	0.306	0.030	Exclude
	Having a chronic disease (like diabetes, heart disease, or asthma) is a good reason to get the flu vaccine.	bene6	-0.305	0.107	0.191	0.001	0.102	0.279	0.270	Exclude
Barriers	Getting the flu vaccine is not easy for me.	barr1	0.538	-0.031	0.016	-0.106	-0.097	-0.134	-0.145	Maintain	Barriers: 0.81
	To get the flu vaccine, I’d have to go without a lot of things.	barr2	0.684	-0.071	-0.012	0.025	0.105	0.143	-0.164	Maintain
	Getting the flu vaccine can be painful.	barr3	0.492	0.076	0.005	-0.067	-0.030	0.050	0.086	Exclude
	Getting the flu vaccine would take too much time for me.	barr4	0.571	0.017	0.024	-0.042	-0.024	0.040	-0.052	Maintain
	Getting the flu vaccine interferes in my daily activities.	barr5	0.731	-0.089	-0.010	-0.026	0.073	0.203	-0.011	Maintain
	There are many risks associated with the flu vaccine.	barr6	0.687	-0.053	0.057	-0.044	0.058	-0.179	0.057	Maintain
	Getting the flu vaccine costs too much.	barr7	0.316	0.142	-0.027	0.027	-0.018	-0.015	-0.103	Exclude
	I get worried about having a reaction to the flu vaccine.	barr8	0.574	0.058	-0.005	0.090	0.071	-0.264	0.070	Maintain
Cues to action	I decided to get the flu vaccine when I saw an ad for the campaign	cues1	-0.111	-0.082	-0.001	0.554	0.153	0.003	0.056	Maintain	Cues to action: 0.73
	I got the flu vaccine because a friend or family member told me it was important.	cues2	0.037	0.123	-0.052	0.652	0.035	0.112	0.000	Maintain
	I got the flu vaccine because a health professional recommended it.	cues3	-0.072	0.063	0.041	0.656	0.010	0.118	0.065	Maintain
	I got the flu vaccine because my boss thought it would be a good idea.	cues4	0.190	-0.005	0.009	0.309	0.077	0.124	-0.069	Exclude
	I got the flu vaccine after hearing information on the media about the vaccine’s benefits.	cues5	-0.121	-0.035	0.082	0.613	0.194	0.121	-0.079	Maintain
Knowledge	People get the flu from eating or drinking from other people with the flu.	know1	0.158	0.291	-0.012	0.130	-0.127	0.234	0.138	Exclude	Exclude
	People catch the flu from breathing the same air as other people with the flu.	know2	0.015	0.141	0.135	0.124	0.034	0.188	0.014	Exclude
	The flu lasts three to five days.	know3	0.025	0.182	0.129	0.180	-0.081	0.066	0.075	Exclude
	The flu can cause a more serious disease, like pneumonia.	know4	-0.069	0.151	0.115	0.001	0.057	0.020	0.259	Exclude
	A person can catch the flu by getting the flu vaccine.	know5	0.441	-0.084	-0.027	0.078	-0.050	-0.358	-0.015	Exclude
	People often get sick when they get the flu vaccine.	know6	0.528	-0.011	-0.014	0.121	-0.023	-0.335	-0.009	Move
Self-efficacy	I have a balanced diet.	effi1	0.098	-0.029	0.516	0.089	-0.144	0.132	0.082	Maintain	Self-efficacy: 0.69
	I follow doctor’s instructions because I believe they’re good for my health.	effi2	-0.240	0.075	0.197	0.138	0.109	0.060	0.165	Exclude
	I often do things on my own to improve my health.	effi3	0.047	0.203	0.494	0.139	-0.147	-0.025	0.080	Exclude
	I research new information related to my health.	effi4	-0.026	0.064	0.679	-0.029	0.152	0.069	0.096	Maintain
	I do regular preventive tests besides seeing the doctor when necessary.	effi5	-0.129	-0.018	0.618	-0.028	0.188	-0.062	0.054	Maintain
	I have regular dental checkups besides dental visits for specific problems.	effi6	-0.049	0.038	0.584	-0.005	0.047	-0.042	0.016	Maintain
	I exercise regularly, at least 3 times a week.	effi7	0.057	-0.027	0.449	-0.035	-0.151	0.101	-0.047	Exclude

• Note: colors identify the constructs according to the original theoretical model: Susceptibility, Severity, Benefits, Barriers, Cues to action, Knowledge, and Self-efficacy. Columns F1 to F7 show the item loading in each of the 7 factors in the exploratory factor analysis with values greater than 0.5, marked in bold, considered indicative of the factor’s pertinence. Original distribution (left) and final distribution (right). Last column on the right shows the internal consistency measured by Cronbach’s alpha for the four final constructs.

Construct validity

The survey for the construct validity included participation by 407 persons. The analysis only considered the answers from 396 persons, since 9 persons were excluded due to contraindication to vaccination, 1 person failed to specify whether she (or he) had a contraindication to vaccination, and another was only 17 years old. Participants were mostly women (75%), 18-50 years of age (74.5%), and born and living in Southeast Brazil (64.7% and 70.8%, respectively). Of the total, 39.7% reported having no religion, and among those who had a religion, most were Catholic (30.1%) (Table 2). The sample consisted mostly of persons with more than secondary schooling, and 67.1% had graduate degrees. Slightly over half were married or in a stable union (55.7%) and had children (51%). Family income was greater than 5 times the monthly minimum wage in 72.5% of the sample, 37.6% were public employees, or employees with or without signed work papers (21.6%). A large share consisted of health professionals (41.8%), and most had private health insurance (82.3%) (Table 2).

Table 2
Characteristics of participants in the questionnaire’s validation and influenza vaccine uptake in 2017.

Characteristics	Total	No (%)	Yes (%)	Statistical test	p-value
Total	396	160	235
Sex				χ2 (1 df) = 4.36	0.037
Male	98	49 (50.0)	49 (50.0)
Female	297	111 (37.4)	186 (62.6)
Age bracket (years)				χ2 (1 df) = 12.23	< 0.001
(18,50]	292	133 (45.5)	159 (54.5)
(50+	100	25 (25.0)	75 (75.0)
Region of birth in Brazil				χ2 (4 df) = 2.3	0.68
Central	62	29 (46.8)	33 (53.2)
North	9	3 (33.3)	6 (66.7)
Northeast	51	21 (41.2)	30 (58.8)
South	16	8 (50.0)	8 (50.0)
Southeast	253	97 (38.3)	156 (61.7)
Region of current residence				Fisher’s exact test	0.772
Central	58	24 (41.4)	34 (58.6)
North	7	1 (14.3)	6 (85.7)
Northeast	39	16 (41.0)	23 (59.0)
South	10	4 (40.0)	6 (60.0)
Southeast	277	113 (40.8)	164 (59.2)
Race				Fisher’s exact test	0.755
Yellow	9	3 (33.3)	6 (66.7)
White	247)	100 (40.5)	147 (59.5)
Indigenous	1	1 (100.0)	0 (0.0)
Brown	96	37 (38.5)	59 (61.5)
Black	28	13 (46.4)	15 (53.6)
Religion				χ2 (4 df) = 16.17	0.003
None	153	77 (50.3)	76 (49.7)
Catholic	116	33 (28.4)	83 (71.6)
Spiritist	57	23 (40.4)	34 (59.6)
Evangelical	45	13 (28.9)	32 (71.1)
Other	14	7 (50.0)	7 (50.0)
Schooling				χ2 (2 df) = 4.43	0.109
Secondary	62	32 (51.6)	30 (48.4)
Undergraduate university	67	23 (34.3)	44 (65.7)
Graduate	263	103 (39.2)	160 (60.8)
Marital status				χ2 (4 df) = 1.25	0.869
Married/Stable union	219	86 (39.3)	133 (60.7)
Single (never married)	119	48 (40.3)	71 (59.7)
Other	17	9 (52.9)	8 (47.1)
Separated/Divorced	30	12 (40.0)	18 (60.0)
Widow/Widower	8	3 (37.5)	5 (62.5)
Children?				χ2 (1 df) = 0.97	0.325
No	193	83 (43.0)	110 (57)
Yes	184	69 (37.5)	115 (62.5)
Family income (minimum wage)				χ2 (2 df) = 0.31	0.858
Up to 2 times	22	10 (45.5)	12 (54.5)
2 to 5	77	32 (41.6)	45 (58.4)
More than 5	266	106 (39.8)	160 (60.2)
Current work				χ2 (5 df) = 23.01	< 0.001
Public employee	148	47 (31.8)	101 (68.2)
Employer/Self-employed	45	32 (71.1)	13 (28.9)
Scholarship holder	51	21 (41.2)	30 (58.8)
Employee with/without work papers	85	31 (36.5)	54 (63.5)
Not currently working	48	21 (43.8)	27 (56.2)
Other	17	7 (41.2)	10 (58.8)
How many persons living in household?				χ2 (4 df) = 5.07	0.28
1	43	14 (32.6)	29 (67.4)
2	120	51 (42.5)	69 (57.5)
3	129	49 (38.0)	80 (62.0)
4	67	34 (50.7)	33 (49.3)
5+	32	11 (34.4)	21 (65.6)
Health professional?				χ2 (1 df) = 48	< 0.001
No	230	127 (55.2)	103 (44.8)
Yes	165	33 (20.0)	132 (80.0)
Teacher?				χ2 (1 df) = 0.01	0.919
No	355	143 (40.3)	212 (59.7)
Yes	40	17 (42.5)	23 (57.5)
Pregnant?				Fisher’s exact test	0.024
No	387	160 (41.3)	227 (58.7)
Yes	8	0 (0.0)	8 (100.0)
Comorbidities?				χ2 (1 df) = 1.16	0.281
No	373	154 (41.3)	219 (58.7)
Yes	22	6 (27.3)	16 (72.7)
Private health plan?				χ2 (1 df) = 1.91	0.167
No	70	34 (48.6)	36 (51.4)
Yes	325	126 (38.8)	199 (61.2)
Ever had hepatitis B vaccine?				χ2 (2 df) = 22.98	< 0.001
Zero doses	46	23 (50.0)	23 (50.0)
Incomplete schedule	39	25 (64.1)	14 (35.9)
Yes	244	70 (28.7)	174 (71.3)
Had flu vaccine before?				χ2 (1 df) = 84.24	< 0.001
No	75	66 (88.0)	9 (12.0)
Yes	320	94 (29.4)	226 (70.6)
Ever been vaccinated in the private system?				χ2 (2 df) = 12.93	0.002
No	238	105 (44.1)	133 (55.9)
Don’t remember	19	13 (68.4)	6 (31.6)
Yes	137	42 (30.7)	95 (69.3)

• Note: p-value refers to the chi-square test of independence between variables.

Influenza vaccination coverage in the study population in 2017 was 59.5% (235/396), and the great majority of these (80%) belonged to some target public for the national campaign (health professionals, pregnant women, age over 60 years, or persons with comorbidities). Concerning vaccination history, 71.3% had a complete hepatitis B scheme, 70.6% reported having been vaccinated for influenza prior to 2017, and 69.3% had used the private healthcare system for vaccination at some time.

The variables associated with influenza vaccination in 2017 were: female sex, age > 50 years, Catholic or Evangelical religion, public employee, health professional, pregnancy, history of hepatitis B vaccination, history of influenza vaccination before 2017, and having been vaccinated in the private system (Table 2).

Figure 1 shows the Pearson’s correlation among the 45 items in the BVSb questionnaire, calculated from answers to the survey. There was a strong correlation among all the items belonging to Susceptibility except for the items susc2 (“Only persons over 60 years of age catch the flu”) and susc6 (“I’m very concerned about the possibility of getting the flu”). The latter actually showed a stronger association with the items in Severity. There was also a significant correlation among all six items in Severity, except seve2 (“If I caught the flu, it could compromise my work”), which was not associated with any other item. We found strong correlation among all the items in Benefits, and the same was true for Barriers. Interestingly, Benefits and Barriers were inversely correlated. All the items in Cues to action were correlated except cues4 (“I got the flu vaccine because my boss thought it would be a good idea”), which was not associated with any other item. Knowledge was the construct with the least correlation among its items, except for items know5 (“A person can catch the flu when they’re vaccinated for the flu”) and know6 (“People often get sick when they’re vaccinated for the flu”). Finally, Self-efficacy showed good correlation among all its items except effi2 (“I follow doctor’s instructions because I think they’re good for my health”).

Figure 1
Correlation matrix of the questionnaire’s 45 items.

This initially suggests the existence of six well-demarcated constructs: Susceptibility, Severity, Benefits, Barriers, Cues to action, Self-efficacy, and a poorly defined construct (Knowledge). It also suggests that some items can be moved between the constructs or even removed.

First, in relation to the validity of exploratory factor analysis, the KMO index was 0.71, which is considered good. Bartlett’s sphericity test showed a p-value of < 0.001, allowing to reject the null hypothesis that the data matrix is similar to an identity matrix.

In the factor analysis, four of the seven factors hypothesized by the theoretical model presented 3 or more items with loadings > 0.50 (Table 1). The comparative fit index was 0.88. Factor 1 included 6 of the 8 items originally associated with the Barriers construct, in addition to one item from the Knowledge construct (“People often get sick when they get the flu vaccine”) and one item from the Benefits construct (“I have a lot to gain from getting the flu vaccine”, with the inverted Likert scale). The low loading in the item “The flu vaccine is too expensive” may be due to the fact that the vaccine is supplied free of cost in Brazil and is thus not an important barrier in the country. The item that captured the barrier of local pain, “Getting the flu vaccine can be painful” also showed borderline loading, suggesting that this is a less important barrier for the study population.

Factor 2 included three items in the Susceptibility construct, while the other 4 items were excluded. The three that were maintained suggest an individual perception of acquisition of the virus that is not expressed definitively (as in the excluded item “I’m going to catch the flu next year”) or mixed with excessive concern (“I’m very worried about the possibility of catching the flu”). The other two excluded items suggest a discourse of “the other” (“Only persons over 60 years of age catch the flu” and “Healthy persons can catch the flu”) may be misrepresenting perceived individual susceptibility.

Factor 3 included 4 of the 7 items from Self-efficacy that addressed individual behavior towards diet (“I have a balanced diet”), overall health (“I have regular preventive tests and see the doctor when necessary”), dental health (“I have regular dental appointments, besides seeing the dentist for specific problems”), and the search for new health information (“I research new information related to my health”). The item “I follow doctor’s instructions because I think they’re good for my health” showed low loading, possibly because it evaluated the individual’s agreement with what had been suggested by a physician, more than the individual’s own self-efficacy for care. The other two excluded items suggested more formal regularity in health-related acts, and perhaps they presented lower loading as a result.

Factor 4 included all the items in Cues for action except the item “I got the flu vaccine because my boss thought it would be a good idea”, suggesting that the “boss” is not necessarily a person that makes routine suggestions related to health behaviors, or that his suggestions are followed. The remaining factors did not show strong convergence, so the Severity, Benefits, and Knowledge constructs were not validated.

Table 1 also shows the final distribution of items across the theoretical model’s seven constructs, based on the factor analysis. With the four constructs, the comparative fit index was 0.91. The internal consistency of all four final constructs was satisfactory (Cronbach’s alpha > 0.6), with the highest value for the construct perceived Barriers (Cronbach’s alpha = 0.81) and the lowest for Self-efficacy (Cronbach’s alpha = 0.69).

Logistic regression

Scores were calculated for each participant on perceived Susceptibility, Barriers, Cues to action, and Self-efficacy (the four validated constructs) and for the mean values of the answers on the items belonging to each construct (inverting the scale when applicable). Figure 2 shows the distribution of these scores in the survey sample. This was a population with high perceived susceptibility (quantiles 25; 50; 75 = 3.28; 3.71; 4.14), high self-efficacy (quantiles 25; 50; 75 = 3.33; 3.66; 4.00), and low perceived barriers (quantiles 25; 50; 75 = 1.83; 2.25; 2.58).

Figure 2
Boxplot showing the median and quartiles in the distribution of scores among the study participants for each of the four constructs validated from the BVS model.

Table 3 shows the odds ratio of flu vaccine uptake in 2017 in this population. Two models are shown, without and with adjustment variables (sex, age, health professional, pregnancy). These covariables improved the model’s fit according to the AIC but did not modify the constructs’ effect. Perceived barriers appeared as a strong stimulus for not vaccinating, as expected, and were an important inhibitor to vaccination. Meanwhile, the construct Cues to action showed a significant positive association with vaccine uptake, that is, the more recommendations people receive from health professionals, family members, or the media, the higher the odds of vaccination. The constructs susceptibility and self-efficacy did not reach significance.

Table 3
Means and 95% confidence intervals for scores of validated constructs in the Health Belief Model and its association with influenza vaccine uptake in 2017 measured by odds ratio (OR).

	Score			Without covariables			With covariables
	Mean	2.5%	97.5%	OR	2.5%	97.5%	OR	2.5%	97.5%
Barriers	2.00	0.76	3.25	0.20	0.13	0.31	0.21	0.13	0.33
Susceptibility	3.70	1.93	5.47	0.85	0.66	1.10	0.80	0.60	1.07
Self-efficacy	3.72	2.30	5.13	1.14	0.83	1.56	1.02	0.71	1.46
Cues to action	2.80	1.03	4.56	1.30	1.00	1.70	1.44	1.08	1.93
Sex (females)							1.25	0.70	2.22
Age							1.02	1.00	1.05
Health professional/Pregnant							6.25	3.63	10.75
AIC				455.56			404.67

• AIC: Akaike information criterion.
• Nota: two models presented, one with only the constructs from the Health Belief Model and the other which also included the covariables sex, age, and target group (health professional or pregnant woman).

Discussion

The previous lack of Brazilian questionnaires with evidence of validity to assess influenza vaccine uptake motivated the current study. The questionnaire that was chosen, BVS, has the HBM as its theoretical basis, and among the questionnaires identified in the literature, it was the one with the most progress in the validation process. Adaptation of the questionnaire to Brazilian Portuguese also required cross-cultural adaptation. This process identified different perceived meanings that allowed better adjustment of the wording and expressions to constitute the questionnaire’s items, resulting in a comprehensible and consistent final version, that is comparable to the original version.

Although we initially proposed seven constructs, only four displayed evidence of validity in the current study, nearly all with a reduction in the number of items, two of which proved statistically associated with flu vaccine uptake in the 2017 campaign. Barriers, consisting of seven items, showed the greatest internal consistency. Among the items, several aspects of psychological barriers were represented, ranging from a perception of waste of time to the perception of possible risks. However, two originally proposed items were not kept in the final construct (“getting the flu vaccine can be painful” and “the flu vaccine costs too much”), suggesting low influence of local pain and cost as barriers in this sample. This finding makes sense in the Brazilian context, where the vaccine is supplied free of cost to a major share of the population by the Unified National Health System (SUS). Meanwhile, the susceptibility construct was reduced to only three of the original items, which deal objectively with the individual (when compared to persons in general, item susc4 excluded) and reflect a degree of uncertainty (when compared to item susc7, which suggests certainty, also excluded). Our interpretation is that items containing impersonal sentences need to be reworded. Meanwhile, overly emphatic or superlative items such as “I am very worried about catching the flu” may be exaggerated in the case of influenza, which most of the population views as a minor problem. This would also explain the lack of validation of the severity construct, consisting of items suggesting that influenza would jeopardize one’s “job”, “family”, and “daily activities”. The item “I got the flu vaccine because my boss thought it would be a good idea” was excluded from the construct Cues to action suggesting the need to improve this item’s cultural adequacy. The concept of “boss” may not be perceived the same way as in the original proposed item. In addition, part of the study population was not working (12.2%) and thus would not be able to receive this kind of recommendation. Self-efficacy measures issues pertaining to the individual’s own health behavior, so the item “following doctor’s instructions” did not adequately fit the idea. Likewise, no sufficient representation was found for frequent physical activity and the search for actions to improve health, suggesting that the “regularity” achieved by the study population is lower than suggested in the items or that in fact the study population does not practice regular physical activity and other activities in general.

The evaluation of the association between the four constructs with evidence of validity and influenza vaccine uptake in 2017 revealed perceived barriers as an important inhibitor of vaccination. Various other studies have also identified this effect, associated with low vaccine uptake ^{7,10,16,17,18,24,25}. These studies adopt a broad definition of “barriers”, which can range from fear of the vaccine and possible adverse reactions to barriers involving time, place, and cost.

The Cues to action construct increased the odds of vaccination, showing that advertisements, information in the mass media, and recommendations by health professionals and friends or family members help increase influenza vaccine uptake. Mo & Lau ⁷ found that government recommendations were positive for vaccination, in the form of recommendations in the mass media and advertisements. Meanwhile, Avery & Lriscy ²⁶ found that stimulus for vaccination was associated with communication via social networks. Corace et al. ²⁷ found higher vaccination rates among individuals whose family members and friends thought that vaccination was important, demonstrating that other people can positively influence vaccination behavior. However, the stimulus factor that shows the strongest association with influenza vaccination is the recommendation by a health professional ^6,7,8,10,27.

In our study sample, increasing age was identified as a predictor of influenza vaccination. In two recent systematic reviews, increasing age was consistently identified as a predictor of influenza vaccine uptake ^28,29. The gender variable, which our study found to be associated with vaccine uptake in the bivariate analysis, did not show consistent results in the literature. Some studies showed male gender more associated with vaccine, while in others study it was female gender, and in still others there was no association between gender and vaccination ^24,28,29. The association we detected may have resulted from the sample’s bias (mostly females and from the health field), or it may actually have been a correct result, suggesting a difference from studies in North America and Europe, which comprise a large share of the findings in the systematic reviews cited above. A systematic review in Brazil ¹, although not focused primarily on factors associated with vaccine uptake, did not highlight sociodemographic factors as predictors of adherence. Prior behaviors in relation to influenza vaccine and other vaccines (hepatitis B in this case) were also associated with vaccination in 2017. Finally, health professionals and pregnant women also contributed to vaccination, corroborating Luz et al. ²⁵, who showed that belonging to a group in which the vaccine is highly recommended was a strong predictor of influenza vaccine uptake in 2016-2017 in a U.S. sample. We found in the multiple regression model, which considered the four constructs of HBM, belonging to a group in which the vaccine is highly recommended greatly increased the odds of vaccination.

In our study, susceptibility was not associated with vaccination. Other studies involving the same profile of adult participants without comorbidities (only 5.6% reported comorbidities) also showed no association of this dimension of HBM and influenza vaccination ^7,17. The low correlation of some items resulting exclusively from four of them in the exploratory factor analysis may have contributed to the dimension’s poor performance as a whole and thus low understanding of them by the participants. In fact, studies suggest that the susceptibility construct is quite broad ^30,31. In a meta-analysis of the effect of perceived risk, authors grouped the susceptibility dimension as a risk dimension, with this subdivided in three dimensions: perceived susceptibility to an infection, the probability of harm, and the degree of harm ³⁰. Another study suggested that perceived risk has affective, decision-making, and experiential dimensions ³¹. In short, more studies are necessary to better define and assess susceptibility.

As study limitations, our sample involved voluntary participation in the questionnaire’s validation, which ended up selecting individuals with a similar profile, mostly persons with more schooling, health professionals, and residents of the state of Rio de Janeiro, a profile similar to that of the study’s researchers. These results suggest that the participants belong to the upper stratum of the Brazilian population in terms of income, schooling, and job stability. The presence of numerous health professionals also suggests a sample with different health-related knowledge and attitudes than the general population. In addition, the cross-sectional design meant that the questions and outcome (vaccination in 2017) were collected at the same time (after the campaign). Ideally, the questions would have been asked before the campaign, and the individuals would have been contacted again at the end of the campaign to learn whether or not they had been vaccinated. The use of the digital format for completion and the social networks to recruit participants for constructs’ validation had the advantage of decreasing the time and financial costs involved in conducting the study. The disadvantage was the selection of a specific population with access to computers, tablets, or cellphones and that use social networks, in this case Facebook.

Conclusion

Given the current scenario of growing discussion on the antivaccine movement and the difficulties in achieving adequate vaccination coverages, we provide the final modified Brazilian version of the questionnaire (BVSc) for use in new studies. Given the majority participation by health professionals in the study’s validation stages, we found that the questionnaire was better adapted and validated for application in this target public. We particularly highlight the need for future studies with more representative samples of the Brazilian population, aimed at confirmation of the constructs proposed here, using confirmatory factor analysis and other techniques. Reliable and validated questionnaires are extremely important for capturing health information, contributing to comparison of the results between Brazilian and international studies. As far as we know, the BVSc is the instrument with the best documentation of the construction and validation for use in behavioral studies on uptake to the seasonal influenza vaccine in the adult population. Thus, there are points that need to be developed in future studies, such as reproduction of the study in populations with different characteristics in order to prove the validity proposed in the constructs and the development of alternative items for constructs not validated in the current study.

Acknowledgments

The authors wish to thank Fiotec for research scholarship 61587.2 under the VPEIC-002-FIO-15 Project and CNPq for research scholarship 305553/2014-3 and post-doctoral scholarship 150213/2016-6.

Referências

• ¹
  Luna EJA, Gattás VL, Campos SRSLC. Effectiveness of the Brazilian influenza vaccination policy: a systematic review. Epidemiol Serv Saúde 2014; 23:559-75.
• ²
  Bós ÂJG, Mirandola AR. Cobertura vacinal está relacionada à menor mortalidade por doenças respiratórias. Ciênc Saúde Colet 2013; 18:1459-62.
• ³
  Larson HJ, de Figueiredo A, Xiahong ZX, Schulz WS, Verger P, Johnston IG, et al. The state of vaccine confidence 2016: global insight through a 67-country survey. EBioMedicine 2016; 12:295-301.
• ⁴
  Janz NK, Becker MH. The Health Belief Model: a decade later. Health Educ Q 1984; 11:1-47.
• ⁵
  Champion VL, Skinner CS. The Health Belief Model. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: theories, research, and practice. San Francisco: Jossey-Bass; 2008. p. 45-65.
• ⁶
  Lau JTF, Au DWH, Tsui HY, Choi KC. Prevalence and determinants of influenza vaccination in the Hong Kong Chinese adult population. Am J Infect Control 2012; 40:e225-7.
• ⁷
  Mo PKH, Lau JTF. Influenza vaccination uptake and associated factors among elderly population in Hong Kong: the application of the Health Belief Model. Health Educ Res 2015; 30:706-18.
• ⁸
  Shahrabani S, Benzion U, Yom Din G. Factors affecting nurses' decision to get the flu vaccine. Eur J Health Econ 2009; 10:227-31.
• ⁹
  Matsui D, Shigeta M, Ozasa K, Kuriyama N, Watanabe I, Watanabe Y. Factors associated with influenza vaccination status of residents of a rural community in Japan. BMC Public Health 2011; 11:149.
• ¹⁰
  Blue CL, Valley JM. Predictors of influenza vaccine. Acceptance among healthy adult workers. AAOHN J 2002; 50:227-33.
• ¹¹
  Champion VL. Instrument development for health belief model constructs: advances in nursing. Science 1984; 6:73-85.
• ¹²
  Champion VL. Revised susceptibility, benefits, and barriers scale for mammography screening. Res Nurs Health 1999; 22:341-8.
• ¹³
  Nichol KL, Hauge M. Influenza vaccination of healthcare workers. Infect Control Hosp Epidemiol 1997; 18:189-94.
• ¹⁴
  Centers for Disease Control and Prevention. Influenza vaccine information. http://www.cdc.gov/ncidod/diseases/flu/fluvac.htm (acessado em Abr/2002).
  » http://www.cdc.gov/ncidod/diseases/flu/fluvac.htm
• ¹⁵
  Strecher VJ, Rosenstock IM. The Health Belief Model. In: Glanz K, Lewis FM, Rimer BK, editors. Health behavior and health education: theory, research and pratice. 2nd Ed. San Francisco: Jossey-Bass; 1997. p. 41-59.
• ¹⁶
  Teitler-Regev S, Shahrabani S, Benzion U. Factors affecting intention among students to be vaccinated against A/H1N1 influenza: a Health Belief Model approach. Adv Prev Med 2011; 2011:353207.
• ¹⁷
  Shahrabani S, Benzion U. Workplace vaccination and other factors impacting influenza vaccination decision among employees in Israel. Int J Environ Res Public Health 2010; 7:853-69.
• ¹⁸
  Shahrabani S, Benzion U. How experience shapes health beliefs: the case of influenza vaccination. Health Educ Behav 2012; 39:612-9.
• ¹⁹
  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000; 25:3186-91.
• ²⁰
  Dziuban CD, Shirkey EC. When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychol Bull 1974; 81:358-61.
• ²¹
  Tabachnick B, Fidell L. Using multivariate analysis. Boston: Allyn & Bacon/Pearson; 2007.
• ²²
  Kline RB. Principles and practice of structural equation modeling. 4th Ed. Montreal: Guilford Press; 2016.
• ²³
  Neves CR. Instrumentos de avaliação da adesão à vacina contra influenza sazonal: revisão da literatura e adaptação para uso em profissionais de saúde brasileiros [Dissertação de Mestrado]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2017.
• ²⁴
  Yeung MPS, Lam FLY, Coker R. Factors associated with the uptake of seasonal influenza vaccination in adults: a systematic review. J Public Health (Oxf) 2016; 38:746-53.
• ²⁵
  Luz PM, Johnson RE, Brown HE. Workplace availability, risk group and perceived barriers predictive of 2016-17 influenza vaccine uptake in the United States: a cross-sectional study. Vaccine 2017; 35:5890-6.
• ²⁶
  Avery EJ, Lariscy RW. Preventable disease practices among a lower SES, multicultural, nonurban, U.S. community: the roles of vaccination efficacy and personal constraints. Health Commun 2014; 29:826-36.
• ²⁷
  Corace K, Prematunge C, McCarthy A, Nair RC, Roth V, Hayes T, et al. Predicting influenza vaccination uptake among health care workers: what are the key motivators? Am J Infect Control 2013; 41:679-84.
• ²⁸
  Schmid P, Rauber D, Betsch C, Lidolt G, Denker M-L. Barriers of influenza vaccination intention and behavior: a systematic review of influenza vaccine hesitancy, 2005-2016. PLoS One 2017; 12:e0170550.
• ²⁹
  Nguyen T, Henningsen KH, Brehaut JC, Hoe E, Wilson K. Acceptance of a pandemic influenza vaccine: a systematic review of surveys of the general public. Infect Drug Resist 2011; 4:197-207.
• ³⁰
  Brewer NT, Chapman GB, Gibbons FX, Gerrard M, McCaul KD, Weinstein ND. Meta-analysis of the relationship between risk perception and health behavior: the example of vaccination. Health Psychol 2007; 26:136-45.
• ³¹
  Ferrer RA, Klein WMP, Persoskie A, Avishai-Yitshak A, Sheeran P. The Tripartite Model of Risk Perception (TRIRISK): distinguishing deliberative, affective, and experiential components of perceived risk. Ann Behav Med 2016; 50:653-63.

Publication Dates

• Publication in this collection
  26 Oct 2020
• Date of issue
  2020

History

• Received
  17 Nov 2018
• Reviewed
  31 Jan 2020
• Accepted
  07 Feb 2020