Validation of the Brazilian Version of the Screening Tool for Psychosocial Distress (Stop-D) for Cardiac Patients

Abstract

Background:

Distress has a potentiating effect on complications of heart disease. Early identification of distress and psychosocial management could help patients to deal with the disease and improve their quality of life. However, in Brazil, there is no specific instrument for evaluation of distress in cardiac patients.

Objectives:

To describe the validation process of the Screening Tool for Psychosocial Distress (STOP-D) for the Brazilian population.

Methods:

Cross-sectional, observational study with a quantitative approach. A total of 144 patients (including outpatients and inpatients) were interviewed at the waiting room of the outpatient cardiology clinic or in cardiology wards. Sociodemographic and clinical data were collected, and distress was assessed using two instruments - the Brazilian version of the STOP-D and the Hospital Anxiety and Depression Scale (HADS). First, we performed an exploratory factor analysis and analysis of the accuracy of the STOP-D score by the receiver operating characteristic (ROC) curvet.

Results:

The factorability analysis of the correlation matrix did not detect any factor that made the factorial solution unfeasible. The instrument showed a single-factor nature, confirmed by the criterion of eigenvalues, with an 85% accuracy in predicting distress. A cut-off point of 15.5 was chosen for distress using the ROC curve.

Conclusions:

The Brazilian version of the STOP-D is an adequate instrument for the screening of heart disease patient for distress. It can be easily used by any health professional and would contribute to the promotion of a comprehensive support to cardiac patients.

Keywords:
Stress,Psychological; Health Delivery; Cardiovascular Diseases; Social Conditions; Quality of Life; Anxiety; Depression

Introduction

Regardless of the disease stage, a cardiac patient is in an “alert state”, and a negative perception of illness may lead to a high level of distress that affects treatment response and disease progression.¹

In 1976, Selye proposed different definitions for positive and negative stress, naming them as “eustress” and “distress”, respectively. Eustress refers to a bearable level of stress; motivation is present in spite of threatening factors. It can be described as the ideal level of stress, compatible with the psychosocial resources of an individual. On the other hand, distress refers to an overload, i.e., an unbearable level of stress related to suffering towards the need of adaptation; when experienced for a long time, distress can cause exhaustion and make the individual more vulnerable to psychological disorders.^2,3

In clinical practice, five implications of distress have been recognized - difficulty in coping with illness, changes in emotional well-being, physical and emotional discomfort, verbal and non-verbal cues of discomfort, in addition to individual damages that may become permanent.³

Some risk predictors for distress can be found in the literature, including female sex,^4-6 poor social support,⁷ lack of access to specialized support (psychologists and/or psychiatrists),⁸ and type D personality (negative affectivity),⁹ low socioeconomical status and lower educational attainment.^4,7

Distress is recognized as an unpleasant emotional change which, when experienced for a long time, leads to an increase in sympathetic nervous system activity. Increased levels of distress-related hormones can affect the cardiovascular system and worsen the prognosis of the patients.¹⁰ In addition, distress has been associated with fatigue level,¹¹ increased risk of rehospitalization,¹² higher mortality rate,^7,13,14 and poor cardiac prognosis.^15,16 Thus, distress can potentiate the complications of heart disease.

Denollet, Schiffer and Spek¹⁵ describe depression, anxiety, anger and posttraumatic stress as specific markers of distress. Besides, the authors report an association of distress with a poor prognosis and decompensation of heart failure.

A strategy to promote a comprehensive support to patients is to perform the screening for distress using specific, sensitive tools. Systematic evaluation and early identification of distress can promote adequate intervention that requires individual’s confrontation that facilitates the whole process.¹⁶ Young et al.,¹⁷ developed, in St. Paul’s Hospital, Canada, a screening tool for distress called Screening Tool for Psychological Distress (STOP-D). The instrument provides risk scores for five items - depression, anxiety, stress, anger, and low social support. STOP-D is a brief, free instrument that can be self-administered, and applied in both outpatient and inpatient settings.^17,18

The Brazilian Society of Cardiology recognizes that emotional repercussions of heart diseases as comorbidities of these conditions. It also highlights the importance of a regular psychosocial evaluation and screening of heart disease patients for unfavorable psychosocial conditions.^19,20 However, so far, there is no instrument for distress screening adapted and validated to the Brazilian population, making cultural adaptation and statistical validation of well-established international instruments necessary.^21,22 Therefore, the aim of the present study was to describe the validation process of the Screening Tool for Psychosocial Distress (STOP-D) to the Brazilian population.

Methods

Study design and ethical aspects

This was a cross-sectional, observational study with a quantitative approach performed at Hospital das Clínicas da Universidade Federal de Goiás (HC-UFG), between August and December 2016. After approval by the local ethics committee, data collection was started at the outpatient cardiology department of the Hospital das Clínicas da Universidade de Goiás (HC-UFG), and at the internal medicine ward.

Participants

Patients older than 18 years attending the outpatient cardiology department and patients hospitalized at the Division of Cardiology of the Hospital das Clínicas da Universidade Federal de Goiás (HC-UFG), regardless of heart disease diagnosis and time of diagnosis were considered eligible for the study. Patients with hearing, oral communication or cognitive impairment that could affect their ability to answer to the assessment tools were not included. A total of 144 patients were included, and all of them signed an informed consent form.

Instruments

• Sociodemographic questionnaire: developed specifically for the study, for characterization of the study population. The following data were collected - sex, age, place of residence, marital status, number of children, educational attainment, occupational status, income and religion/beliefs.

• Clinical questionnaire: developed specifically for the study to gather information about the diagnosis of cardiac disease that was the cause of hospitalization or outpatient follow-up, time of diagnosis, number of hospitalizations due to the cardiac condition, comorbidities and patients’ perception about their health status.

• Brazilian version of the STOP-D for heart disease patients (appendix): adapted from the English language version¹⁷ to Portuguese language by evaluation of the target population and back-translation. The cross-cultural adaptation was performed according to the International Test Commission (UTC) guidelines.²² We obtained the permission of the first author of the STOP-D, Dr. Quincy Young, to have it translated and adapted to the Brazilian culture. This process was conducted by four independent translators, specialists in the field of health psychology. After two translation steps, the Brazilian version of the STOP-D was administered in four patients with different educational levels at the outpatient department and wards. These patients answered the questionnaire and evaluated the instrument. Finally, the back-translation was performed by two independent translators, who translated the instrument from Portuguese to English. Both translated versions were sent to Dr Quincy Young, for her analysis. The steps of evaluation of the instrument by the target public and of back-translation are essential for adequate adaptation of the instrument, as they guarantee both conceptual and idiomatic equivalence.^21,22 The Brazilian version of the STOP-D is composed of five items that evaluate markers of distress - depression, anxiety, stress, anger, and quality of social support - each one rated on a 10-point (0 to 9) ordinal scale. This instrument was developed to be used by an interprofessional staff, and it is brief and free to use.

• Hospital Anxiety and Depression Scale (HADS): 14-item scale that evaluates anxiety and depression using a Likert-type scale. The instrument can also be used as a single-factor tool to measure distress, using a score ≥ 15 as the cut-off.²³ The HADS has a mean application time of four minutes, good specificity and sensitivity, and is free to use.²⁴

Data analysis

Participants were invited to answer the questionnaire at the waiting room while waiting to be seen by the physician, or on the bed in case of inpatients. The instrument was answered with the help of one of the investigators, who is a qualified researcher in psychology, who also explained the aims of the study. All participants signed an informed consent form.

Statistical analysis

Descriptive and inferential statistics were performed using the Statistical Package for the Social Sciences (SPSS) software, version 22. The prerequisites for performing multivariate analysis (atypical data, missing data, normal distribution of the variables, multicollinearity, linearity, homoscedasticity and singularity). Decisions on inclusion and exclusion of data based were also made on recommendations by Tabachnick & Fidell.²⁵ The Kolmogorov-Smirnov test was used to verify the normality of the Brazilian version of the STOP-D scores and the HADS. The present study was divided in two phases:

First phase: Evaluation of psychometric characteristics of STOP-D for validity evidence by exploratory factor analysis. The method consists in describing the correlation structure between variables based on the number of non-observable variables (latent variables).^26,27

The initial sample was randomly divided into two subgroups, of approximately the same number, using a random number generator. The number of both subgroups was sufficient to perform factor analysis, since a proportion of ten participants to each item of the scale was maintained.^25,26 Formation of these subgroups made it possible to perform both an exploratory factor analysis (n = 69 patients) to evaluate the psychometric characteristics of the instrument and a confirmatory factor analysis (n = 75), to assess the stability of the factorial structure of the Brazilian version of the STOP-D.²⁶

Factor analysis can be divided into four stages: (1) factorability analysis of the correlation matrix, performed by adequacy of the variance caused by the sample (KMO -Kaiser-Meyer-Olkin)²⁵ and correlation between variables by the Bartlett’s test of sphericity tests; (2) determination of the number of factors to be extracted, performed based on factor retention criteria - factors that overcome the variance, (i.e., eigenvalues > 1.00) were maintained.²⁷ The number of factors was confirmed by analysis of internal consistency using the Cronbach’s alpha (> 0.70);²⁵ (3) extraction of factors, conducted by factor rotation, in which the values of the highest factor loadings are put in evidence. However, in a single-factor analysis, it is not necessary to perform rotations, but rather use the criteria for maintenance of the variables.²⁷ For the present study, factor loadings greater than 0.40 were considered for analysis; and (4) interpretation of the factor - in this phase, the factorial structure obtained was compared with the theoretical model proposed.

Although the exploratory analysis allows a higher degree of leniency, we adopted the same criteria to both exploratory and confirmatory analyses.

Second phase: (n = 144) analysis of the STOP-D score accuracy by the ROC (Receiver Operating Characteristics) curve, using the HADS as reference. The STOP-D score calculation was made by summation of the scores, which allows rapid corrections for future applications. A HADS ≥ 15 was used as cut-off.

The level of significance was set at 5% (0.05).

Results

A total of 114 patients aged from 18 to 84 years (mean of 55.85 ± 14.55 years) were studied. Most participants were women (n = 85, 59.0%). Ninety-four (65.3%) lived in the city the hospital was located. Regarding the marital status, only 55 (38.20%) were single.

With respect to educational attainment, most participants had some elementary education (n = 66, 45.8%) and 31 (21.5%) had completed high school. The source of income for most patients was the pension (n = 61, 42.4%), with a monthly income of up to one minimum wage (n = 62; 41.3%). Most patients reported to have religious beliefs (n = 129, 89.7%) and religious practice was reported by 95 patients (66.0%). Most patients were seen at the outpatient department (n = 122, 84.7%); the majority had cardiac arrhythmia (n = 51; 35.4%), followed by heart failure (n = 34, 23.6%). Eighty-eight (n = 61.1%) were older than 60 years, and 72 (50.0%) had comorbidities.

Properties of the Brazilian version of the STOP-D

For validation of an instrument, a minimum of five patients is required per variable. Therefore, for validation of the Brazilian version of the STOP-D, only 25 patients would be needed, but we opted for a larger sample to perform concurrent validation.²⁸ Construct validity was established by exploratory and confirmatory factor analysis (Table 1).

Table 1
Psychometric values - factorial loading and communalities of confirmatory and exploratory analysis of the Brazilian version of the STOP-D

Item	Confirmatory		Exploratory
	Factorial loading	Communality	Factorial loading	Communality
Feeling sad, down, or uninterested in life?	0.664	0.441	0.751	0.564
Feeling anxious or nervous?	0.564	0.318	0.720	0.518
Feeling stressed?	0.772	0.596	0.782	0.612
Feeling angry?	0.710	0.503	0.668	0.435
Not having the social support (family members and friends) you feel you need?	0.483	0.234	0.544	0.296
Eigenvalue	2.092		2.425
Explained variation (%)	41.84		48.503
Internal consistency coefficient (α)	0.780		0.726

• α: Cronbach's alpha: in the exploratory analysis, 69 heart disease patients were evaluated.

Factorability analysis of the correlation matrix did not reveal any factor that would make the factorial solution unfeasible. The KMO identifies sampling adequacy, with values ranging from 0.728 to 0.729. TEB showed significant differences (p < 0.001) between correlation matrix and identity matrix, corroborating the evidence of factorability of the matrix.

As predicted by the theory, the instrument showed a single-factor nature, confirmed by the criteria of eigenvalues, in which only the first component of the scale had a value greater than 1. Reliability index of the instrument corroborated the choice of a unique factor, ranging from 0.782 to 0;726 (Table 1).

The answers of all patients (n = 144) were used for the analysis of sensitivity and specificity of the STOP-D, as well as the cut-off for detection of distress by the instrument; the results were measured using the ROC curve. The adoption of a cut-off of 15 resulted in a specificity greater than 92.9% and specificity greater than 32.2% (Figure 1). In the present study, we obtained an area under the ROC curve of 0.85%, representing 85% of accuracy. Therefore, the Brazilian version of the STOP-D showed a diagnostic ability of 85%.

Figure 1
ROC curve showing the cut-off point and respective sensitivity and specificity values.

Discussion

The development and the adaptation of international instruments have emerged as strategies for the understanding of psychosocial factors associated with health-disease process. As mentioned by Duarte et al.,²⁹ in Brazil there are few standardized instruments that evaluate psychosocial aspects of patients with chronic diseases. In addition to limited access to these instruments, the inclusion of multidimensional signs (e.g. distress and pain) as important variables of the clinical course is still recent. However, increasing evidence has indicated a direct relationship of psychosocial aspects with disease development and prognosis.^7,18,30,31

Translation is a complex, accurate process, beyond the mere translation of the instrument items. In the present study, translation was successfully performed, in terms of developing an adapted version that was adequate and proportional to the original version, considering cultural variations and language adequacies. In addition, in this process, cultural, idiomatic, linguistic and contextual aspects were considered, as recommended in the literature.^21,22 At the end of this phase, a simple, brief instrument was developed, consistent with the Brazilian population reality. Finally, with the back-translation process, a final version of the instrument was developed, with conceptual consistency comparable with that of the original version. These findings indicate that both translation and adaption of the STOP-D to the Brazilian culture met important quality criteria.²²

It is worth mentioning that distress encompasses not only a psychological concept; in the international scenario, the screening for distress is performed by other professionals, including nurses and physicians. In Brazil, the role of psychologists in hospitals has expanded; these professionals have been increasingly involved in the translation, adaptation, and validation processes of international instruments. This promotes the understanding of the phenomena aspects and communication with the health staff by means of clear and objective data and evidence-based measures.^21,22,32

The access to a good instrument, with adequate psychometric characteristics is determinant for an effective screening, and one of the main steps of this process. With the Brazilian version of the STOP-D, we obtained not only a general score for distress, but also a time-effective instrument, in conformity with the concept of distress described in the literature.^3,17,18 The ROC curve was used to analyze this score and the cut-off point of 15 yielded an excellent sensitivity (> 92.9%). A good sensitivity was a priority, since it contributes to an effective identification of individuals at a low risk of psychosocial distress. This points out the need for referral psychosocial support services, since once individuals at risk for distress are identified by the instrument, they should be referred for complementary interprofessional care.

Regarding the statistic validation of the Brazilian version of the STOP-D, its exploratory and confirmatory validations also corroborated the quality of the instrument. In these processes, psychometric properties of the instrument were assessed by factor analysis, which identified the stability of the tool. Also, the factorability indicators obtained in the factor analysis supported the proposal of using the instrument as a general distress score.

Finally, we found a high area under the ROC curve, indicating an 85% accuracy of the Brazilian version of the STOP-D in identifying distress. According to Margotto,³³ an area above 0.70 indicates a satisfactory performance. These findings demonstrate that the instrument evaluated in the present study has good sensitivity and accuracy in detecting distress in cardiac disease patients.

The Brazilian public health faces a deficit in human resources and increased patient demand, in addition to a limited number of multidisciplinary professionals. For this reason, the structuration of services by psychosocial screening models seems a beneficial strategy, able to promote equity. Here we present a low-cost and fast instrument with high sensitivity and high accuracy for screening of distress in patients with heart diseases that can be of help in the treatment of this population.

Limitations of the study include the use of a convenience rather than a probabilistic sample. Clinical information was not collected from patients’ medical record but reported by the patients. This may represent a bias in the quality of the information, as it depends on patients’ ability to recall and educational level. In the original study, other screening instruments were used and compared with the STOP-D, while in the present study, only the HADS was used in this regard. Although it is true that these limitations may have influenced the results, they did not invalidate our study, which succeeded in achieving the objectives proposed.

Conclusions

In heart disease patients, distress is associated with worsening of the clinical course, and in this scenario, the Brazilian version of the STOP-D would be of great help in promoting adequate full support to these patients. The first topic to be explored is the concept of distress, which should receive an interprofessional approach to early detect and efficiently treat this condition. Second, the screening for psychosocial demands should be implemented in routine clinical practice, using simple, free instruments, that are accessible to all healthcare providers involved. Finally, patients with a high level of distress should be referred to a psychosocial support team.

In this context, more interdisciplinary professionals - psychologists, dietitians, social assistants, physiotherapists, among others - should be added to the healthcare team and establish an efficient communication with cardiologists. The Brazilian version of the STOP-D can also act as a facilitator in this process, as it can be an indicator of the quality of the interprofessional support, guiding and measuring team interventions. Therefore, this strategy is believed to improve the quality of life of the patients.

The psychometric analysis of the Brazilian version of the STOP-D confirmed that this is an appropriate instrument to measure distress. The translated, adapted, and validated version of the tool also maintained the properties of the original version (no item of the original version was excluded). Another positive characteristic of the Brazilian version of the STOP-D was its single-factor nature (cut-off point of 15 for distress). This finding contrasts with what is reported in the literature about the concept of distress - a psychosocial phenomenon, influenced by emotional, physical and social aspects.

Therefore, the Brazilian version of the STOP-D can be used in routine practice for psychosocial screening in cardiology; the time of application is from three to five minutes, and the correction of the instrument is simple (summation of the patients’ items/answers). The identification of distress using the STOP-D could also serve as a strategy by which patients can better deal with their disease.

• Sources of Funding
  This study was funded by CAPES.
• Study Association
  This article is part of the thesis of master submitted by Isabella Barros Rabelo Gontijo, from Hospital das Clínicas da Universidade Federal de Goiás.
• Ethics approval and consent to participate
  This study was approved by the Ethics Committee of the Ebserh - HC-UFG under the protocol number 1.547.698. All the procedures in this study were in accordance with the 1975 Helsinki Declaration, updated in 2013. Informed consent was obtained from all participants included in the study.

Appendix STOP-D Brazilian version

Distress in cardiac patients

Here are five questions about your emotional state. Please read the questions carefully and draw a circle around the number the best represents the intensity or strength of how you have felt over the last week, including today. There is no right or wrong answer. You should answer each question according to the alternatives below.

Over the last week, including today, how much have you been bothered by:

1. Feeling sad, down, or uninterested in life? 0 1 2 3 4 5 6 7 8 9 Not at all A little Moderately Severely

1. Feeling sad, down, or uninterested in life?
0	1	2	3	4	5	6	7	8	9
Not at all		A little		Moderately			Severely

2. Feeling anxious or nervous? 0 1 2 3 4 5 6 7 8 9 Not at all A little Moderately Severely

2. Feeling anxious or nervous?
0	1	2	3	4	5	6	7	8	9
Not at all		A little		Moderately			Severely

3. Feeling stressed? 0 1 2 3 4 5 6 7 8 9 Not at all A little Moderately Severely

3. Feeling stressed?
0	1	2	3	4	5	6	7	8	9
Not at all		A little		Moderately			Severely

4. Feeling angry? 0 1 2 3 4 5 6 7 8 9 Not at all A little Moderately Severely

4. Feeling angry?
0	1	2	3	4	5	6	7	8	9
Not at all		A little		Moderately			Severely

5. Not having the social support (family members and friends) you feel you need? 0 1 2 3 4 5 6 7 8 9 Not at all A little Moderately Severely

5. Not having the social support (family members and friends) you feel you need?
0	1	2	3	4	5	6	7	8	9
Not at all		A little		Moderately			Severely

Current distress (sum of all items): _________ points

Cutoff for distress: 15 points

References

• ¹
  Poole L, Kidd T, Leigh E, Ronaldson A, Steptoe A. Psychological distress and intensive care unit stay after cardiac surgery?: the role of illness concern. 2015; 34(3):283-7.
• ²
  Selye H. Further thoughts on "stress without distress". Med Times. 1976;104(11):124-44.
• ³
  Ridner SH. Psychological distress?: concept analysis. J Adv Nurs. 2004;45(5):536-45.
• ⁴
  Sparrenberger F. Epidemiologia do distress psicológico?: estudo transversal de base populacional. Rev Saúde Pública. 2003;37(4):434-9.
• ⁵
  Sveinsdóttir H, Ingadóttir B. Predictors of psychological distress in patients at home following cardiac surgery?: an explorative panel study. Eur J Cardiovasc Nurs. 2012; 11(3): 339-48.
• ⁶
  Whitehead DL, Strike P, Perkins-Porras L, Steptoe A. Frequency of distress and fear of dying during acute coronary syndromes and consequences for adaptation. Am J Cardiol. 2005;96(11):2-6.
• ⁷
  Lazzarino AI, Hamaer M, Stmatakis E, Steptoe D. Low socioeconomic status and psychological distress as synergistic predictors of mortality from stroke and coronary heart disease. Psychosom Med. 2013;75(3):311-6.
• ⁸
  Maria R, Amado S, Miguel P, Cardoso M, Marques J, Santos SF, et al. ansiedade?: uma intervenção multidisciplinar com recurso ao termómetro de distress. Rev Onco News. 2010;13:4-10.
• ⁹
  Denollet J, Schiffer A, Spek V. Evidence from research on the type D ( distressed) Personality Profile. 2010;3(5):546-57.
• ¹⁰
  Ruiz MÁ, Sanjuan P, Pérez-garcía AM, Rueda B. Relations between Life satisfaction , adjustment to illness , and emotional distress in a sample of men with ischemic cardiopathy. 2011;14(1):356-65.
• ¹¹
  Falk K, Patel H, Swedberg K, Ekman I. Fatigue in patients with chronic heart failure - A burden associated with emotional and symptom distress. Eur J Cardiovasc Nurs. 2009;8(2):91-6.
• ¹²
  Sveinsdóttir H. Factors associated with psychological distress at home following elective surgery in a representative group of surgical patients: an explorative panel study. Nursing Science & Research in the Nordic Countries. 2010; 30(1): 34-9.
• ¹³
  Rasul F, Stansfeld SA, Hart CL, Gillis CR, Davey G. Psychological distress, physical illness and mortality risk. J Psychosom Res. 2004;57(3):231-6.
• ¹⁴
  Broek KC Van Den, Tekle FB, Habibovi M, Alings M, Voort PH ,Van Der Denollet J. Emotional distress , positive affect , and mortality in patients with an implantable cardioverter de fi brillator. Int J Cardiol. 2013;165(2):327-32.
• ¹⁵
  Denollet J, Sciffer A, Spek V. General Propensity to Psychological Distress Affects Cardiovascular Outcomes. Circ Cardiovasc Qual Outcomes. 2010;3(5):546-57.
• ¹⁶
  Rafanelli C, Gostoli S, Tully PJ, Roncuzzi R, Rafanelli C, Gostoli S, et al. Hostility and the clinical course of outpatients with congestive heart failure. Psychol Health. 2016; 31(2):228-38.
• ¹⁷
  Young Q, Ignaszewski A. Brief screen to identify 5 of the most common forms of psychosocial distress in cardiac patients validation of the screening tool for psychological distress. J Cardiovasc Nurs. 2007; 22(6):525-34.
• ¹⁸
  Young Q, Nguyen M, Roth S, Broadberry A, Mackay MH. Single-item measures for depression and anxiety?: Validation of the Screening Tool for Psychological Distress in an inpatient cardiology setting. Eur J Cardiovasc Nurs. 2015;14(6):544-51.
• ¹⁹
  Bacal F, Souza-Neto JD, Fiorelli AI, Mejia J, Marcondes-Braga FG, Mangini S, et al. II Diretriz Brasileira de Transplante Cardíaco. Arq Bras Cardiol. 2009;94(1 supl 1):e16-e73.
• ²⁰
  Bocchi EA, Marcondes BFG, Bacal F, Sociedade Brasileira de Cardiologia. Atualização da Diretriz Brasileira de Insuficiência Cardíaca Crônica . Arq Bras Cardiol. 2012;98(1supl):1-33.
• ²¹
  Borsa JC, Damásio BF, Bandeira DR. Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia.2012; 22(53):423-32.
• ²²
  Muñiz J, Elosua P. Hambleton RK. International Test Commission Guidelines for test translation and adaptation: second edition. Psicothema. 2013; 25(2): 151-157.
• ²³
  Jacobsen PB, Ph D, Donovan KA, Ph D, Trask PC, Ph D, et al. Screening for Psychologic Distress in Ambulatory Cancer Patients A Multicenter Evaluation of the Distress Thermometer. Cancer. 2005; 103(7):1494-502.
• ²⁴
  Botega NJ, Bio MR, Zomignani MA Jr CG, Pereira WAB. Transtornos do humor em enfermaria de clínica médica e validação de escala de medida ( HAD ) de ansiedade e depressão. Rev de Saúde Pública. 1995; 29(5): 355-63.
• ²⁵
  Tabachnik BG, Fidell LS. Using multivariate analysis. Need Heights: Allyn & Bacon, 2001.
• ²⁶
  Damásio BF. Uso da análise fatorial exploratória em psicologia. Avaliação Psicológica. 2012;11(2):213-28.
• ²⁷
  Pasquali L. Análise fatorial para pesquisadores. Brasília: Labpam; 2012.
• ²⁸
  Hair JF, Black B, Babin B, Anderson RE. Multivariate Data Analysis. Upper Saddle River : Prentice Hall; 2009.
• ²⁹
  Duarte PS, Miyazi MCOS, Ciconelli RM SR. Tradução e adaptação cultural do instrumento de avaliação de qualidade de vida para pacientes renais crônicos (KDQOL-SFTM). Rev Assoc Med Bras. 2009; 49(4):375-81.
• ³⁰
  Marco MA. Do modelo biomédico ao modelo biopsicossocial?: um projeto de educação permanente. Rev Bras Educ Méd. 2006;30(1):60-72.
• ³¹
  Ayers C, Joekes K, Copland C. Recognising anxiety and post-traumatic stress disorder in cardiac patients. Brit J Card Nurs. 2010;5(3):118-22.
• ³²
  Bertolazi AN. Adaptação Cultural e Validação de dois instrumentos de avaliação do sono: escala de sonolência de EPWORTH e Índice de qualidade de sono de Pittsburgh. (tradução e adaptação). [Dissertação]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2008.
• ³³
  Margotto PR. CURVA ROC: Como fazer e interpretar no SPSS. Brasília; 2010.

Publication Dates

• Publication in this collection
  15 Apr 2019
• Date of issue
  Sep-Oct 2019

History

• Received
  28 Aug 2018
• Reviewed
  10 Apr 2018
• Accepted
  24 Sept 2018