Depression and empathy in health professionals who work in the long-term care institutions for older adults

Depressão e empatia em profissionais de saúde de Instituições de Longa Permanência para Idosos

ABSTRACT

Empathy is an important factor to guarantee the quality of care provided in the long-term care institutions (LTCIs) for older adults, and depression is a factor that affects the health of the professional and, consequently, the care. Thus, it is important that studies are conducted on the relationship of these variables in this context.

Objective:

The aim of this study is to verify the relationship between empathy and depressive symptoms among health professionals working in the LTCIs.

Methods:

A cross-sectional study was carried out at LTCIs in the state of São Paulo, Brazil. The final sample was constituted by 101 health professionals (i.e., caregivers and nursing technicians) with direct participation in the care of institutionalized older adults. The instruments were used as follows: the Interpersonal Reactivity Index (IRI) to assess empathy and the Patient Health Questionnaire-9 (PHQ-9) for the diagnosis of depression. For the analyses, the patients were divided into groups with and without depression, according to the score of the PHQ-9.

Results:

The prevalence of depression among health professionals was 19.8%. Significant statistical differences were found between the groups for the total score of the IRI (p=0.029), for the emotional domain (p=0.023), and for the personal distress (p=0.009).

Conclusions:

The findings indicate that the presence of depression among health professionals at LTCIs is related to the higher levels of empathy, especially in the emotional domain. Thus, future studies that contribute to understanding how care must be provided with empathy, but without harming the health of the professional, should be carried out.

Keywords:
depression; empathy; homes for the aged; aged

RESUMO

A empatia é um importante fator na garantia da qualidade do cuidado prestado em Instituições de Longa Permanência para Idosos (ILPIs), e a depressão é um fator que prejudica a saúde do profissional e, consequentemente, esse cuidado. Dessa forma, é importante que se realizem estudos sobre a relação dessas variáveis nesse contexto.

Objetivo:

Verificar a relação entre empatia e sintomas depressivos em profissionais de saúde de ILPIs.

Métodos:

Estudo transversal, realizado em ILPIs no estado de São Paulo. A amostra final foi composta por 101 profissionais da saúde (cuidadores e técnicos de enfermagem), que participavam diretamente do cuidado de idosos institucionalizados. Os instrumentos utilizados foram: a Escala Multidimensional de Reatividade Interpessoal (EMRI) para avaliar empatia e o Patient Health Questionnaire-9 (PHQ-9) para o diagnóstico de depressão. Para as análises, os participantes foram divididos em grupos com e sem depressão, de acordo com a pontuação do PHQ-9.

Resultados:

A prevalência de depressão entre os profissionais de saúde foi de 19,8%. Foram encontradas diferenças estatísticas significativas entre os grupos para a pontuação total do EMRI (p=0,029), para o domínio emocional (p=0,023) e angústia pessoal (p=0,009).

Conclusões:

Nossos achados apontam que a presença de depressão entre os profissionais de saúde de ILPIs está relacionada a maiores os níveis de empatia, principalmente no domínio emocional. Assim, pesquisas futuras que contribuam para entender como deve se dar o cuidado prestado com empatia, porém sem prejudicar a saúde do profissional, devem ser realizadas.

Palavras-chave:
depressão; empatia; instituição de longa permanência para idosos; idosos

INTRODUCTION

Currently, the long-term care institutions (LTCIs) for older adults have shown to be important as a supporting cornerstone for families that need help to provide care to older adults.¹ Loss of functionality and cognitive impairment are frequent in part of the aged population and are the significant factors that can lead to their institutionalization, especially in the cases where the family cannot meet the demands derived from this process.^2,3

In this context, the way in which care is provided to these older adults is the main point of discussion, and health professionals need to be technically and emotionally prepared for this role, with the development of empathy being essential in this process.⁴ By definition, empathy refers to the human ability to put oneself in someone else’s place and is divided into two main dimensions: (1) emotional/affective empathy, which is the appearance of feelings or emotions in response to someone else’s condition and (2) cognitive empathy, which refers to the ability to identify and understand the condition of another person.⁴

These health professionals, especially the nursing team, can be more susceptible to negative mental states, such as depression, anxiety, and stress, which can be related to some working conditions, such as workload and the need to work in shifts.⁵ In addition, the prevalence of dementia and depression in institutionalized older adults is high, i.e., a factor that causes increased dependence,⁶ which contributes to the fact that the professionals who provide care under these circumstances are at high risk of developing psychiatric disorders.⁷ The impairment of the mental health of these individuals can harm not only their own health but also the quality of the assistance provided.⁵

Faced with this situation of population aging and the need to provide quality care for older adults, this study aimed to identify the relationship between empathy and depressive symptoms among health professionals working in LTCIs.

METHODS

Setting and participants

This is a quantitative, cross-sectional, correlational, and observational study, with a convenience sample, carried out with health professionals working in LTCIs. The study sample consisted of the health professionals who directly participated in the care of institutionalized older adults. Nursing technicians and caregivers were included. According to the Brazilian legislation, nursing technicians are professionals with the qualification of high school and some technical training courses, and are linked to the Regional Nursing Council (Conselho Regional de Enfermagem – COREN).⁸ The caregivers of older adults need at least a completion of elementary school and a proven professional course or training. This last category is not linked to the COREN and corresponds to number 5162-10 of the Brazilian Code of Occupations (Código Brasileiro de Ocupações – CBO).⁹

It was decided not to include higher education professionals, considering that they do not spend much time with the older adults and that their weekly workload in the LTCIs is also lower.

In total, 34 LTCIs were invited to participate in this study, and only 20 accepted the invitation. Of the institutions participating in the study, 85% were private and 15% were philanthropic, and most of the resident older adults were partially dependent on the activities of daily life. Of these institutions, 23 professionals refused to participate. The final sample consisted of 101 health professionals, who were divided into groups with and without depression, according to the score of the Patient Health Questionnaire-9 (PHQ-9).

Procedures

Initially, an invitation was made to the technical heads of the LTCIs. After acceptance, all caregivers and nursing technicians working in the institutions were invited to participate voluntarily. All the volunteers consented to their participation by signing the informed consent form, which was approved by the Research Ethics Committee at Universidade Federal de São Carlos (CAAE 81016717.0.0000.5504).

Subsequently, the professionals completed a sociodemographic questionnaire including information on age, gender, marital status, schooling, weekly workload, time in the profession, function, and if they work in more than one place. The data were collected during the professional’s shift, at the time chosen by the professional, and according to the day-to-day activities of each institution, so as not to disturb the work processes. The tests lasted around 30–45 min.

The following instruments were applied:

Patient Health Questionnaire-9

The PHQ-9 was used to assess the presence of depression.¹⁰ The test has nine items, and the answers can vary from 0 (i.e., “not at all”) to 3 (i.e., “nearly every day”) in each item, referring to the previous 2 weeks. The total score is based on the sum of the points assigned to each item, ranging from 0 to 27 points. The cutoff point considered for the presence of depression was a score greater than or equal to 10.^11,12

Interpersonal Reactivity Index

The IRI, created by Mark Davis in 1983, was used to assess empathy.¹³ The scale assesses the affective/emotional dimension of empathy and the cognitive dimension, and it consists of three subscales, namely, Empathic Concern (EC), Personal Distress (PD), and Perspective-Taking (PT). The first two subscales, EC and PD, mentioned refer to the affective dimension of empathy, and the last one, PT, refers to the cognitive dimension. It consists of 21 items, and each subscale has 7 items. The general level of empathy is obtained by the sum of the scores of the three subscales, with the score for each item varying from 1 (i.e., “Does not describe me well”) to 5 (i.e., “Describes me very well”). In this scale, the higher the score, the greater the empathy. The scale used in this study was the version in Portuguese, validated by Koller, Camino, and Ribeiro.¹⁴

Data analysis

The descriptive analysis was performed to characterize the sociodemographic profile of the groups. The Kolmogorov-Smirnov test was used to verify the normality of the data. The differences between groups were assessed by the Student’s t-test or by the Mann-Whitney U-test according to the sample distribution, and the χ² test was used for the categorical variables. The Spearman’s correlation test was used to assess the correlation between the variables. The statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS) statistical package, version 21.0. The significance level considered was p≤0.05.

RESULTS

The prevalence of depression among the health professionals was 19.8%. Table 1 presents the sociodemographic data of the participants, stratified by the groups with and without depression. As expected, a significant statistical difference was found between the groups with and without depression in the PHQ-9 (p<0.001).

Table 1.
Sociodemographic data of the participants, stratified by groups with and without depression.

	Group without depression (n=81)	Group with depression (n=20)	p-value
	Mean (SD)
Age	36.54 (10.38)	37.45 (9.72)	0.724
Time in the profession (years)	5.53 (4.50)	5.41 (6.92)	0.344
Workload (h/week)	42.17 (2.91)	41.55 (4.40)	0.408
	N (%)
Shift systems
Fixed	20 (25)	2 (10)	0.154
Variable	61 (75)	18 (90)
Gender
Female	74 (91)	20 (100)	0.202
Male	7 (9)	0
Marital status
Single	35 (43)	11 (55)	0.669
Married	29 (36)	5 (25)
Divorced	15 (19)	4 (20)
Widow/widower	2 (2)	0
Schooling
0–9	14 (17)	4 (20)
10–12	26 (32)	11 (55)	0.097
≥12	41 (51)	5 (25)
Function
Caregiver	39 (48)	14 (70)	0.065
Nursing technician	42 (52)	6 (30)
Works in more than one place
Yes	12 (15)	2 (2)	0.799
No	69 (85)	18 (18)

• SD: standard deviation; *statistically significant difference (p<0.05).

Table 2 shows the mean and standard deviation of the groups with and without depression in the IRI task and its respective domains. Significant statistical differences were found between the groups for the total score of the IRI task (p=0.029), for the emotional (p=0.023) and personal distress (p=0.009) domains.

Table 2.
Mean and standard deviation of the groups with and without depression in the Interpersonal Reactivity Index task and its respective domains.

	Group without depression (n=81)	Group with depression (n=20)	Student’s t-test or Z score	p-value
	Mean (SD)
IRI total	71.01 (±12.81)	79.42 (±12.91)	t99=-2.216	0.029*
Emotional domain	44.96 (±10.80)	51.78 (±9.72)	t99=-2.308	0.023*
Empathic concern	28.06 (±5.47)	29.42 (±4.40)	Z=-0.546	0.585
Personal distress	16.90 (±7.44)	22.36 (±8.35)	Z=-2.627	0.009*
Cognitive domain	26.04 (±4.99)	27.63 (±5.82)	Z=-1.380	0.168

• SD: standard deviation; IRI: Interpersonal Reactivity Index; *statistically significant difference (p<0.05).

In addition, weak positive correlations were found between the PHQ-9 scores and the total IRI scores (r=0.256; p=0.010), between the PHQ-9 and the emotional domain (r=0.293; p=0.002), and between the PHQ-9 and the personal distress domain (r=0.299; p=0.003). There was no correlation between the PHQ-9 scores and the cognitive domain (r=0.053; p=0.601) (Figure 1).

Figure 1.
Scatter plot of correlation between PHQ-9 scores and the total IRI; between PHQ-9 and the emotional domain; and between the PHQ-9 scores and the cognitive domain.

In relation to the presence of diseases, in the group without depression, 9% of the professionals reported systemic arterial hypertension; 2%, diabetes mellitus; 6%, cephalea; and 22%, low back pain. Among the professionals with depression, 5% reported arterial hypertension; 10%, cephalea; and 50%, low back pain.

DISCUSSION

In this study, the prevalence of depression in caregivers and nursing technicians working in LTCIs was 19.8%, and the presence of depression was associated with a higher level of empathy, especially in the emotional domain. There were no differences between the groups in the other variables studied, including weekly workload and time working in the LTCI.

Despite the significant differences involved in care, recent studies show that the prevalence of depression in informal caregivers does not present significant differences when compared with the prevalence in formal caregivers, and the numbers found are close to what was identified in our study.^15-19 In health professionals, the presence of depression can be a predictive factor for exhaustion and, consequently, can directly interfere in the quality of care provided to the patient.²⁰

In relation to the care provided, according to Duarte et al.,²¹ although empathy is one of the main factors that lead to the quality of the care, if there is no balance, it can cause harms to the health of the professionals.²¹ In the study by Karanikola et al.,²² conducted with health professionals, a positive correlation was found between depressive symptoms and empathy, the results that corroborate with the findings of our study.²²

The frequent use of the empathic capacity to deal with the patient’s suffering can cause a syndrome called “compassion fatigue,” still few studied in the Brazilian context.²¹ The exposure of the professional to stressful and traumatic events in relation to the patients can be a predictive factor for the onset of this syndrome, which leads to physical and emotional exhaustion in the professional, and which also presents rapid evolution.²³ In the study conducted by Duarte et al.,²¹ it was found that the high levels of empathy in the emotional domain are a risk factor for compassion fatigue.²¹ In our study, we did not evaluate the presence of this syndrome, but it was found that the professionals who presented some indication of depression had higher levels of empathy in the emotional domain. In this way, we can infer that they could be more prone to the “compassion fatigue” syndrome.

Most of the studies involving actors who participate in the care of older adults are carried out with informal caregivers. In the study by Jütten et al.²⁴ conducted with this category, a positive association was found between anxiety and affective empathy, and a negative association was found between depression and cognitive empathy.²⁴ Although our sample has consisted of health professionals and not of informal caregivers, these results are similar to the findings of our study, as they show that affective empathy can be related to the appearance of negative emotions.

In the study carried out by Lee et al.,²⁵ caregivers with higher levels of cognitive empathy assessed the task of caring as less stressful, were less depressed, and had greater satisfaction with life compared with caregivers with lower levels of cognitive empathy, while emotional empathy showed a negative correlation with life satisfaction.²⁵ In this way, comparing these findings with the results of our study, we can conclude that cognitive empathy can be a protective factor for the mental health of the professional.

In the study carried out by Sampaio et al.²⁶ with health professionals in the Brazilian context, positive associations were found between depression, anxiety, and one of the emotional components of empathy, namely, personal distress, also corroborating with the results of this study.²⁶ Another finding of this study was that the use of cognitive aspects of empathy can prevent the onset of depression symptoms, reinforcing the possibility that empathy is associated with the mental health of the professionals who provide care.²⁶

As a limitation of the study, it should be mentioned that only 20 LTCIs (58.8%) from the 34 that were invited participated in the survey. Although PHQ-9 is a widely used and validated instrument in different contexts, the use of structured or semi-structured clinical interviews is the gold standard for the diagnosis of depression; therefore, using the PHQ-9 to divide clinical groups with and without depression can be considered a limitation of this study.

In addition to that, considering that overload is a frequent condition within the care context and that it can result in the presence of the burnout syndrome, not evaluating this syndrome can be considered as a limitation since it shares similar characteristics of the mood disorders.

As a strength of this study, it can be highlighted that there are few studies addressing this theme in the Brazilian context. In addition, a large part of the national and international studies related to the care of older adults have informal caregivers as samples, not health professionals, i.e., individuals who have played an important role in the provision of care to older adults due to population aging and to the increase in the number of LTCIs. Thus, this study contributes to the research of empathy and depression in the institutional context, enabling the planning of actions aimed at the quality of the care provided, also paying attention to the health of the professional.

We observed a high prevalence of depression in health professionals of LTCIs. In addition, empathy, especially the affective domain, seems to be a factor to be considered in the evaluation of the mental health of caregivers and nursing technicians working in LTCIs.

ACKNOWLEDGMENTS

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).

REFERENCES

• ¹
  Küchemann B. Envelhecimento populacional, cuidado e cidadania: velhos dilemas e novos desafios. Soc Estado. 2012;27(1):165-80. https://doi.org/10.1590/S0102-69922012000100010.
  » https://doi.org/https://doi.org/10.1590/S0102-69922012000100010
• ²
  Lini E, Portella M, Doring M. Factors associated with the institutionalization of the elderly: a case-control study. Rev Bras Geriatr Gerontol. 2016;19(6):1004-14. https://doi.org/10.1590/1981-22562016019.160043
  » https://doi.org/https://doi.org/10.1590/1981-22562016019.160043
• ³
  Cepoiu-Martin M, Tam-Tham H, Patten S, Maxwell C, Hogan D. Predictors of long-term care placement in persons with dementia: a systematic review and meta-analysis. Int J Geriatr Psychiatry. 2016;31(11):1151-71. https://doi.org/10.1002/gps.4449
  » https://doi.org/https://doi.org/10.1002/gps.4449
• ⁴
  Ratka A. Empathy and the development of affective skills. Am J Pharm Educ. 2018;82(10):7192. https://doi.org/10.5688/ajpe7192
  » https://doi.org/https://doi.org/10.5688/ajpe7192
• ⁵
  Togo F, Yoshizaki T, Komatsu T. Association between depressive symptoms and morningness-eveningness, sleep duration and rotating shift work in Japanese nurses. Chronobiol Int. 2017;34(3):349-59. https://doi.org/10.1080/07420528.2016.1273942
  » https://doi.org/https://doi.org/10.1080/07420528.2016.1273942
• ⁶
  Helvik AS, Engedal K, Benth JŠ, Selbæk G. Prevalence and Severity of Dementia in Nursing Home Residents. Dement Geriatr Cogn Disord. 2015;40(3-4):166-77. https://doi.org/10.1159/000433525
  » https://doi.org/https://doi.org/10.1159/000433525
• ⁷
  Tsaras K, Papathanasiou IV, Vus V, Panagiotopoulou A, Katsou MA, Kelesi M, Fradelos EC. Predicting factors of depression and anxiety in mental health nurses: a quantitative cross-sectional study. Med Arch. 2018;72(1):62-7. https://doi.org/10.5455/medarh.2017.72.62-67
  » https://doi.org/https://doi.org/10.5455/medarh.2017.72.62-67
• ⁸
  Conselho Federal de Enfermagem. Resolução COFEN nº 609/2019. Atualiza, no âmbito do Sistema Cofen/Conselhos Regionais de Enfermagem, os procedimentos para registro de especialização técnica de nível médio em Enfermagem concedida aos Técnicos de Enfermagem e aos Auxiliares de Enfermagem. Brasília: Diário Oficial da União; 2019.
• ⁹
  Brasil. Ministério do Trabalho e Emprego. Portaria nº. 397, de 10 de dezembro de 1999. Institui a Classificação Brasileira de Ocupações. Brasília: Ministério da Saúde; 2002.
• ¹⁰
  Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-13. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
  » https://doi.org/https://doi.org/10.1046/j.1525-1497.2001.016009606.x
• ¹¹
  de Lima Osório F, Vilela Mendes A, Crippa JA, Loureiro SR. Study of the discriminative validity of the PHQ-9 and PHQ-2 in a sample of Brazilian women in the context of primary health care. Perspect Psychiatr Care. 2009;45(3):216-27. https://doi.org/10.1111/j.1744-6163.2009.00224.x
  » https://doi.org/https://doi.org/10.1111/j.1744-6163.2009.00224.x
• ¹²
  He C, Levis B, Riehm KE, Saadat N, Levis AW, Azar M, et al. The accuracy of the Patient Health Questionnaire-9 algorithm for screening to detect major depression: an individual participant data meta-analysis. Psychother Psychosom. 2020;89(1):25-37. https://doi.org/10.1159/000502294
  » https://doi.org/https://doi.org/10.1159/000502294
• ¹³
  Davis M. Measuring individual differences in empathy: Evidence for a multidimensional approach. J Pers Soc Psychol. 1983:44(1):113-26. https://doi.org/10.1037/0022-3514.44.1.113
  » https://doi.org/https://doi.org/10.1037/0022-3514.44.1.113
• ¹⁴
  Koller S, Camino C, Ribeiro J. Adaptação e validação interna de duas escalas de empatia para uso no Brasil. Estud Psicol. 2001;18(3):43-53. https://doi.org/10.1590/s0103-166x2001000300004
  » https://doi.org/https://doi.org/10.1590/s0103-166x2001000300004
• ¹⁵
  Oliver D, Washington K, Smith J, Uraizee A, Demiris G. The prevalence and risks for depression and anxiety in hospice caregivers. J Palliat Med. 2017;20(4):366-71. https://doi.org/10.1089/jpm.2016.0372
  » https://doi.org/https://doi.org/10.1089/jpm.2016.0372
• ¹⁶
  Ostojić D, Vidović D, Baceković A, Brecić P, Jukić V. Prevalence of anxiety and depression in caregivers of Alzheimer’s dementia patients. Acta Clin Croat. 2014;53(1):17-21. PMID: 24974662
  » https://doi.org/PMID: 24974662
• ¹⁷
  Hu P, Yang Q, Kong L, Hu L, Zeng L. Relationship between the anxiety/depression and care burden of the major caregiver of stroke patients. Medicine (Baltimore). 2018;97(40):1-6. https://doi.org/10.1097/md.0000000000012638
  » https://doi.org/https://doi.org/10.1097/md.0000000000012638
• ¹⁸
  Derajew H, Tolessa D, Feyissa G, Addisu F, Soboka M. Prevalence of depression and its associated factors among primary caregivers of patients with severe mental illness in southwest, Ethiopia. BMC Psychiatry. 2017;17(1):88. https://doi.org/10.1186/s12888-017-1249-7
  » https://doi.org/https://doi.org/10.1186/s12888-017-1249-7
• ¹⁹
  Yıkılkan H, Aypak C, Görpelioğlu S. Depression, anxiety and quality of life in caregivers of long-term home care patients. Arch Psychiatr Nurs. 2014;28(3):193-6. https://doi.org/10.1016/j.apnu.2014.01.001
  » https://doi.org/https://doi.org/10.1016/j.apnu.2014.01.001
• ²⁰
  Golonka K, Mojsa-Kaja J, Blukacz M, Gawlowska M, Marek T. Occupational burnout and its overlapping effect with depression and anxiety. Int J Occup Med Environ Health. 2019;32(2):229-44. https://doi.org/10.13075/ijomeh.1896.01323
  » https://doi.org/https://doi.org/10.13075/ijomeh.1896.01323
• ²¹
  Duarte J, Pinto-Gouveia J, Cruz B. Relationships between nurses’ empathy, self-compassion and dimensions of professional quality of life: A cross-sectional study. Int J Nurs Stud. 2016;60:1-11. https://doi.org/10.1016/j.ijnurstu.2016.02.015
  » https://doi.org/https://doi.org/10.1016/j.ijnurstu.2016.02.015
• ²²
  Karanikolaa M, Zartaloudib A, Nystazakic M, Zavroud R, Papathanassoglou E. Is there any association among depressive symptoms, job satisfaction and self-assessed empathy? A correlational study in Greek Psychiatric/Mental Health Nurses. Arch Psychiatr Nurs. 2020;34(4):230-36. https://doi.org/10.1016/j.apnu.2020.04.006
  » https://doi.org/https://doi.org/10.1016/j.apnu.2020.04.006
• ²³
  Slatten L, Carson K, Carson P. Compassion fatigue and burnout: what managers should know. Health Care Manag (Frederick). 2011;30(4):325-33. https://doi.org/10.1097/hcm.0b013e31823511f7
  » https://doi.org/https://doi.org/10.1097/hcm.0b013e31823511f7
• ²⁴
  Jütten LH, Mark RE, Sitskoorn MM. Empathy in informal dementia caregivers and its relationship with depression, anxiety, and burden. Int J Clin Health Psychol. 2019;19(1):12-21. https://doi.org/10.1016/j.ijchp.2018.07.004
  » https://doi.org/https://doi.org/10.1016/j.ijchp.2018.07.004
• ²⁵
  Lee HS, Brennan PF, Daly BJ. Relationship of empathy to appraisal, depression, life satisfaction, and physical health in informal caregivers of older adults. Res Nurs Health. 2001;24(1):44-56. https://doi.org/10.1002/1098-240x(200102)24:1<44::aid-nur1006>3.0.co;2-s
  » https://doi.org/https://doi.org/10.1002/1098-240x(200102)24:1<44::aid-nur1006>3.0.co;2-s
• ²⁶
  Sampaio L, Oliveira L, Pires M. Empatia, depressão, ansiedade e estresse em Profissionais de Saúde Brasileiros. Ciênc Psicol. 2020;14(2):e2215. https://doi.org/10.22235/cp.v14i2.2215
  » https://doi.org/https://doi.org/10.22235/cp.v14i2.2215

• This study was conducted by the Research Group on Mental Health, Cognition and Aging, Universidade Federal de São Carlos, São Carlos, SP, Brazil.

• Funding: Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP; process 2017/26221-4) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES; Finance Code 001).

Publication Dates

• Publication in this collection
  20 Sept 2021
• Date of issue
  Jul-Sep 2021

History

• Received
  08 Dec 2020
• Accepted
  21 Mar 2021