Acessibilidade / Reportar erro

What Makes Risk Communication Efficient? The Stakeholder View

¿Qué hace que la Comunicación de Riesgos sea eficiente? La visión de los involucrados

Abstract

Environmental risk communication comprises an essential step in contaminated sites management, being one of its basic principles. However, there are few studies and technical guidelines on how to perform risk communication. This research aimed to identify the main factors that can contribute to efficient risk communication from the stakeholders’ perspective: those affected or potentially affected by the risks, representatives of institutions responsible for risk communication, environmental agencies, and consultants. To achieve that, the Q technique, a methodology specialized in the study of people and their subjectivities, was applied to 24 individuals. Five factors representing the stakeholders’ perspectives were identified: 1 - Trust; 2 - Secure communication; 3 - Dialogue; 4 - Partnership; and 5 - Health, in addition to a set of consensual information between all groups. Concern for the health of those involved was the most important point of view for most participants.

Keywords:
Technical risk communication; Democratic risk communication; Contaminated sites; Environmental legislation; Q Methodology

Resumen

La comunicación de riesgos ambientales constituye un paso fundamental en la gestión de sitios contaminados, siendo uno de sus principios básicos. Sin embargo, existen pocos estudios y lineamientos técnicos que orienten su realización. Así, esta investigación buscó identificar los principales factores que pueden contribuir a una comunicación de riesgos eficiente, desde la perspectiva de las partes involucradas: afectados por los riesgos, representantes de instituciones responsables de la comunicación de riesgos, agencias ambientales y consultoras. Para ello se utilizó la técnica Q, metodología especializada en el estudio de las personas y sus subjetividades, aplicada a 24 individuos. Se identificaron cinco factores que representan la opinión de los sujetos de investigación: 1-Confianza; 2-Comunicación segura; 3-Diálogo; 4- Asociaciones; y 5-Salud, además de un conjunto de información consensuada entre todos los grupos. La preocupación por la salud de los afectados fue el punto de vista más importante para la mayoría de los encuestados.

Palabras-clave:
Comunicación técnica de riesgos; Comunicación democrática de riesgos; Sitios contaminados; Legislación ambiental; Técnica Q

Resumo

A comunicação de risco ambiental compreende uma etapa essencial do gerenciamento de áreas contaminadas, sendo um de seus princípios básicos. Contudo, são escassos estudos e orientações técnicas para a sua condução. Esta pesquisa buscou identificar os principais fatores que podem contribuir para uma comunicação de risco eficiente, a partir da visão das partes envolvidas: afetados ou potencialmente afetados pelos riscos, representantes de instituições responsáveis pela comunicação de risco, de agências ambientais e consultorias. Para isso, utilizou-se a técnica Q, metodologia especializada no estudo do ponto de vista das pessoas e suas subjetividades, aplicada a 24 indivíduos. Foram identificados cinco fatores representantes da visão dos sujeitos da pesquisa: 1-Confiança; 2-Comunicação segura; 3-Diálogo; 4-Parcerias; e 5-Saúde, além de um conjunto de informações consensuais entre todos os grupos. A preocupação com a saúde dos afetados foi o ponto de vista de maior importância para a maioria dos entrevistados.

Palavras-chave:
Comunicação de risco técnica; Comunicação de risco democrática; Áreas contaminadas; Legislação ambiental; Técnica Q

Introduction

The concentration of polluting activities in urban centers is one of the causes of the emergence of contaminated sites. These areas are a specific case of degraded lands, where there is contamination of one or more environmental compartments such as soil, air, surface water, or groundwater. Contaminants can be chemicals fromvarious sources, such as improper storage and disposal, leaks in production processes and tanks, and accidents (ARAÚJO, 2014ARAÚJO, Antônia Angélica Correia. O Brasil no contexto do gerenciamento de áreas contaminadas: Um Olhar Crítico para as Regiões Desiguais do País. Dissertação (Mestrado em Engenharia Civil, Recursos Hídricos e Saneamento). Universidade Federal de Alagoas, Maceió, 2014.; ARAÚJO-MOURA; CAFFARO FILHO, 2015ARAÚJO-MOURA, Antônia Angélica Correia; CAFFARO FILHO, Roberto Augusto. Panorama do gerenciamento de áreas contaminadas no Brasil após a resolução Conama 420/09. Águas Subterrâneas, v. 29, n. 2, p. 202-2012, 2015.).

Thus, these sites are environmental liabilities that can pose risks to ecosystems and human health. In the state of São Paulo alone, according to the registry of the São Paulo Environmental Company (Cetesb), there were a total of 6,585 contaminated sites in 2022, of which 1,996 are considered rehabilitated (CETESB, 2022). Depending on the hazard degree and contaminant type, some sites may present high toxicity and carcinogenicity. In these cases, there may be an increase in the incidence of malignant neoplasms and congenital diseases in populations exposed to such liabilities (PANAGOS et al., 2013PANAGOS, Panos; VAN LIEDEKERKE, Marc; YIGINI, Yusuf; MONTANARELLA, Luca. Contaminated sites in Europe: review of the current situation based on data collected through a European network. Journal of Environmental and Public Health, v. 2013, 2013. 11 p.; SILVA, 2007SILVA, Renan Finamore Gomes. Gestão de áreas contaminadas e conflitos ambientais: o caso da Cidade dos Meninos. 2007. 118 f. Tese (Doutorado em Engenharia), Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2007.; SPÍNOLA, 2011SPÍNOLA, Ana Luiza Silva. Inserção das áreas contaminadas na gestão municipal: desafios e tendências. 2011. Tese de Doutorado. Faculdade de Saúde Pública, Universidade de São Paulo, São Paulo, São Paulo, 2011.). These populations are potentially exposed to risks.

The term “risk” cannot be easily defined and remains a subject of debate (LIEBER; LIEBER, 2002LIEBER, Renato Rocha; ROMANO-LIEBER, Nicolina Silvana. O conceito de risco: Janus reinventado. In: Minayo, M. C. S.; Miranda, A. C. Saúde e ambiente: estreitando nós. Rio de Janeiro: Fiocruz, 2002. p. 69-112.; RODRIGUES; ZANIRATO, 2018RODRIGUES, Letícia Stevanato; ZANIRATO, Silvia Helena. Contaminação, comunicação e remediação dos riscos ambientais no conjunto Heliópolis-Gleba L-SP: um estudo das representações dos moradores do local. IX Congresso Brasileiro de Gestão Ambiental. São Bernardo, 2018. 10 p.). The probabilistic approach is commonly adopted for characterizing and quantifying risks in some fields of knowledge, such as engineering and epidemiology, while qualitative characteristics of risk are valued in the social sciences, considering cultural and social perspectives (AREOSA, 2008AREOSA, J. O risco no âmbito da teoria social. VI Congresso Português de Sociologia - Mundo sociais: saberes e práticas. Lisboa: Universidade Nova de Lisboa, Faculdade de Ciências Sociais e Humanas, 2008.; QUEIRÓS et al., 2006QUEIRÓS, Margarida; VAZ, Teresa; PALMA, Pedro. Uma reflexão a propósito do risco. Lisboa: Centro de Estudos Geográficos. v. 10, n. 7, p. 2009, 2006.). Risk is typically associated with the concept of uncertainty (BECK, 1986). However, in national legislation, risk has a probabilistic nature and disregards social and cultural factors, as seen in Decree No. 59.263/2013 (SÃO PAULO, 2013) and Resolution of the National Environment Council (Conama) No. 420/2009 (BRASIL, 2009).

According to current national standards, the management of contaminated sites comprises a set of activities that ensure the identification and quantification of risks and the adoption of appropriate intervention measures to reduce risks to acceptable levels for human health and the environment (SÃO PAULO, 2013; FEAM, 2016). The levels considered acceptable or tolerable for human health risks are defined in legislation for carcinogenic and non-carcinogenic contaminants as the probability of an additional case of cancer occurring in a population of 100,000 and the absence of proven deleterious effects on human health, respectively (BRASIL, 2009).

In the context of risk management, communication plays a fundamental role as it enables dialogue among stakeholders and the creation of better coping conditions for the affected community. Therefore, it reduces risks to human and environmental health, contributing to the maintenance and protection of public well-being and future uses of these sites (BRASIL, 2009; FISCHHOFF, 2009FISCHHOFF, Baruch. Risk perception and communication. Oxford textbook of public health, Volume 2: the methods of public health. Oxford: Oxford University Press, 2009, p. 940-953., LOURENÇO; MARCHIORI, 2012; TEIXEIRA; MOTTA; MORAES, 2016TEIXEIRA, Cláudia Echevenguá; MOTTA, Flávia Gutierrez; MORAES, Sandra Lúcia de. Panorama GAC: mapeamento da cadeia de gerenciamento de áreas contaminadas. 1. ed. São Paulo: IPT, 2016.; NIELSON; KLEFFNER; LEE, 2005NIELSON, Norma L.; KLEFFNER, Anne E.; LEE, Ryan B. The evolution of the role of risk communication in effective risk management. Risk management and insurance review, v. 8, n. 2, p. 279-289, 2005.).

The way risks are communicated has undergone profound transformations over time (LEISS, 1996LEISS, W. Three phases in the evolution of risk communication practice. Annals of the American Academy of Political and Social Science, , Challenges in Risk Assessment and Risk Management, v. 545, p. 85-94, 1996.; FISCHFFOR, 1996; VICTOR, 2015VICTOR, Cilene. Comunicação de riscos de desastres no contexto das mudanças climáticas: muito além do jornalismo. XXXVIII Congresso Brasileiro de Ciências da Comunicação. Rio de Janeiro, RJ, 4 - 7 set. 2015. 15 p.; LOURENÇO, 2015LOURENÇO, M. R. Comunicação de risco como um processo: uma análise do caso Shell/Basf. 2015. 139 p. Dissertação (Mestrado em Gestão de Sustentabilidade) - Universidade Estadual de Londrina. Londrina, Paraná, 2015.). Initially, the affected public was considered a passive recipient of information provided by managers. Gradually, those affected began to participate in decision-making related to risk management in contaminated sites. Thus, communication strategies shifted from persuasive to integrative and participatory approaches (LOURENÇO, 2015; VICTOR, 2015), corresponding respectively to two perspectives on risk communication described in the literature: technical risk communication and democratic risk communication (FIORINO, 1989FIORINO, Daniel J. Technical and Democratic Values in Risk Analysis 1. Risk Analysis, v. 9, n. 3, p. 293-299, 1989.; ROWAN, 1994ROWAN, K. E. The technical and democratic approaches to risk situations: Their appeal, limitations, and rhetorical alternative. Argumentation, v. 8, n. 4, p. 391-409, 1994.), simplifying something that is recognized as complex (FIORINO, 1989).

The technical view of risk communication is based on the idea of a knowledge deficit among the affected population and occurs in a unidirectional manner - from experts to the affected community. This approach consists of translating data with the aim of informing and educating, in order to bring about behavioral change and assure the population that the risk in question is acceptable (RAMIREZ-ANDREOTTA et al., 2014RAMIREZ-ANDREOTTA, M. D. et al. Environmental research translation: Enhancing interactions with communities at contaminated sites. Science of the Total Environment, v. 497, p. 651-664, 2014.; ROWAN, 1994ROWAN, K. E. The technical and democratic approaches to risk situations: Their appeal, limitations, and rhetorical alternative. Argumentation, v. 8, n. 4, p. 391-409, 1994.). Implicitly, it attributes rational and valid knowledge to experts, considering it superior to the knowledge and judgments of the affected population. From this perspective, failures in risk communication are attributed to fear, irrationality, and other emotional factors of the affected population, rather than to the ineffectiveness of the communication performed by experts (FIORINO, 1989FIORINO, Daniel J. Technical and Democratic Values in Risk Analysis 1. Risk Analysis, v. 9, n. 3, p. 293-299, 1989.; RAMIREZ-ANDREOTTA, 2014).

In contrast to the technical model of risk communication, there is democratic risk communication, which is based on bidirectional dialogue and grounded in political issues, such as familiarity and acceptance of risks, whether risks are voluntary or imposed, and equity in the distribution of risks and benefits among stakeholders (FIORINO, 1989FIORINO, Daniel J. Technical and Democratic Values in Risk Analysis 1. Risk Analysis, v. 9, n. 3, p. 293-299, 1989.; ROWAN, 1994ROWAN, K. E. The technical and democratic approaches to risk situations: Their appeal, limitations, and rhetorical alternative. Argumentation, v. 8, n. 4, p. 391-409, 1994.).

Historically, the technical model of risk communication has been dominant, although it has been criticized for being ineffective and marginalizing the affected population from the risk management and communication process. It should be emphasized that when this form of communication is adopted, those affected by the risk become more vulnerable in terms of understanding the risk and, consequently, confronting it (DI GIULIO et al., 2010DI GIULIO, G. M. Comunicação e governança do risco: exemplos de comunidades expostas à contaminação por chumbo no Brasil e Uruguai. 2010. Tese (Doutorado em Ambiente e Sociedade) -Instituto de Filosofia e Ciências Humanas, Universidade Estadual de Campinas, Campinas, São Paulo, 2010.; VICTOR, 2015VICTOR, Cilene. Comunicação de riscos de desastres no contexto das mudanças climáticas: muito além do jornalismo. XXXVIII Congresso Brasileiro de Ciências da Comunicação. Rio de Janeiro, RJ, 4 - 7 set. 2015. 15 p.).

Listening and open dialogue as communicative processes are strengths of the democratic model, as they prioritize the concerns of the involved parties with less decision-making power. It is important to highlight that risk management strategies are more effective when different stakeholders participate in their management because everyone who lives with risk has some capacity to discuss it. From this perspective, decisions about risk are as important as the probability and acceptance of risks (ROWAN, 1994ROWAN, K. E. The technical and democratic approaches to risk situations: Their appeal, limitations, and rhetorical alternative. Argumentation, v. 8, n. 4, p. 391-409, 1994.).

However, the democratic model also has weak points and flaws. Although democratic risk communication is based on listening and popular participation, it does not guarantee fair and equitable solutions, nor does it ensure a minimum understanding of the problem to be addressed. Moreover, inadequately developed negotiation skills can result in ineffective risk management procedures. It should also be noted that full participation of the affected population may not occur due to inequalities and the absence of rules (ROWAN, 1994ROWAN, K. E. The technical and democratic approaches to risk situations: Their appeal, limitations, and rhetorical alternative. Argumentation, v. 8, n. 4, p. 391-409, 1994.). Despite these limitations, for the development of this research, it was considered that democratic risk communication represents an alternative with greater potential to generate positive outcomes for the parties involved in the process.

Thus, despite its importance, risk communication may not develop efficiently in some situations, leading to problems in contaminated sites management. For example, Di Giulio and colleagues (2012) identified errors in the risk communication conducted in cases of contamination in the cities of Adrianópolis (PR) and Santo Amaro da Purificação (BA). Among the errors and mistakes, the absence of an adequate risk communication plan stands out, as it did not consider different aspects of the site -for instance, the sense of belonging to the place where individuals lived and the breakdown in trust between the parties involved were not taken into account.

Some issues considered important for efficient risk communication include the autonomy of the involved public based on the availability of information (GENUIS; JARDINE, 2015); contextualization and encouragement of dialogue (RAMIREZ-ANDREOTTA et al., 2014RAMIREZ-ANDREOTTA, M. D. et al. Environmental research translation: Enhancing interactions with communities at contaminated sites. Science of the Total Environment, v. 497, p. 651-664, 2014.); and the analysis of risk perceptions (SATO, 2015SATO, A. Understanding effective risk communication in the context of a radiological accident. Fukushima Global Communication Programme Working Paper. N. 7. 2015. 17 p.). Efficient risk communication provides individuals with a means of participating in risk management. It begins with the provision of clear, objective, consistent, and complete information, which tends to contribute to making individuals engaged, informed, cooperative, and interested (COVELLO, 2011COVELLO, V. T. Risk communication, radiation, and radiological emergencies: strategies, tools, and techniques. Health physics, v. 101, n. 5, p. 511-530, 2011.).

Studies addressing risk communication in contaminated sites and the perspectives of the actors involved in this process are scarce. Therefore, understanding aspects considered important for risk communication by those involved can contribute to the development and improvement of the norms that guide the process. In this regard, the present study aims to investigate which parameters are relevant for efficient risk communication in contaminated sites, according to the perspectives of different groups involved in the process.

Methodology

In order to understand different perspectives on what constitutes effective risk communication, the Q-methodology was used, which is a specialized technique for studying people’s viewpoints and subjectivities. This involved presenting statements that were evaluated by the research participants, known as “judges,” and the results of these evaluations underwent inverted factor analysis. In this way, the statements assume the role traditionally occupied by the subjects of a study (BROWN, 1980BROWN, S. R. Political subjectivity: Applications of Q methodology in political science. New Haven: Yale University Press, 1980.), with the quantity of statements being more important than the number of judges.

To create the statements, information was gathered from the literature and interviews with individuals experienced in contaminated site management and risk communication. These individuals included environmental consultants, researchers in the field, and responsible parties for contaminated sites, who were selected based on recommendations and their involvement in previous risk communication processes. Based on the collected information, 67 statements were created (referred to as the Q-set) (OLIVEIRA 2020OLIVEIRA, Uilians Vieira. A visão dos envolvidos para uma comunicação de risco eficiente: investigação metodológica. 2020. 124 f. Dissertação (Mestrado em Ciências), Universidade Federal de São Paulo, Diadema, 2020. Available at: https://repositorio.unifesp.br/xmlui/handle/11600/64297
https://repositorio.unifesp.br/xmlui/han...
), categorized into six groups according to their content: (1) Communication strategies; (2) Transparency of information; (3) Economic aspects; (4) Relationship between the involved parties; (5) Legal aspects; and (6) Aspects related to human health. It is important to note that the Q-set should be broad enough to represent different viewpoints while also being concise enough to prevent participant fatigue. Therefore, the number of statements in this study was considered appropriate.

A matrix (Q-grid) with 67 fields was created, divided into 11 columns with a range of values from +5 indicating greater agreement to -5 indicating lesser agreement. The 67 statements were printed on laminated cards, and the matrix was printed on A0-sized paper to facilitate the visualization and organization of the cards (Figure 1). It is worth noting that the matrix can be created and presented in various formats or even administered electronically, with the researcher selecting the most suitable form for the study. The statements underwent a pre-test with three individuals from the target audience. After the pre-test, some statements were modified, and therefore, these individuals were not included in the final target audience of the research.

Figure 1
Materials used in the research. a) Example of cards; e b) Q-Grid.

The target audience of the study (referred to as the P-set) consisted of individuals from four groups: 1) individuals affected or potentially affected by the risk of four different contaminated sites in the Metropolitan Region of São Paulo (A); 2) consultants working with contaminated sites (C); 3) responsible parties for contaminated sites (R); and 4) representatives from environmental agencies (P). Each individual was identified by a letter (A, C, R, or P) indicating their group, followed by a unique number to differentiate them from each other.

At the beginning of the interview, a form was filled out to gather information about the profile of each interviewee, including questions about gender, age, educational background, and years of experience with contaminated sites. The participants were also asked to rate their own knowledge of risk communication on a scale of 0 to 10, based on their experiences and familiarity with the topic, without providing specific evaluation parameters. Subsequently, the participants in this research were given instructions on the procedures and were provided with all the cards (Q-set), randomly organized, and the Q-Grid. The matrix was filled out by each participant according to their degree of agreement or disagreement, by placing each statement in a space within the matrix, considering the guiding question: “Given that risk communication is one of the basic principles of contaminated sites management, which of the following statements represent effective risk communication from your point of view?”

At the end of each completion, which took approximately 50 minutes, the matrix was identified and photographed. The data were analyzed using statistical analysis software PQMethod ®, following the methodology proposed by Brown (1980BROWN, S. R. Political subjectivity: Applications of Q methodology in political science. New Haven: Yale University Press, 1980.; 2004). The statistical analysis allows for the identification of clusters of individuals who share similar viewpoints on the investigated topic, referred to as factors. Thus, each factor represents a common or closely related viewpoint of a group of individuals, distinct from the others. After identifying the factors, factor loadings for each individual were calculated, indicating the coefficient of relationship between each individual and the established factors. Following Brown’s guidelines (2004), individuals with factor loadings above 0.44 were considered significant.

Results and Discussion

A total of 24 individuals were interviewed, 13 males and 11 females, achieving a ratio of 2.7 statements per research participant (referred to as judges), a number within the recommended range for the Q-method (WEBLER; DANIELSON; TULER, 2009). Among the total number of interviewees, Group C had the highest number of participants (eight in total), followed by Group A (six individuals). The remaining groups had five individuals each, as described in Table 1. Only two of the interviewees, both from the group of those affected or potentially affected by the risk, did not have a completed undergraduate degree. Among the total interviewees, 54% (n=13) were engineers, and 12% (n=3) had degrees in geology. Other fields of study included biology, environmental sciences, environmental management, and education/pedagogy. The participants from the environmental agency had more experience in working with contaminated sites and had the highest number of individuals with doctoral degrees (three out of five). Interestingly, this group gave themselves the lowest average rating in the self-assessment of their knowledge of risk communication. The members of this group may consider the subject highly complex, making it difficult to have a high level of knowledge about it. However, it is worth noting that no group rated themselves as having a very high level of knowledge on the subject.

Table 1
Individuals profile by group

From the statistical analysis, five factors were identified, representing a common perspective, and individuals were associated with these factors according to factor loadings. In the Q technique, factors are usually named based on the characteristics of the statements that compose them, extracted from the statistical analysis. Thus, Factor 1 was named the ‘Trust’ factor, referring to the credibility given to the words of others, transparency of information, and cooperative relationships among the involved parties. Factor 2 was named ‘Secure Communication,’ relating to the means and methods by which risk communication occurs, avoiding, for example, generating panic. Factor 3 was named ‘Dialogue’, indicating an exchange of information between the parties. Factor 4 was named ‘Partnerships’, concerning the collaborations that can be established among the stakeholders in risk communication. Lastly, Factor 5 was named ‘Health’, relating to communication aimed at minimizing health risks. Factor 4 had the highest number of individuals (n=7), followed by Factors 1, 3, and 5 (n=5), and finally Factor 2 (n=4). Two individuals were associated with two factors: C3 with Factors 1 and 2, and A5 with Factors 4 and 5.

None of the studied groups concentrated solely on a single factor (Figure 2; Table 1). Group P was distributed across a smaller number of factors (two factors - 1 and 4), while Groups C and A were present in four out of the five factors (C in Factors 1, 2, 4, and 5, and A in Factors 1, 3, 4, and 5). Group R was the only group with no individuals in Factors 1 and 4, whereas Group P did not have any individuals in Factor 5. Interestingly, these two groups (P and R) were never in the same factor, highlighting that the priorities valued by managers and those responsible for environmental liabilities (P), and the representatives of environmental agencies (R) in risk communication are distinct, possibly due to having different objectives with risk communication. The individuals from environmental agencies were found in the ‘Trust’ and ‘Partnerships’ factors, indicating a focus on closer collaboration among the involved parties. On the other hand, those responsible for the areas seek security for their actions, mainly through the provision of information.

Figure 2
Distribution of individuals by factors, being A: affected or potentially affected by the risk; C: consultants from contaminated areas; R: managers and people responsible for environmental liabilities; and P: representatives of environmental agencies

The variables of the interviewees’ profiles by factor are presented in Table 2. The main trend observed is a higher average age and professional experience in Factor 1 (Trust). However, this is directly linked to the higher presence of individuals representing environmental agencies in this factor, proportionally. It was also found that within the same factor, there are individuals under the age of 30 and others over the age of 50. The same applies to the other variables. Thus, it can be concluded that it is not possible to establish a direct relationship between the profile variables and the factors. This is likely because, with the use of the Q technique, individual differences among the research participants - the judges - tend to be of lesser importance than the obtained results related to the judges’ evaluation of the subject, corresponding to the identified factors.

Table 2
Relationship of factors with variables age, time of experience with contaminated sites, self-assessment of knowledge in risk communication

Factors: Common Viewpoints

Based on the statements associated with each factor, the distinguishing points of view for each factor are presented below.

Factor 1 - Trust

Factor 1 represents the point of view of five individuals: P1, P3, C3, C7, and A3. The individuals in this factor consider information transparency as a priority in maintaining a relationship of trust between the parties involved, as well as showing an interest in problem resolution.

The framework established by Factor 1 is supported by Covello (2011COVELLO, V. T. Risk communication, radiation, and radiological emergencies: strategies, tools, and techniques. Health physics, v. 101, n. 5, p. 511-530, 2011.). According to this author, honesty, openness to dialogue, and transparency are some of the requirements for effective risk communication. Trust and transparency are values that are built through ethical, consistent, and conscious actions, such as valuing and listening to those involved and communicating reliable messages. Transparency of information is an important characteristic of risk communication in contemporary times, as it can promote and/or enhance trust and credibility among the involved parties.

The legal aspects of risk communication are also relevant in this factor. According to the individuals in Factor 1, guidance from environmental agencies, such as specific and detailed regulations on how to conduct risk communication, should be followed. However, in the country, there are no regulations that guide risk communication or metrics to evaluate the effectiveness of the actions taken. This can lead to inefficient risk communication, especially detrimental to the affected population, highlighting the need for stakeholders, potentially coordinated by the government through its environmental agencies, to establish norms and guidelines for risk communication.

In this factor, there is also a rejection of risk communication from the beginning of the contaminated site investigation. According to some interviewees, this rejection arises because, in the initial stages, the data is still incomplete, preventing a comprehensive understanding of the situation. There is also a rejection of communication regarding the environmental impacts caused by the planned remediation in the area. This may be because the assessment of remediation impacts is still a relatively recent topic, particularly in Brazil, and often considered secondary compared to the benefits of eliminating contamination risks (BRAUN et al., 2020BRAUN, A. B. et al. Relevance of sustainable remediation to contaminated sites manage in developed and developing countries: Case of Brazil. Land Use Policy, v. 94, p. 104533, 2020.).

Factor 2 - Secure Communication

The “Secure Communication” factor reflects the point of view of four individuals - C3, C4, C6, and R1 - who consider it essential to adopt strategies to prevent panic among the affected population and to address rumors related to the contaminated site. The individuals in this factor also reject communication through long-distance media channels (social media, websites, print and digital media), indicating a preference for in-person risk communication with groups and representatives of the involved parties, which they consider more important than other forms of communication.

This position aligns with the fact that when risk information is disseminated through news media, it often focuses on negative situations, conflicts, and dramas, which can alter the meaning of the message and negatively shape the risk perception of those involved. Furthermore, “many risk-related news stories contain substantial omissions or present simplistic views, distortions, and inaccuracies” (VELOSO NETO; AREOSA, 2017VELOSO NETO, H.; AREOSA, J. Comunicação do risco como instrumento de mobilização e participação de trabalhadores/as. In: VELOSO NETO, H. et al. (eds.), Liderança e Participação em Segurança e Saúde no Trabalho. Porto: Civeri Publishing, 2017. p. 131-154., p. 142, translated)

The individuals in this factor show a slight rejection of openly providing information to the public, as well as offering information about potential health impacts on the affected population. On one hand, these facts can be seen as non-transparent communication, but on the other hand, they can be viewed as a strategy to prevent panic, for example, by informing affected populations, whether potentially or not, that contaminants from contaminated areas often have carcinogenic effects and pose other cumulative health risks (HOU; AL-TABBAA, 2014HOU, D.; AL-TABBAA, A. Sustainability: A new imperative in contaminated land remediation. Environmental Science and Policy, v. 39, p. 25-34, 2014.). In this sense, it can be noted that in cases involving the risk of serious diseases, there is a subjective limit between the provision of information and avoiding panic among the affected population.

Factor 3 - Dialogue

Despite representing different viewpoints, the factors can contain similarities among them. Factors 2 and 3 share similarities in statements with a higher indication of agreement (close to 5+). For example, both consider it important to provide guidance on how to avoid exposure to contaminants and prioritize compliance with legal requirements. This factor is composed of five individuals - A2, A6, R2, R4, and R5 - and it concentrates the majority of group R, with three individuals. This factor is the only one that does not include any individuals from group C. Thus, it is possible to affirm that dialogue and public participation in risk communication are not seen as a priority for individuals in group C (consultants), who likely practice a more technical form of risk communication.

The individuals in this factor stand out for considering important the dialogue among stakeholders, without specifying how it should occur. It is the only factor that does not reject the idea of creating space for everyone to express their opinions on the environmental remediation strategy and share their views, concerns, and expectations. These results align the individuals in this factor with democratic risk communication. This is in line with what Martini Júnior (1995) states: there is a need for risk communication models based on dialogue and capable of integrating the interests, values, and concerns of the actors involved in decision-making, where representatives from different parties participate in the risk communication process, in order to encompass the various nuances that surround the risk.

Although not listed as one of the most important points, Factor 3 also stands out for valuing the communication of the contaminated site’s history and preserving the perception of the area’s safe use, even after environmental remediation.

Factor 4 - Partnerships

The “partnerships” factor represents the viewpoint of seven individuals: A1, A5, C1, C8, P2, P5, and P4, who consider important the establishment of partnerships for risk communication in contaminated sites. The selection of representatives from each involved party and joint communication with departments and environmental agencies are other important points for these individuals.

This group considers it equally important to communicate the contaminants present in the area and the risk to which affected individuals, whether potential or not, are exposed. When these aspects are not considered, risk communication tends to be less effective. An example described in the literature that seems to fit this situation is the contamination of the Basf/Shell companies in Paulínia (SP), where the companies did not provide information about the risks present in the contaminated area, nor did they take care of the health of the affected individuals. These facts triggered feelings of helplessness and of being harmed among the affected population (LOURENÇO, 2015LOURENÇO, M. R. Comunicação de risco como um processo: uma análise do caso Shell/Basf. 2015. 139 p. Dissertação (Mestrado em Gestão de Sustentabilidade) - Universidade Estadual de Londrina. Londrina, Paraná, 2015.).

Within this factor, maintaining the image of the institution responsible for the contaminated site should not be one of the objectives of risk communication.

Factor 5 - Health

In Factor 5, concern for the health of those involved is considered a matter of great importance. Therefore, Factor 5 is named “Health” and is common to five individuals - A4, A5, C2, C5, and R3. In this factor, importance is given to investigating, informing, and communicating the potential health impacts on the affected individuals. Transparent risk communication that complies with all legal requirements is also considered important, even in the absence of a well-established normative framework.

Chart 1 presents the groups of stakeholders analyzed in this research and the factors in which they are distributed.

Chart 1
Groups of stakeholders analyzed in this research and the factors considered important for effective risk communication

Consensus statements

The correlation between factors justifies the presence of statements that do not distinguish themselves in any of the factors, known as consensus statements. There can be consensus in agreement, in which all groups agree on the importance of the statement; consensus in disagreement, corresponding to statements with which all groups disagree; or consensus in neutrality of the statement. In this study, three consensus statements with significance p > 0.5 were identified, one indicating agreement and two indicating neutrality. In addition to the consensus statements, several other statements were identified that show agreement in four out of five factors, referred to as relevant statements. These are also mentioned as they represent important points for the majority of those involved.

The consensus of agreement among all factors is the importance of adapting language to different audiences. Regarding communication strategies, social networks are not indicated as an appropriate means of risk communication, being rejected by Factors 2, 3, and 5, and considered irrelevant for Factors 1 and 4. It is noteworthy that Factors 2, 3, and 5 include all individuals from the group responsible for contaminated sites. Therefore, it is one viewpoint of this group. Most of the factors also consider communication through the institution’s website and email as irrelevant (rejected by Factor 1). Newspapers and magazines are rejected by Factors 2 and 5 and considered irrelevant by the others. There is also a rejection of providing information through radio for Factors 1, 3, and 5, and is irrelevant for the other factors. Individual risk communication is rejected by Factors 2, 3, 4, and 5, and considered irrelevant by Factor 1. Furthermore, conducting risk communication in a non-individualized manner is a common viewpoint among all individuals responsible for managing contaminated sites, and those affected or potentially affected by the risk. This may be the case, from the viewpoint of those responsible for managing contaminated areas, because non-individualized communication can enhance communication security by minimizing rumors, as it ensures that everyone receives the same information, in addition to reducing costs. In the case of those affected or potentially affected by the risk, it is possible that they perceive themselves as less vulnerable with risk communication conducted in a more group-oriented or collective manner, where incompletely communicated information, for example, would have a greater chance of being identified by the group.

One consensus of neutrality is to clarify what acceptable risk is. In the words of one of the interviewees, “the acceptability of risk should be defined by those experiencing it” (individual belonging to group R, male, 49 years old). Risk acceptance is influenced by social, cultural, and sometimes individual contexts. For example, when a person is more vulnerable to risk due to a health problem, their perception of it will likely diverge from the definitions established by others involved in risk analysis.

The third consensus statement, also of neutrality, relates to communicating strategies for mitigating environmental and social damage, without any notable acceptance or rejection within the set of statements.

Additionally, when analyzing Chart 1, it can be observed that representatives from the environmental agency and affected parties are distributed among Factors 1, 3, 4, and 5, which give greater importance to risk assessment and communication actions related to health. Thus, it can be stated that this is a common viewpoint among individuals from the environmental agency and those affected by the risk. Only Factor 2 does not consider any statement related to health assessment of the affected individuals to be important, with most statements of that nature being considered irrelevant.

Furthermore, all factors demonstrate neutrality regarding the provision of health teams and the promotion of health education strategies for the community. It should be emphasized that this data does not invalidate the importance given to the health of those affected, as described in Factor 5, but it shows that priority is given to investigating and communicating health aspects, without defining more direct and assertive actions. Additionally, community-based health promotion strategies are not valued, even though they could be an alternative for dealing with these cases. This reflects what is usually done in contaminated sites, where some health procedures have been developed for specific cases of contaminated areas, involving institutions such as municipal governments and the Ministry of Health through the Unified Health System (SUS), as in the case of Mansões Santo Antônio neighborhood (MUNICIPALITY OF CAMPINAS, 2007).

For individuals in Factors 1, 3, 4, and 5, the institution responsible for the contaminated site should not have autonomy to conduct risk communication, which is considered irrelevant for Factor 2. Consistently, the same individuals in these factors see the need to comply with legal requirements. This is also the viewpoint of the group representing environmental agencies, as well as three out of four individuals from the group of those affected or potentially affected by the risk, and four out of five individuals responsible for managing contaminated sites.

It is noteworthy that in all factors there is rejection or indifference among the stakeholders regarding environmental remediation strategies. Therefore, this corresponds to a common viewpoint, including those affected or potentially affected by the risk. This consensus among different stakeholders may be related to the perception that the affected individuals supposedly lack technical knowledge about environmental remediation methods. This is an element that characterizes technical risk communication. However, in many situations where contamination is present in residences or their vicinity, remediation strategies can directly affect the routine of these people, and at least in this aspect, they are fully capable of expressing their opinions.

The need to safeguard the aesthetics of contaminated sites was considered irrelevant by Factor 1 and strongly rejected by the other factors.

The notion of saving financial resources in risk communication is unanimously rejected by all factors, particularly Factors 1 and 4, which strongly oppose this idea. Additionally, these two factors also reject the communication of economic impacts. Upon examining Chart 1, it becomes evident that individuals in group P are exclusively concentrated in Factors 1 and 4, indicating a shared perspective within group P. In other words, the representatives of the environmental agency do not consider it crucial to communicate the economic impacts to those affected during risk communication.

Chart 2 provides a comprehensive overview of the distinguishing features of each factor and presents the consensus statements.

Chart 2
Summary of factor characteristics, consensus and relevant statements identified as important for effective risk communication

The perspectives of the stakeholders: technical or democratic risk communication?

The stakeholders’ viewpoints encompassed both the technical and democratic dimensions of risk communication. Notably, aspects of democratic risk communication were found in Factors 1, 3, and 4. As mentioned previously, these factors emphasize the importance of engaging in multilateral dialogue (Factor 3), fostering mutual trust among stakeholders, promoting transparency, and demonstrating a genuine interest in resolving the underlying issues (Factor 1). Additionally, Factor 4 emphasize the establishment of partnerships.

It was not possible to establish a clear predominance of either technical or democratic risk communication in the other factors. All factors exhibit characteristics that sometimes align them more closely with technical communication, and at other times with democratic risk communication. In Factor 2, for example, it can be noted that open provision of information is not a priority, which aligns it more closely with technical risk communication. However, at the same time, it demonstrates an interest in problem-solving through participatory means, which aligns it more closely with democratic risk communication.

Final Remarks

In this study, it was possible to observe that the participants, referred to as judges, belonging to the four groups of actors involved in the risk communication process defined for the research - those affected or potentially affected by the risk, managers of the environmental liability, consultants on contaminated sites, and representatives of environmental agencies - present different views on what would constitute effective risk communication. These views correspond to the five factors identified from the statistical analysis conducted: 1 - Trust, 2 - Secure Communication, 3 - Dialogue, 4 - Partnerships, and 5 - Health, each representing a specific perspective for effective risk communication. In general, the groups of environmental liability managers and representatives of environmental agencies presented distinct positions: for the former, the factors “Secure Communication,” “Dialogue,” and “Health” were considered important, while for the latter, the factors “Trust” and “Partnerships” assumed greater importance. Furthermore, the environmental liability managers have an agreement to comply with legal requirements. The representatives of the environmental agency, on the other hand, strongly agree with the idea that the costs of communication should not be minimized and that the economic impacts of remediation should not be communicated.

The groups of those affected or potentially affected and consultants presented varied viewpoints. Those affected or potentially affected by the risk share diffuse perspectives among four factors (Trust, Dialogue, Partnerships, and Health). They agree on the importance of monitoring the health of those affected by the risk, while social and environmental aspects and interest in resolving the presented problems were deemed irrelevant. The consultants, on the other hand, have dispersed viewpoints across four factors: Trust, Secure Communication, Partnerships, and Health. In general, this group agrees on providing guidance on strategies to minimize exposure to contaminants in the area, as well as communicating aspects related to the health of those affected, although it is not a consensus. For this group of stakeholders, promoting dialogue among the involved parties is of little importance.

In addition to the aforementioned five factors, there are consensus statements that reflect the agreement among individuals across all factors. One agreement it the high relevance of tailoring the language used to different audiences. A majority of respondents recommend avoiding risk communication through social networks, emails, and the website of the institution responsible for environmental liability, as well as avoiding individual-based approaches. Furthermore, there is a notable lack of concern or indifference towards the aesthetic aspect of the area. Additionally, a noteworthy aspect is the prioritization of the health of those involved, as evidenced by the recognition of the importance of communicating information about exposure to contaminants and strategies to minimize risks. This perspective holds considerable weight among most judges.

Finally, it should be noted that the absence of a legal framework on risk communication generates dissatisfaction among the involved parties in all analyzed groups, particularly among the group responsible for contaminated sites and those affected or potentially affected by the risk. Presumably, the presence of specific legislation would provide greater security for the development and adoption of risk communication strategies by the party responsible for the contaminated site, protecting the institution. At the same time, it could provide mechanisms for those affected and representatives of environmental agencies to demand actions from the responsible party for the environmental liability, ensuring that the measures imposed by the legislation are taken.

Acknowledgments

The authors would like to express their gratitude to all the participants who were interviewed for their collaboration in the research. They also extend their thanks to the support provided by the grant process number 2020/15434-0 from the São Paulo Research Foundation (Fapesp).

References

  • ARAÚJO, Antônia Angélica Correia. O Brasil no contexto do gerenciamento de áreas contaminadas: Um Olhar Crítico para as Regiões Desiguais do País. Dissertação (Mestrado em Engenharia Civil, Recursos Hídricos e Saneamento). Universidade Federal de Alagoas, Maceió, 2014.
  • ARAÚJO-MOURA, Antônia Angélica Correia; CAFFARO FILHO, Roberto Augusto. Panorama do gerenciamento de áreas contaminadas no Brasil após a resolução Conama 420/09. Águas Subterrâneas, v. 29, n. 2, p. 202-2012, 2015.
  • AREOSA, J. O risco no âmbito da teoria social. VI Congresso Português de Sociologia - Mundo sociais: saberes e práticas. Lisboa: Universidade Nova de Lisboa, Faculdade de Ciências Sociais e Humanas, 2008.
  • BECK, U. Sociedade de risco: rumo a uma outra modernidade. São Paulo: Editora 34, 2011.
  • BRASIL. Conselho Nacional do Meio Ambiente - Conama. Resolução n.º 420, de 28 de dezembro de 2009. Dispõe sobre critérios e valores orientadores de qualidade do solo quanto à presença de substâncias químicas e estabelece diretrizes para o gerenciamento ambiental de áreas contaminadas por essas substâncias em decorrência de atividades antrópicas. Diário Oficial da União: Seção 1, Brasília, DF, n. 249, p. 81-84, 30 dez. 2009. Disponível em: http://www.mma.gov.br/port/conama/legiabre.cfm?codlegi=620 Acesso em: 15 ago 2019.
    » http://www.mma.gov.br/port/conama/legiabre.cfm?codlegi=620
  • BRAUN, A. B. et al. Relevance of sustainable remediation to contaminated sites manage in developed and developing countries: Case of Brazil. Land Use Policy, v. 94, p. 104533, 2020.
  • BROWN, M. Illuminating patterns of perception: An overview of Q methodology. Carnegie-Mellon Univ Pittsburgh PA Software Engineering Inst, 2004.
  • BROWN, S. R. A Primer on Q methodology: Operant Subjectivity. New Haven: Yale University Press, 1993.
  • BROWN, S. R. Political subjectivity: Applications of Q methodology in political science. New Haven: Yale University Press, 1980.
  • COVELLO, V. T. Risk communication, radiation, and radiological emergencies: strategies, tools, and techniques. Health physics, v. 101, n. 5, p. 511-530, 2011.
  • DI GIULIO, G. M. Comunicação e governança do risco: exemplos de comunidades expostas à contaminação por chumbo no Brasil e Uruguai. 2010. Tese (Doutorado em Ambiente e Sociedade) -Instituto de Filosofia e Ciências Humanas, Universidade Estadual de Campinas, Campinas, São Paulo, 2010.
  • DI GIULIO, Gabriela Marques, FIGUEIREDO, Bernardino Ribeiro; FERREIRA, Lúcia da Costa; ANJOS, José Ângelo Sebastião Araújo dos. Experiências brasileiras e o debate sobre comunicação e governança do risco em áreas contaminadas por chumbo. Ciência & Saúde Coletiva, v. 17, p. 337-349, 2012.
  • FEAM - Fundação Estadual do Meio Ambiente de Minas Gerais. Inventário de áreas contaminadas do estado de Minas Gerais: 2019. Belo Horizonte: FEAM, 2019.
  • FIORINO, Daniel J. Technical and Democratic Values in Risk Analysis 1. Risk Analysis, v. 9, n. 3, p. 293-299, 1989.
  • FISCHHOFF, Baruch. Risk perception and communication. Oxford textbook of public health, Volume 2: the methods of public health. Oxford: Oxford University Press, 2009, p. 940-953.
  • GENUIS, S K.; JARDINE, C. G. Civil society organizations: providing and exchanging information about environmental health risk. In: Proceedings of ISIC: the information behaviour conference, Leeds, 2-5 September, 2014: Part 2, v. 20, n. 12015.
  • HOU, D.; AL-TABBAA, A. Sustainability: A new imperative in contaminated land remediation. Environmental Science and Policy, v. 39, p. 25-34, 2014.
  • LEISS, W. Three phases in the evolution of risk communication practice. Annals of the American Academy of Political and Social Science, , Challenges in Risk Assessment and Risk Management, v. 545, p. 85-94, 1996.
  • LIEBER, Renato Rocha; ROMANO-LIEBER, Nicolina Silvana. O conceito de risco: Janus reinventado. In: Minayo, M. C. S.; Miranda, A. C. Saúde e ambiente: estreitando nós. Rio de Janeiro: Fiocruz, 2002. p. 69-112.
  • LINDENFELD, L. et al. Risk communication and sustainability science: lessons from the field. Sustainability science, v. 9, n. 2, p. 119-127, 2014.
  • LOURENÇO, M. R. Comunicação de risco como um processo: uma análise do caso Shell/Basf. 2015. 139 p. Dissertação (Mestrado em Gestão de Sustentabilidade) - Universidade Estadual de Londrina. Londrina, Paraná, 2015.
  • LOURENÇO-LEITZKE, M. R; MARCHIORI, M. Comunicação de risco: um descompasso entre discurso e prática no caso Shell/Basf. Revista Internacional de Relaciones Públicas, v. 6, n. 12, p. 107-124, 2016
  • MARTINI JÚNIOR, L. C. et al. Comunicação de riscos tecnológicos ambientais. Production, v. 5, n. 2, p. 135-143, 1995.
  • NIELSON, Norma L.; KLEFFNER, Anne E.; LEE, Ryan B. The evolution of the role of risk communication in effective risk management. Risk management and insurance review, v. 8, n. 2, p. 279-289, 2005.
  • OLIVEIRA, U. V.; FREITAS, J. G.; COSTA, R. C. Stakeholder View of Efficient Risk Communication in Contaminated Sites. In: Manuel Abrunhosa; António Chambel; Silvia Peppoloni; Helder I. Chaminé. (Org.). Advances in Science, Technology & Innovation. 1ed.: Springer International Publishing, 2021, v. 1, p. 115-118.
  • OLIVEIRA, Uilians Vieira. A visão dos envolvidos para uma comunicação de risco eficiente: investigação metodológica. 2020. 124 f. Dissertação (Mestrado em Ciências), Universidade Federal de São Paulo, Diadema, 2020. Available at: https://repositorio.unifesp.br/xmlui/handle/11600/64297
    » https://repositorio.unifesp.br/xmlui/handle/11600/64297
  • PANAGOS, Panos; VAN LIEDEKERKE, Marc; YIGINI, Yusuf; MONTANARELLA, Luca. Contaminated sites in Europe: review of the current situation based on data collected through a European network. Journal of Environmental and Public Health, v. 2013, 2013. 11 p.
  • PEPPOLONI, S.; CHAMINÉ, H.I. (Eds.) Advances in Geoethics and Groundwater Management: Theory and Practice for a Sustainable Development. Cham: Springer International Publishing AG, 2021. p. 115-118.
  • PREFEITURA MUNICIPAL DE CAMPINAS. Protocolo de atenção à saúde das populações expostas aos contaminantes ambientais no bairro Mansões Santo Antonio, Campinas - São Paulo. Março de 2007. Disponível em: https://saude.campinas.sp.gov.br/saude/visa/vig_ambiental/manuais/04_Protocolo_3_modificado_maro07.pdf
    » https://saude.campinas.sp.gov.br/saude/visa/vig_ambiental/manuais/04_Protocolo_3_modificado_maro07.pdf
  • QUEIRÓS, Margarida; VAZ, Teresa; PALMA, Pedro. Uma reflexão a propósito do risco. Lisboa: Centro de Estudos Geográficos. v. 10, n. 7, p. 2009, 2006.
  • RAMIREZ-ANDREOTTA, M. D. et al. Environmental research translation: Enhancing interactions with communities at contaminated sites. Science of the Total Environment, v. 497, p. 651-664, 2014.
  • RODRIGUES, Letícia Stevanato; ZANIRATO, Silvia Helena. Contaminação, comunicação e remediação dos riscos ambientais no conjunto Heliópolis-Gleba L-SP: um estudo das representações dos moradores do local. IX Congresso Brasileiro de Gestão Ambiental. São Bernardo, 2018. 10 p.
  • ROWAN, K. E. The technical and democratic approaches to risk situations: Their appeal, limitations, and rhetorical alternative. Argumentation, v. 8, n. 4, p. 391-409, 1994.
  • SÃO PAULO (Estado). Decreto n.º 59.263, de 5 de junho de 2013. Regulamenta a Lei n.º 13.577, de 8 de julho de 2009, que dispõe sobre diretrizes e procedimentos para a proteção da qualidade do solo e gerenciamento de áreas contaminadas e dá providências correlatas. Diário Oficial do Estado, Seção 1, São Paulo, n. 127, p. 123, 6 jun. 2013.
  • SATO, A. Understanding effective risk communication in the context of a radiological accident. Fukushima Global Communication Programme Working Paper. N. 7. 2015. 17 p.
  • SILVA, Renan Finamore Gomes. Gestão de áreas contaminadas e conflitos ambientais: o caso da Cidade dos Meninos. 2007. 118 f. Tese (Doutorado em Engenharia), Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2007.
  • SPÍNOLA, Ana Luiza Silva. Inserção das áreas contaminadas na gestão municipal: desafios e tendências. 2011. Tese de Doutorado. Faculdade de Saúde Pública, Universidade de São Paulo, São Paulo, São Paulo, 2011.
  • TEIXEIRA, Cláudia Echevenguá; MOTTA, Flávia Gutierrez; MORAES, Sandra Lúcia de. Panorama GAC: mapeamento da cadeia de gerenciamento de áreas contaminadas. 1. ed. São Paulo: IPT, 2016.
  • VELOSO NETO, H.; AREOSA, J. Comunicação do risco como instrumento de mobilização e participação de trabalhadores/as. In: VELOSO NETO, H. et al. (eds.), Liderança e Participação em Segurança e Saúde no Trabalho. Porto: Civeri Publishing, 2017. p. 131-154.
  • VICTOR, Cilene. Comunicação de riscos de desastres no contexto das mudanças climáticas: muito além do jornalismo. XXXVIII Congresso Brasileiro de Ciências da Comunicação. Rio de Janeiro, RJ, 4 - 7 set. 2015. 15 p.

Publication Dates

  • Publication in this collection
    10 Nov 2023
  • Date of issue
    2023

History

  • Received
    22 Apr 2022
  • Accepted
    09 Feb 2023
ANPPAS - Revista Ambiente e Sociedade Anppas / Revista Ambiente e Sociedade - São Paulo - SP - Brazil
E-mail: revistaambienteesociedade@gmail.com