Acessibilidade / Reportar erro

A design process to define public challenges addressing SDGs

Un proceso de diseño para definir los desafíos públicos relacionados con los ODS

Abstract

How might we define local innovation challenges that are user-centered but also address strategic goals, such as the Sustainable Development Goals (SDGs)? We propose a design process that introduces a relevant contribution to the practices of the Mission-oriented Innovation Policies (MIPs) field as it goes from Grand Societal Challenges (GSCs) to local innovation challenges, which are more relevant to local needs and capabilities of solution. The main goal of the design process is to go from a GSC, such as “access to health” and one of its missions, like “reducing chronic Non-Communicable Diseases (NCD),” to a more able-to-solve innovation challenge, passing by its mission, and different problems and aspects. Snowball sampling methods allow us to find and mobilize electable subjects and design thinking techniques to process problem mapping, election, and clustering, as well as create challenge statements. It was possible to translate GSC to the local reality while reducing contestation and complexity and producing a problem-grouped map with prioritization and three problem statements (challenges) with open phrasing and solution criteria to foster the users’ desired outcomes.

Keywords:
process; challenges; SDG; design thinking; mission-oriented innovation policies

Resumen

¿Cómo podemos definir desafíos locales de innovación local que estén centrados en el usuario pero que también aborden objetivos estratégicos, como los Objetivos de Desarrollo Sostenible (ODS)? Proponemos un proceso de diseño que introduce una contribución relevante a las prácticas en el campo de las políticas de innovación orientadas por misión (en inglés, Mission-oriented Innovation Policies - MIPs), ya que va desde los grandes desafíos sociales (en inglés, Grand Societal Challenges - GSCs), comenzando con los ODS, hasta los desafíos locales de innovación, que son más relevantes para necesidades locales y capacidades de solución. El objetivo principal del proceso de diseño es ir desde un GSC, como “acceso a la salud” y una de sus misiones, como “reducir las Enfermedades Crónicas No Transmisibles (ECNT)”, a un desafío de innovación más concreto, pasando por la misión, y por diferentes problemas y aspectos. Los métodos de muestreo de bola de nieve nos permiten encontrar y movilizar sujetos elegibles, y técnicas de design thinking, procesar el mapeo, la elección y la agrupación de problemas; y crear declaraciones de desafío. Fue posible traducir GSC a la realidad local, al mismo tiempo que se reducía la contestación y la complejidad, y se producía un plano agrupado de problemas con priorización, y tres declaraciones de problemas (desafíos) con redacción abierta y criterios de solución para promover los resultados deseados por los usuarios.

Palabras clave:
proceso; desafíos; ODS; pensamiento de diseño; políticas de innovación orientadas a la misión

Resumo

Como podemos definir os desafios locais de inovação que são centrados no usuário, mas também abordam objetivos estratégicos, como os Objetivos de Desenvolvimento Sustentável (ODS)? Propomos um processo de design que apresenta uma contribuição relevante para as práticas no campo das políticas de inovação orientadas para a missão (em inglês, Mission-oriented Innovation Policies - MIPs), uma vez que vai dos grandes desafios societais (em inglês, Grand Societal Challenges - GSCs) aos desafios locais de inovação, que são mais relevantes para as necessidades locais e capacidades de solução. O principal objetivo do processo de desenho é passar de um GSC, como o “acesso à saúde” e uma de suas missões, como “reduzir as Doenças Crônicas Não Transmissíveis (DCNT)”, para um desafio de inovação mais capaz de resolver, passando pela sua missão, diferentes problemas e aspectos. Os métodos de amostragem bola de neve nos permitem encontrar e mobilizar assuntos elegíveis e técnicas de design thinking para processar o mapeamento de problemas, eleição e agrupamento; e criar declarações de desafio. Foi possível traduzir o GSC para a realidade local, reduzindo a contestação e a complexidade, e produzindo um mapa agrupado de problemas com priorização e três declarações de problemas (desafios) com formulação aberta e critérios de solução para promover os resultados desejados pelos usuários.

Palavras-chave:
processo; desafios; ODS; pensamento de design; políticas de inovação orientadas para a missão.

1. INTRODUCTION

How might we define local innovation challenges that are user centred but also address strategic goals, such as the Sustainable Development Goals (SDGs)? We believe that the answer to the question is directly related to the capabilities of public institutions to develop strategic goals (or missions) for public innovation policies to achieve and coherently define local innovation challenges from Grand Societal Challenges (GSC)1 1 Can be also referred as Grand Social Challenge, Wicked Issue, or Grand Challenge. .

Understanding that the target problems for public institutions cannot be handled with a single solution nor have definitive answers, Mission-oriented Innovation Policies (MIPs)2 2 Can be also referred as MOIP. perspective proposes not only a shift from “technology-led missions” (supply) to “societal challenge-led mission” (demand), but also a change in the way public challenges are comprehended (Janssen, Torrens, Wesseling, & Wanzenböck, 2021Janssen, M. J., Torrens, J., Wesseling, J. H., & Wanzenböck, I. (2021). The promises and premises of mission-oriented innovation policy - a reflection and ways forward. Science and Public Policy, 48(3), 438-444. Retrieved from https://doi.org/10.1093/scipol/scaa072
https://doi.org/10.1093/scipol/scaa072...
; Mazzucato 2018Mazzucato, M. (2018, October). Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change, 27(5), 803-815. Retrieved from https://doi.org/10.1093/icc/dty034
https://doi.org/10.1093/icc/dty034...
; Miedzinski, Mazzucato, & Ekins, 2019Miedzinski, M., Mazzucato, M., & Ekins, P. (2019). A framework for mission-oriented innovation policy roadmapping for the SDGs: the case of plastic-free oceans (IIPP WP 2019-03). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
; Wanzenböck, Wesseling, Frenken, Hekkert, & Weber, 2020).

Since GSC are ambivalent and have emergent components that may only reveal themselves later, innovation needs to be addressed throughout a learning cycle of creating and testing different solutions (Rizardi & Metello, 2022Rizardi, B., & Metello, D. (2022). Design sistêmico: abraçando a complexidade no setor público. Brasília, DF: Enap.), keeping focus on strategic goals, or “missions”, that together will address the GSC (Miedzinski et al., 2019Miedzinski, M., Mazzucato, M., & Ekins, P. (2019). A framework for mission-oriented innovation policy roadmapping for the SDGs: the case of plastic-free oceans (IIPP WP 2019-03). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
). The fact that MIP concentrates efforts on how to (i) formulate and design innovation policies, and (ii) select and plan missions and projects shows limitations on how to put mission-oriented innovation policies to work (Roth, Wittmann, Hufnagl, & Lindner, 2022Roth, F., Wittmann, F., Hufnagl, M., & Lindner, R. (2022). Putting mission-oriented innovation policies to work: a case study of the German high-tech strategy 2025 (Fraunhofer ISI Discussion Papers, Innovation Systems and Policy Analysis, n. 75). Karlsruhe, Germany: Fraunhofer ISI. Retrieved from http://hdl.handle.net/10419/262141
http://hdl.handle.net/10419/262141...
), i.e., make the movement from GSC to more able-to-solve and user centered local innovation challenges (Brown, 2020Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Rio de Janeiro, RJ: Elsevier.; Wanzenböck et al., 2020Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
https://doi.org/10.1093/scipol/scaa027...
).

Considering this, we offer a design process that contributes to the practice of MIP as it goes from GSC, starting with SDGs, to innovation challenges more relevant to local needs and capabilities of solution, by combining MIP and the design thinking perspective of user centered innovation challenges (Brown, 2020Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Rio de Janeiro, RJ: Elsevier.), developing a multilevel approach that uses snowball sampling methods to find electable subjects from organizations that work with SDGs or that deal with social problems; and common design techniques to map, elect and group problems, and create challenge statements.

2. LITERATURE REVIEW

The notion of MIP is facing a “re-emerging interest” (Wanzenböck et al., 2020Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
https://doi.org/10.1093/scipol/scaa027...
, p. 475) with the reuse of its perspective in different countries and the recent production of technical guides or policy papers, like the Observatory of Public Sector Innovation (OPSI)3 3 Retrieved from https://oecd-opsi.org/ , the Mission-Oriented Innovation Policies Online Toolkit4 4 Retrieved from https://stip.oecd.org/MIP/the-overall-methodological-framework , the Innovation Facets Methodology5 5 Retrieved from https://oecd-opsi.org/pet/ 6 6 Retrieved from https://oecd-opsi.org/wp-content/uploads/2021/10/OECD-Innovation-Facets-Brief-Mission-Oriented-Innovation-2021.pdf , the UCL policy briefs published by the Institute for Innovation and Public Purpose (IIPP), or the Brazilian case, the Centro de Gestão e Estudos Estratégicos (CGEE) (Mazzucato & Penna, 2016Mazzucato, M., & Penna, C. (2016). The Brazilian innovation system: a mission-oriented policy proposal. Sumário executivo. Brasília, DF: Centro de Gestão e Estudos Estratégicos. Retrieved from https://www.cgee.org.br/documents/10195/1774546/Sistema_Brasileiro_de_Inovacao-Mazzucato_Penna-Sumario_Executivo.pdf
https://www.cgee.org.br/documents/10195/...
). There is also an extensive discussion on scientific papers: Albala, Holloway, Austin, and Kattel (2021Albala, S., Holloway, C., Austin, V., & Kattel, R. (2021). New economics of assistive technology: a call for a missions approach (IIPP WP 2021/04). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/wp2021-04
https://www.ucl.ac.uk/bartlett/public-pu...
), Larrue (2021Larrue, P. (2021). The design and implementation of mission-oriented innovation policies: a new systemic policy approach to address societal challenges (ECD Science, Technology and Industry Policy Papers, n. 100). Paris, France: OECD Publishing. Retrieved from https://doi.org/10.1787/23074957
https://doi.org/10.1787/23074957...
), Leadbeater (2018Leadbeater, C. (2018). Movements with missions make markets (IIPP WP 2018-07). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/wp2018-07
https://www.ucl.ac.uk/bartlett/public-pu...
), Mazzucato and Dibb (2019)Mazzucato, M., & Dibb, G. (2019). Missions: a beginner’s guide (IIPP PB 09).. London, UK: UCL Institute for Innovation and Public Purpose Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/iipp_policy_brief_09_missions_a_beginners_ guide.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
, Mazzucato and Macfarlane (2019)Mazzucato, M., & Macfarlane, L. (2019). A mission-oriented framework for the Scottish national investment bank (IIPP WP 2019-02). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/publicpurpose/wp2019-02
https://www.ucl.ac.uk/bartlett/publicpur...
, and Miedzinski et al. (2019Miedzinski, M., Mazzucato, M., & Ekins, P. (2019). A framework for mission-oriented innovation policy roadmapping for the SDGs: the case of plastic-free oceans (IIPP WP 2019-03). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
) on technical discussion; Janssen et al. (2021Janssen, M. J., Torrens, J., Wesseling, J. H., & Wanzenböck, I. (2021). The promises and premises of mission-oriented innovation policy - a reflection and ways forward. Science and Public Policy, 48(3), 438-444. Retrieved from https://doi.org/10.1093/scipol/scaa072
https://doi.org/10.1093/scipol/scaa072...
), and Robinson and Mazzucato (2019Robinson, D. K. R., & Mazzucato, M. (2019, May). The evolution of mission-oriented policies: exploring changing market creating policies in the US and European space sector. Research Policy, 48(4), 936-948. Retrieved from https://doi.org/10.1016/j.respol.2018.10.005
https://doi.org/10.1016/j.respol.2018.10...
), on definitions and classification; Mazzucato (2018)Mazzucato, M. (2018, October). Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change, 27(5), 803-815. Retrieved from https://doi.org/10.1093/icc/dty034
https://doi.org/10.1093/icc/dty034...
and Mazzucato, Kattel, and Ryan-Collins (2020)Mazzucato, M., Kattel, R., & Ryan-Collins, J. (2020). Challenge-driven innovation policy: towards a new policy toolkit. Journal of Industry, Competition and Trade, 20, 421-437. Retrieved from https://doi.org/10.1007/s10842-019-00329-w
https://doi.org/10.1007/s10842-019-00329...
, prescriptions and toolkits; Wanzenböck et al. (2020)Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
https://doi.org/10.1093/scipol/scaa027...
, on problem-solution space; and, in Brazilian context, Lucena (2022Lucena, R. M. (2022). Missão em países em desenvolvimento: o caso brasileiro pelo recorte do plano inova empresa. Revista de Administração, Sociedade e Inovação, 8(3), 68-86. Retrieved from https://doi.org/10.20401/rasi.8.3.670
https://doi.org/10.20401/rasi.8.3.670...
), on the analysis of a specific innovation policy.

These studies show two relevant limitations: the drill-down from missions to problems and from them to local innovation challenges, and the lack of user centered perspective. The first aspect is a persistent concern when grand strategies need to be placed locally, as detailed by Herrera (2019Herrera, V. (2019, June). Reconciling global aspirations and local realities: challenges facing the sustainable development goals for water and sanitation. World Development, 118, 106-117. Retrieved from https://doi.org/10.1016/j.worlddev.2019.02.009
https://doi.org/10.1016/j.worlddev.2019....
), or by Leavesley, Trundle, and Oke (2022Leavesley, A., Trundle, A., & Oke, C. (2022). Cities and the SDGs: realities and possibilities of local engagement in global frameworks. Ambio, 51(6), 1416-1432. Retrieved from https://doi.org/10.1007/s13280-022-01714-2
https://doi.org/10.1007/s13280-022-01714...
). The second is also recurrent due to the presence of an innovation perspective that is more connected to broad projects than to contextual and user centered challenges.

The innovative aspect of this paper is offering a design process that contextualizes GSC with local problems and defines problems in a user centered and open perspective (Brown, 2020Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Rio de Janeiro, RJ: Elsevier.; Dyer, Ding, & Sun, 2013Dyer, S. W., Ding, B., & Sun, J. (2013, January 23). The innovator’s secret weapon. Kearney. Retrieved from https://www.es.kearney.com/operations-performance-transformation/article/-/insights/the-innovator-s-secret-weapon
https://www.es.kearney.com/operations-pe...
; Guimarães, Carício, & Oliveira, 2017Guimarães, P. B. V., Carício, M. R., & Oliveira, A. C. S. (2017). Inovação no setor público com estratégias de design thinking (Série Inovação no Setor Público, Vol. 1). Salvador, BA: Ed. Motres. Retrieved from https://sigaa.ufrn.br/sigaa/verProducao?idProducao=7579315&key=b69666 36f4dbb70a4033b0790a09fd24
https://sigaa.ufrn.br/sigaa/verProducao?...
).

3. RESEARCH METHOD

According to Tripp (2005Tripp, D. (2005). Action Research: a methodological introduction. Research and Education, 31(3), 443-466. Retrieved from https://doi.org/10.1590/S1517-97022005000300009
https://doi.org/10.1590/S1517-9702200500...
), action research is a methodology that tries to answer the question: “how might I/we improve this practice?” (Tripp, 2005, p. 15), by building a cycle of plan, act, describe and evaluate steps. Our practice in need of improvement was the definition of innovation challenges, which we felt was focused on finding innovation challenges that were locally relevant and user centered, lacking the connection with grand challenges.

One example is the process to define innovation challenges on call for proposals to Edital FACEPE Nº 21/20217 7 Retrieved from https://www.facepe.br/wp-content/uploads/2021/10/Edital_FACEPE_21-2021-Pro-Startups-Operação1.pdf : organization team selected the main themes based on Pernambuco Strategy; representatives of public and private institutions were asked to submit challenges related to the themes; a small committee selected the better challenges. This resulted in absence of a clear mission, disconnection with grand challenges, and narrow participation of users.

In face of these limitations, we built a plan of studying literature on MIP to create a new practice for challenge definition. Upon studying MIP, we found out that there was no “recipe” to be implemented, since, as stated on section 1 and 2, literature was focused on mission projects. Hence, we decided to produce a “practical action research” (Tripp, 2005Tripp, D. (2005). Action Research: a methodological introduction. Research and Education, 31(3), 443-466. Retrieved from https://doi.org/10.1590/S1517-97022005000300009
https://doi.org/10.1590/S1517-9702200500...
, p. 13), in which the researchers craft a new solution to a practice, having a specific goal and question in mind. Our question was “How might we define local innovation challenges that are user centered but also address strategic goals, such as the Sustainable Development Goals?” and our goal was to go GSC to more able-to-solve innovation challenges, passing by missions and its different problems and aspects, in attention to the criteria of large user participation, preferably in synchronous activities.

To better describe and evaluate, we kept notes of every encounter that happened on five cycles (one per GSC) of 3 hours hands-on activities, in which we tried to go from GSC to 3 challenge statements with solution criteria. We applied different design techniques (Brown, 2020Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Rio de Janeiro, RJ: Elsevier.) (cf. Section 4), with a mature design process on iteration three of five.

In 2022, the organization team selected five GSC and declared mission statements using the road mapping framework (Miedzinski et al., 2019Miedzinski, M., Mazzucato, M., & Ekins, P. (2019). A framework for mission-oriented innovation policy roadmapping for the SDGs: the case of plastic-free oceans (IIPP WP 2019-03). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
) and evaluating criteria for MIPs (Mazzucato, 2018Mazzucato, M. (2018, October). Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change, 27(5), 803-815. Retrieved from https://doi.org/10.1093/icc/dty034
https://doi.org/10.1093/icc/dty034...
; Mazzucato & Dibb, 2019Lucena, R. M. (2022). Missão em países em desenvolvimento: o caso brasileiro pelo recorte do plano inova empresa. Revista de Administração, Sociedade e Inovação, 8(3), 68-86. Retrieved from https://doi.org/10.20401/rasi.8.3.670
https://doi.org/10.20401/rasi.8.3.670...
; Mazzucato & Macfarlane, 2019Mazzucato, M., & Penna, C. (2016). The Brazilian innovation system: a mission-oriented policy proposal. Sumário executivo. Brasília, DF: Centro de Gestão e Estudos Estratégicos. Retrieved from https://www.cgee.org.br/documents/10195/1774546/Sistema_Brasileiro_de_Inovacao-Mazzucato_Penna-Sumario_Executivo.pdf
https://www.cgee.org.br/documents/10195/...
; Mazzucato & Penna, 2016Mazzucato, M., & Penna, C. (2016). The Brazilian innovation system: a mission-oriented policy proposal. Sumário executivo. Brasília, DF: Centro de Gestão e Estudos Estratégicos. Retrieved from https://www.cgee.org.br/documents/10195/1774546/Sistema_Brasileiro_de_Inovacao-Mazzucato_Penna-Sumario_Executivo.pdf
https://www.cgee.org.br/documents/10195/...
). Following these two approaches, we associated GSC with SDGs, e.g., SDG 3 - “Good Health and Well-being” became the GSC on the design process to frame Health local innovation challenges. As the GSC is a big picture with many possible frames, we selected one locally relevant mission, e.g., “reducing chronic noncommunicable diseases.”

Between May 4th and June 2nd, we conducted the five workshops for the social inequalities call for proposals from the Foundation for the Support of Science and Technology of the State of Pernambuco (Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco - FACEPE). The GSC raised were: Poverty (SDG 1 and SDG 2), Access to Health (SDG 3), Equitable Education (SDG 4), Gender Equality (SDG 5), Sustainability (SDG 12, SDG 13, SDG 14, and SDG 15).

Before starting the workshops, as organizers, we selected participants based on a set of criteria through the snowball sampling technique (Biernacki & Waldorf, 1981Biernacki, P., & Waldorf, D. (1981). Snowball sampling: problems and techniques of chain referral sampling. Sociological Methods & Research, 10(2), 141-163. Retrieved from https://doi.org/10.1177/004912418101000205
https://doi.org/10.1177/0049124181010002...
). Those participants acted as hubs by indicating other people that fit the criteria, forming a network of contacts. This strategy enables workshop organizers to identify critical stakeholders, who could be neglected or simply not be easy to map and invite, because they are related to a rare phenomenon or do not want to join a study for specific reasons (Dragan & Isaic-Maniu, 2013Dragan, I. M., & Isaic-Maniu, A. (2013). Snowball sampling completion. Journal of Studies in Social Sciences, 5(2), 160-177. Retrieved from https://core.ac.uk/download/pdf/229607586.pdf
https://core.ac.uk/download/pdf/22960758...
). Such “active search” engages a rich population, although this may cause limitations in terms of generalizability, i.e., the selection of participants is based on invitations instead of a random sampling (Lopes, Rodrigues, & Sichieri, 1996Lopes, C. S., Rodrigues, L. C., & Sichieri, R. (1996, December). The lack of selection bias in a snowball sampled case-control study on drug abuse. International Journal of Epidemiology, 25(6), 1267-1270. Retrieved from https://doi.org/10.1093/ije/25.6.1267
https://doi.org/10.1093/ije/25.6.1267...
). We used time criteria to stop the search for new participants: all indications should be made up to 7 days before the workshop.

The study reported here considered two eligibility criteria: (i) act as state, municipal or federal public servant and (ii) be involved with the public policy and the population affected by its GSC. During the workshop, the challenge team used Google Jam board, and the organization team facilitated and registered the complementary information on a google docs. After the workshop, the organization team performed a debriefing reunion in which they debated the pros and cons of that iteration and built improvements for the next cycle. The activities are summarized below.

Box 1
Summary of Activities

In sum, we observed that our practice for challenge definition had no connection with GSC; posed the question “How might we define local innovation challenges that are user centered but also address strategic goals, such as the Sustainable Development Goals?”; built a plan, act, describe, evaluate cycle with 5 iteration; proposed a starting set of practice combining snowball sampling and user centered design technique, in a process of 2 main phases and 8 activities; evolved this starting set from iteration to iteration, resulting in the design process described in section 4.

4. A DESIGN PROCESS TO DEFINE PUBLIC CHALLENGES ADDRESSING SDGS

4.1 Description

The process starts in phase 1, Problem Framing (Figure 1), when the organizers present the institution and the methodology to participants (Activity 1), explaining the GSC that will guide the participants in future activities. Then, they organize participants in teams (Activity 2) that must identify specific problems of the mission (Activity 3), using the rule “one problem per card”. At this moment, they must consider two aspects (Wedell-Wedellsborg, 2021Wedell-Wedellsborg, T. (2021). Qual é o seu problema? Para resolver seus problemas mais difíceis, mude os problemas que você resolve. São José dos Campos, SP: Benvirá.):

  • Difficulties or barriers - when we try to accomplish this mission, what may go wrong? When do we stop? What hampers our activity?

  • Pains - once we get involved with the big challenge/mission, what bothers us? What is our pain? What takes our energy?

After that, organizers present the cards with the problems (Activity 4) and inspire participants to use the Affinity Diagram to classify the cards in groups according to similarities among them (Activity 5). This avoids repetition, identifies information saturation (Guimarães, Carício, & Oliveira, 2017Guimarães, P. B. V., Carício, M. R., & Oliveira, A. C. S. (2017). Inovação no setor público com estratégias de design thinking (Série Inovação no Setor Público, Vol. 1). Salvador, BA: Ed. Motres. Retrieved from https://sigaa.ufrn.br/sigaa/verProducao?idProducao=7579315&key=b69666 36f4dbb70a4033b0790a09fd24
https://sigaa.ufrn.br/sigaa/verProducao?...
; Pernice, 2018Pernice, K. (2018, February 18). Affinity diagramming for collaboratively sorting UX findings and design ideas. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles/affinity-diagram/
https://www.nngroup.com/articles/affinit...
), and reduces contestation and complexity (Wanzenböck et al., 2020Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
https://doi.org/10.1093/scipol/scaa027...
). In the following step, participants must select the groups of problems that seem to be more critical (Activity 6), using dot votes to distribute on cards that they perceive as more relevant. The top-3 most voted sets are taken to the next phase, when they are processed to finally become local innovation challenges (Gibbons, 2019Gibbons, S. (2019, July 07). Dot voting: a simple decision-making and prioritizing technique. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles/dot-voting/
https://www.nngroup.com/articles/dot-vot...
; Guimarães, Carício, Oliveira, 2017Guimarães, P. B. V., Carício, M. R., & Oliveira, A. C. S. (2017). Inovação no setor público com estratégias de design thinking (Série Inovação no Setor Público, Vol. 1). Salvador, BA: Ed. Motres. Retrieved from https://sigaa.ufrn.br/sigaa/verProducao?idProducao=7579315&key=b69666 36f4dbb70a4033b0790a09fd24
https://sigaa.ufrn.br/sigaa/verProducao?...
).

Once they start Phase 2, Problem Statement, each group of participants need to describe one of the selected groups of problems as a question in the form “How Might We?” (HWE), a technique adopted by Brown (2020Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Rio de Janeiro, RJ: Elsevier.) to guarantee that a problem statement is able-to-solve and is open enough to allow the design of new solutions.

In addition to HWE, participants use “desired outcome” (Dyer et al., 2013Dyer, S. W., Ding, B., & Sun, J. (2013, January 23). The innovator’s secret weapon. Kearney. Retrieved from https://www.es.kearney.com/operations-performance-transformation/article/-/insights/the-innovator-s-secret-weapon
https://www.es.kearney.com/operations-pe...
) to specify expected results, representing the success of the innovation solution in terms of two drivers: requirements (what is essential for the solution to resolve the challenge) and experience (characteristics the solution must have to be useful and attractive). At that time, the problem is not only framed in an open and positive way, but also is connected to success criteria that promote the achievement of a desired outcome.

Figure 1
The process

4.2 Results

Here, we describe the results related to the GSC “Access to Health” (SDG 3 - “Good Health and Well-being”), whose mission was “reducing chronic Non-Communicable Diseases (NCDs)” (Activity 1). The institutions that addressed this challenge were:

  • Pernambuco State Health Secretariat (SES);

  • Pernambuco State Planning and Management Secretariat (SEPLAG), represented by its Center of Results-based management in Health (NGR-SES);

  • Recife Municipal Government;

  • Serra Talhada Municipal Government;

  • Cardiovascular Health Center of Pernambuco (PROCAPE); and

  • Federal University of Pernambuco Clinical Hospital (HC).

The institutions were divided in groups according to: (i) direct or indirect relation to the Health System, (ii) jurisdiction of the institution - state, federal or municipal, and (iii) type of institution - Academy or Government; in a way that every group had approximately the same number of members, and a wide representation (Activity 2).

As presented in Figure 2, during the first part of the workshop, 10 participants (1 from PROCAPE, 3 from HC and 2 from each of the other institutions) identified 34 problems (Activities 3 and 4). These problems were grouped into 6 sets (Activity 5):

  • Access to health services,

  • Prevention;

  • Communication and Information;

  • Diseases;

  • Data and Information; and

  • Assistance.

Figure 2
Problems related to the SDG 3, according to participants

Then, participants undertook a prioritization process to select three groups they considered as the most relevant (Activity 6): Data and Information (9 votes), Assistance (6 votes) and Prevention (6 votes) were considered the top-3 groups to address. Finally, they translated each prioritized group as a question (Activity 7), which was associated with a set of desired outcomes (Activity 8). Figure 3 details the final questions (here named as “challenges”) with the requirements that guide the future innovation project that would address each of them.

Figure 3
Challenges in the form of questions with requirements for the future innovation project

5. CONCLUSION

The literature on MIP has provided important directions: guides to go from GSC to Mission, and then to Mission Projects (Mazzucato, 2018Mazzucato, M. (2018, October). Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change, 27(5), 803-815. Retrieved from https://doi.org/10.1093/icc/dty034
https://doi.org/10.1093/icc/dty034...
; Mazzucato et al., 2020Mazzucato, M., & Dibb, G. (2019). Missions: a beginner’s guide (IIPP PB 09).. London, UK: UCL Institute for Innovation and Public Purpose Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/iipp_policy_brief_09_missions_a_beginners_ guide.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
; Miedzinski et al., 2019Miedzinski, M., Mazzucato, M., & Ekins, P. (2019). A framework for mission-oriented innovation policy roadmapping for the SDGs: the case of plastic-free oceans (IIPP WP 2019-03). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
); discussion on background, concepts, typology and premises to identify MIP (Janssen et al., 2021Janssen, M. J., Torrens, J., Wesseling, J. H., & Wanzenböck, I. (2021). The promises and premises of mission-oriented innovation policy - a reflection and ways forward. Science and Public Policy, 48(3), 438-444. Retrieved from https://doi.org/10.1093/scipol/scaa072
https://doi.org/10.1093/scipol/scaa072...
; Robinson & Mazzucato, 2019Robinson, D. K. R., & Mazzucato, M. (2019, May). The evolution of mission-oriented policies: exploring changing market creating policies in the US and European space sector. Research Policy, 48(4), 936-948. Retrieved from https://doi.org/10.1016/j.respol.2018.10.005
https://doi.org/10.1016/j.respol.2018.10...
); pathways to deal with inherent aspects of wicked problems, such as contestation, complexity, and uncertainty (Wanzenböck et al., 2020Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
https://doi.org/10.1093/scipol/scaa027...
). These contributions are often described for national level and have the premise of broad public policies that are connected to Mission Projects, like “autonomous ocean stations to remove plastic pollution” (Mazzucato, 2018Mazzucato, M. (2018, October). Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change, 27(5), 803-815. Retrieved from https://doi.org/10.1093/icc/dty034
https://doi.org/10.1093/icc/dty034...
; Mazzucato et al., 2020Mazzucato, M., & Dibb, G. (2019). Missions: a beginner’s guide (IIPP PB 09).. London, UK: UCL Institute for Innovation and Public Purpose Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/iipp_policy_brief_09_missions_a_beginners_ guide.pdf
https://www.ucl.ac.uk/bartlett/public-pu...
). Yet they are not effective to define local/subnational public challenges, create user centered challenges.

Our design process was able to contribute with these two points. By using snowball sampling (Biernacki & Waldorf, 1981Biernacki, P., & Waldorf, D. (1981). Snowball sampling: problems and techniques of chain referral sampling. Sociological Methods & Research, 10(2), 141-163. Retrieved from https://doi.org/10.1177/004912418101000205
https://doi.org/10.1177/0049124181010002...
; Dragan & Isaic-Maniu, 2013Dragan, I. M., & Isaic-Maniu, A. (2013). Snowball sampling completion. Journal of Studies in Social Sciences, 5(2), 160-177. Retrieved from https://core.ac.uk/download/pdf/229607586.pdf
https://core.ac.uk/download/pdf/22960758...
; Lopes et al., 1996Lopes, C. S., Rodrigues, L. C., & Sichieri, R. (1996, December). The lack of selection bias in a snowball sampled case-control study on drug abuse. International Journal of Epidemiology, 25(6), 1267-1270. Retrieved from https://doi.org/10.1093/ije/25.6.1267
https://doi.org/10.1093/ije/25.6.1267...
), and different techniques from design thinking (Brown, 2020Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Rio de Janeiro, RJ: Elsevier.; Dyer et al., 2013Dyer, S. W., Ding, B., & Sun, J. (2013, January 23). The innovator’s secret weapon. Kearney. Retrieved from https://www.es.kearney.com/operations-performance-transformation/article/-/insights/the-innovator-s-secret-weapon
https://www.es.kearney.com/operations-pe...
; Guimarães et al., 2017Guimarães, P. B. V., Carício, M. R., & Oliveira, A. C. S. (2017). Inovação no setor público com estratégias de design thinking (Série Inovação no Setor Público, Vol. 1). Salvador, BA: Ed. Motres. Retrieved from https://sigaa.ufrn.br/sigaa/verProducao?idProducao=7579315&key=b69666 36f4dbb70a4033b0790a09fd24
https://sigaa.ufrn.br/sigaa/verProducao?...
), it was possible to translate GSC to local reality (Herrera, 2019Herrera, V. (2019, June). Reconciling global aspirations and local realities: challenges facing the sustainable development goals for water and sanitation. World Development, 118, 106-117. Retrieved from https://doi.org/10.1016/j.worlddev.2019.02.009
https://doi.org/10.1016/j.worlddev.2019....
; Leavesley et al., 2022Leavesley, A., Trundle, A., & Oke, C. (2022). Cities and the SDGs: realities and possibilities of local engagement in global frameworks. Ambio, 51(6), 1416-1432. Retrieved from https://doi.org/10.1007/s13280-022-01714-2
https://doi.org/10.1007/s13280-022-01714...
), while reducing contestation and complexity (Wanzenböck et al., 2020Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
https://doi.org/10.1093/scipol/scaa027...
), and producing a problem grouped map with prioritization, and problem statements (challenges) with open phrasing and solution criteria to promote the users’ desired outcomes.

Our main contribution is a design process that can be replicated in different local contexts, contributing both to translate SDGs to local realities and guide other public processes to create local innovation challenges. Although we only described the health GSC design process, we were able to execute it on other four GSC8 8 Retrieved from https://www.facepe.br/wp-content/uploads/2022/08/Edital_FACEPE_28-2022-Apoio-a-Projetos-de-Pesquisa-para-o-Enfrentamento-de-Desigualdades-Sociais.pdf .

In terms of threats to validity, our snowball sampling may face issues in terms of external validity. Despite the lack of generalization, this strategy is effective in engaging people that could not be reached otherwise. The design thinking steps algo bring a focused view on the topics addressed, reducing internal bias using information clustering and information saturation. Also, there is a limitation concerning the absence of quantification since we did not use surveys to evaluate the process. Without this evaluation, the bias of the researcher increases because we conducted the workshops and the documenting process. However, our premise of mapping some local innovation challenges that are relevant to users is fulfilled. We must highlight that the call for proposals to Edital FACEPE Nº 28/20229 9 Retrieved from https://www.facepe.br/a-analise-da-etapa-i-enquadramento-do-edital-282022-apoio-a-pesquisa-para-o-enfrentamento-de-desigualdades-sociais-foi-concluida/ was a very requested instrument with at least one application for each challenge (60 in total), which reinforces the relevance of the resultant analysis. Another limitation lies in the literature: the guides, reports and papers are all very recent and incipient, with a considerable endogeneity.

For future works, we intend to listen to the applicants of the call for proposals to Edital FACEPE Nº 28/2022. In addition, we plan to collect insights from the participants of the workshops, producing quantitative evaluation on the reaction of both groups.

ACKNOWLEDGMENTS

This research was carried out with the support of the Usina Pernambucana de Inovação (USINA).

REFERENCES

  • Albala, S., Holloway, C., Austin, V., & Kattel, R. (2021). New economics of assistive technology: a call for a missions approach (IIPP WP 2021/04). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/wp2021-04
    » https://www.ucl.ac.uk/bartlett/public-purpose/wp2021-04
  • Biernacki, P., & Waldorf, D. (1981). Snowball sampling: problems and techniques of chain referral sampling. Sociological Methods & Research, 10(2), 141-163. Retrieved from https://doi.org/10.1177/004912418101000205
    » https://doi.org/10.1177/004912418101000205
  • Brown, T. (2020). Design thinking: uma metodologia poderosa para decretar o fim das velhas ideias Rio de Janeiro, RJ: Elsevier.
  • Dragan, I. M., & Isaic-Maniu, A. (2013). Snowball sampling completion. Journal of Studies in Social Sciences, 5(2), 160-177. Retrieved from https://core.ac.uk/download/pdf/229607586.pdf
    » https://core.ac.uk/download/pdf/229607586.pdf
  • Dyer, S. W., Ding, B., & Sun, J. (2013, January 23). The innovator’s secret weapon. Kearney Retrieved from https://www.es.kearney.com/operations-performance-transformation/article/-/insights/the-innovator-s-secret-weapon
    » https://www.es.kearney.com/operations-performance-transformation/article/-/insights/the-innovator-s-secret-weapon
  • Gibbons, S. (2019, July 07). Dot voting: a simple decision-making and prioritizing technique. Nielsen Norman Group Retrieved from https://www.nngroup.com/articles/dot-voting/
    » https://www.nngroup.com/articles/dot-voting/
  • Guimarães, P. B. V., Carício, M. R., & Oliveira, A. C. S. (2017). Inovação no setor público com estratégias de design thinking (Série Inovação no Setor Público, Vol. 1). Salvador, BA: Ed. Motres. Retrieved from https://sigaa.ufrn.br/sigaa/verProducao?idProducao=7579315&key=b69666 36f4dbb70a4033b0790a09fd24
    » https://sigaa.ufrn.br/sigaa/verProducao?idProducao=7579315&key=b69666 36f4dbb70a4033b0790a09fd24
  • Herrera, V. (2019, June). Reconciling global aspirations and local realities: challenges facing the sustainable development goals for water and sanitation. World Development, 118, 106-117. Retrieved from https://doi.org/10.1016/j.worlddev.2019.02.009
    » https://doi.org/10.1016/j.worlddev.2019.02.009
  • Janssen, M. J., Torrens, J., Wesseling, J. H., & Wanzenböck, I. (2021). The promises and premises of mission-oriented innovation policy - a reflection and ways forward. Science and Public Policy, 48(3), 438-444. Retrieved from https://doi.org/10.1093/scipol/scaa072
    » https://doi.org/10.1093/scipol/scaa072
  • Larrue, P. (2021). The design and implementation of mission-oriented innovation policies: a new systemic policy approach to address societal challenges (ECD Science, Technology and Industry Policy Papers, n. 100). Paris, France: OECD Publishing. Retrieved from https://doi.org/10.1787/23074957
    » https://doi.org/10.1787/23074957
  • Leadbeater, C. (2018). Movements with missions make markets (IIPP WP 2018-07). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/wp2018-07
    » https://www.ucl.ac.uk/bartlett/public-purpose/wp2018-07
  • Leavesley, A., Trundle, A., & Oke, C. (2022). Cities and the SDGs: realities and possibilities of local engagement in global frameworks. Ambio, 51(6), 1416-1432. Retrieved from https://doi.org/10.1007/s13280-022-01714-2
    » https://doi.org/10.1007/s13280-022-01714-2
  • Lopes, C. S., Rodrigues, L. C., & Sichieri, R. (1996, December). The lack of selection bias in a snowball sampled case-control study on drug abuse. International Journal of Epidemiology, 25(6), 1267-1270. Retrieved from https://doi.org/10.1093/ije/25.6.1267
    » https://doi.org/10.1093/ije/25.6.1267
  • Lucena, R. M. (2022). Missão em países em desenvolvimento: o caso brasileiro pelo recorte do plano inova empresa. Revista de Administração, Sociedade e Inovação, 8(3), 68-86. Retrieved from https://doi.org/10.20401/rasi.8.3.670
    » https://doi.org/10.20401/rasi.8.3.670
  • Mazzucato, M. (2018, October). Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change, 27(5), 803-815. Retrieved from https://doi.org/10.1093/icc/dty034
    » https://doi.org/10.1093/icc/dty034
  • Mazzucato, M., & Dibb, G. (2019). Missions: a beginner’s guide (IIPP PB 09).. London, UK: UCL Institute for Innovation and Public Purpose Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/iipp_policy_brief_09_missions_a_beginners_ guide.pdf
    » https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/iipp_policy_brief_09_missions_a_beginners_ guide.pdf
  • Mazzucato, M., Kattel, R., & Ryan-Collins, J. (2020). Challenge-driven innovation policy: towards a new policy toolkit. Journal of Industry, Competition and Trade, 20, 421-437. Retrieved from https://doi.org/10.1007/s10842-019-00329-w
    » https://doi.org/10.1007/s10842-019-00329-w
  • Mazzucato, M., & Macfarlane, L. (2019). A mission-oriented framework for the Scottish national investment bank (IIPP WP 2019-02). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/publicpurpose/wp2019-02
    » https://www.ucl.ac.uk/bartlett/publicpurpose/wp2019-0
  • Mazzucato, M., & Penna, C. (2016). The Brazilian innovation system: a mission-oriented policy proposal. Sumário executivo Brasília, DF: Centro de Gestão e Estudos Estratégicos. Retrieved from https://www.cgee.org.br/documents/10195/1774546/Sistema_Brasileiro_de_Inovacao-Mazzucato_Penna-Sumario_Executivo.pdf
    » https://www.cgee.org.br/documents/10195/1774546/Sistema_Brasileiro_de_Inovacao-Mazzucato_Penna-Sumario_Executivo.pdf
  • Miedzinski, M., Mazzucato, M., & Ekins, P. (2019). A framework for mission-oriented innovation policy roadmapping for the SDGs: the case of plastic-free oceans (IIPP WP 2019-03). London, UK: UCL Institute for Innovation and Public Purpose. Retrieved from https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
    » https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/a_framework_for_mission-oriented_policy_roadmapping_for_the_sdgs_final.pdf
  • Pernice, K. (2018, February 18). Affinity diagramming for collaboratively sorting UX findings and design ideas. Nielsen Norman Group Retrieved from https://www.nngroup.com/articles/affinity-diagram/
    » https://www.nngroup.com/articles/affinity-diagram/
  • Rizardi, B., & Metello, D. (2022). Design sistêmico: abraçando a complexidade no setor público Brasília, DF: Enap.
  • Robinson, D. K. R., & Mazzucato, M. (2019, May). The evolution of mission-oriented policies: exploring changing market creating policies in the US and European space sector. Research Policy, 48(4), 936-948. Retrieved from https://doi.org/10.1016/j.respol.2018.10.005
    » https://doi.org/10.1016/j.respol.2018.10.005
  • Roth, F., Wittmann, F., Hufnagl, M., & Lindner, R. (2022). Putting mission-oriented innovation policies to work: a case study of the German high-tech strategy 2025 (Fraunhofer ISI Discussion Papers, Innovation Systems and Policy Analysis, n. 75). Karlsruhe, Germany: Fraunhofer ISI. Retrieved from http://hdl.handle.net/10419/262141
    » http://hdl.handle.net/10419/262141
  • Steinman, D., Murphy, L., & Mehta, N. (2017). Customer success: como as empresas inovadoras descobriram que a melhor forma de aumentar a receita é garantir o sucesso dos clientes São Paulo, SP: Autêntica Business.
  • Tripp, D. (2005). Action Research: a methodological introduction. Research and Education, 31(3), 443-466. Retrieved from https://doi.org/10.1590/S1517-97022005000300009
    » https://doi.org/10.1590/S1517-97022005000300009
  • Wanzenböck, I., Wesseling, J. H., Frenken, K., Hekkert, M. P., & Weber, M. K. (2020). A framework for mission-oriented innovation policy: alternative pathways through the problem-solution space. Science and Public Policy, 47(4), 474-489. Retrieved from https://doi.org/10.1093/scipol/scaa027
    » https://doi.org/10.1093/scipol/scaa027
  • Wedell-Wedellsborg, T. (2021). Qual é o seu problema? Para resolver seus problemas mais difíceis, mude os problemas que você resolve São José dos Campos, SP: Benvirá.
  • 1
    Can be also referred as Grand Social Challenge, Wicked Issue, or Grand Challenge.
  • 2
    Can be also referred as MOIP.
  • 3
    Retrieved from https://oecd-opsi.org/
  • 4
    Retrieved from https://stip.oecd.org/MIP/the-overall-methodological-framework
  • 5
    Retrieved from https://oecd-opsi.org/pet/
  • 6
    Retrieved from https://oecd-opsi.org/wp-content/uploads/2021/10/OECD-Innovation-Facets-Brief-Mission-Oriented-Innovation-2021.pdf
  • 7
    Retrieved from https://www.facepe.br/wp-content/uploads/2021/10/Edital_FACEPE_21-2021-Pro-Startups-Operação1.pdf
  • 8
    Retrieved from https://www.facepe.br/wp-content/uploads/2022/08/Edital_FACEPE_28-2022-Apoio-a-Projetos-de-Pesquisa-para-o-Enfrentamento-de-Desigualdades-Sociais.pdf
  • 9
    Retrieved from https://www.facepe.br/a-analise-da-etapa-i-enquadramento-do-edital-282022-apoio-a-pesquisa-para-o-enfrentamento-de-desigualdades-sociais-foi-concluida/
  • 15
    [Original version]

Reviewers:

  • 18
    Pedro Carlos Resende Junior (Universidade de Brasília, Brasília / DF - Brazil; Centro Universitário IESB, Brasília / DF - Brazil) https://orcid.org/0000-0002-4220-8243
  • 19
    Two of the reviewers did not authorize the disclosure of their identities.

Edited by

Editor-in-chief: Alketa Peci (Fundação Getulio Vargas, Rio de Janeiro / RJ - Brazil) http://orcid.org/0000-0002-0488-1744

Publication Dates

  • Publication in this collection
    18 Sept 2023
  • Date of issue
    Jul-Aug 2023

History

  • Received
    05 Jan 2023
  • Accepted
    12 May 2023
Fundação Getulio Vargas Fundaçãoo Getulio Vargas, Rua Jornalista Orlando Dantas, 30, CEP: 22231-010 / Rio de Janeiro-RJ Brasil, Tel.: +55 (21) 3083-2731 - Rio de Janeiro - RJ - Brazil
E-mail: rap@fgv.br