SciELO - Scientific Electronic Library Online

 
vol.26 issue1Business processes modeling: effect of notation method in the ambiguity levelApplying the operations strategy construct to rural grain-producing enterprises: a theoretical framework author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Gestão & Produção

Print version ISSN 0104-530XOn-line version ISSN 1806-9649

Gest. Prod. vol.26 no.1 São Carlos  2019  Epub Mar 18, 2019

https://doi.org/10.1590/0104-530x1626-19 

Original Article

Quality of the public health system: a systemic comprehension in Brazilian southern region

Marcelo Antunes Marciano1  2 
http://orcid.org/0000-0002-5226-1216

Guilherme Vaccaro2 

Anníbal Scavarda3 

1 Hospital Moinhos de Vento, Rua Ramiro Barcelo, 910, CEP 90035-004, Porto Alegre, RS, Brasil, e-mail: marcelo.marciano2000@gmail.com

2 Programa de Pós-graduação em Engenharia de Produção e Sistemas – PPGEPS, Universidade do Vale do Rio dos Sinos – UNISINOS, Av. Unisinos, 950, Cristo Rei, CEP 93020-190, São Leopoldo, RS, Brasil, e-mail: guilherme.vaccaro@gmail.com

3 Pontifícia Universidade Católica do Rio de Janeiro – PUC-Rio, Rua Marquês de São Vicente, 225, Gávea, CEP 22451-900, Rio de Janeiro, RJ, Brasil, e-mail: annibal@esp.puc-rio.br


Abstract

This article approaches a prospective study on the improvement actions on the quality of public health system in the state of Rio Grande do Sul, aiming to identify gaps that prevent the appropriate functioning of this system and to offer actions to leverage its development, through Systemic Thinking and Scenario Analysis. Concerning the collection and analysis procedures, a public database was used to collect secondary data, the analysis by judgment in multidisciplinary group dynamics and by modeling. As result, the construction of a map was obtained, which highlights the factors and relations of this dynamic system, enabling to identify possible scenarios, leverage points (e.g.: preventive actions; investments in health sector; infrastructure – sanitation and security; social awareness and training the health professionals) and to suggest strategies, classified in the perspectives of the Government (analyzing the investment needs in the sector of health and sanitation; adapting legislation; encourage the health industry development in technology); the public-private Health System (to understand the systemic relation, to integrate the factors and parts; to focus in Primary Actions); Education (to integrate the school with social problems; to improve the capability); Health Organizations (to map the main incident illnesses; to analyze the effectiveness of treatments; to plan preventive actions; to look for innovation; to focus on the client) and to help the improvement on the quality of the public health system in RS.

Keywords:  Quality improvement; Public health system; Systemic thinking

Resumo

Este artigo aborda um estudo prospectivo sobre as ações de melhoria da qualidade do sistema de saúde pública do estado do Rio Grande do Sul (RS), com o objetivo de identificar lacunas que impeçam o adequado funcionamento desse sistema e propor ações para alavancar seu desempenho, por meio da abordagem do Pensamento Sistêmico e Análise de Cenários. Quanto aos procedimentos de coleta e análise, foram utilizadas a coleta de dados secundários em bancos de dados de acesso público e a análise por julgamento em dinâmica de grupo multidisciplinar e por modelagem. Como resultado do trabalho, obteve-se a construção de um mapa, o qual evidencia os atores e as relações desse sistema dinâmico, possibilitando a identificação de possíveis cenários, pontos de alavancagem (como ações preventivas; investimento em saúde; infraestrutura: saneamento e segurança; conscientização da sociedade e capacitação dos profissionais de saúde) e a sugestão de estratégias classificadas nas perspectivas do governo (analisar as necessidades de investimentos na área de saúde e saneamento básico; adequar legislação; fomentar o desenvolvimento tecnológico das indústrias de bens para saúde); do Sistema de Saúde Público-Privados (entender a relação sistêmica; integrar as partes e atores; focar em ações primárias); da Educação (integrar a escola com os problemas da sociedade; melhorar a capacitação); das organizações de saúde (mapear as principais doenças incidentes; analisar a efetividade dos tratamentos; planejar ações preventivas; buscar inovação; focar no cliente) para auxiliar na melhoria de qualidade do sistema de saúde pública do RS.

Palavras-chave:  Melhoria da qualidade; Sistema de saúde pública; Pensamento sistêmico

1 Introduction

The World Health Organization defines a health system, as a system that:

[…] includes all activities which the basic purpose is to promote, restore or maintain health… and includes patients…, health service’s workers… inside organizations and in the community, and the health politics environment in which all health related activities happen (WHO apud Kotler et al., 2010 , p. 35).

Under this perspective, Brazilian public health system, in regional unities branches (state and municipal), present significant problems that prevent the adequate functioning of its function, which be to secure the adequate process of prevention, intervention and rehabilitation of the Country’s citizens. Health Care Units, specially public, present structure and/or equipment either neglected or with the level of improper operation, insufficient hospital beds to meet the demand, number of medical professionals short of the general clinic or medical specialty to the epidemiological profile of the population, extremely bureaucratic processes or on the diverse patient’s focus, among other factors. With effects, we can highlight lines of patients waiting for treatment in hospitals and health care units, including emergency, cases of patients that come to death for the delay or insufficient or inadequate medical conduct, management delinquency and dissatisfaction in the value chain ( Borba et al., 2009 ; André, 2010 ; Nita et al., 2010 ; Lanna, 2011 ).

Brazil occupied the 72nd position in the World Health Organization (WHO) of health investment, list made based on the annual State’s expenses per inhabitant. Other global governments spend, together, an annual average of US$ 317 per person, according to the WHO’s research, with the data from 2008. Brazilian performance was 40% lower than the international average (US$ 517). The leadership on the ranking of 193 countries belongs to Norway and Monaco, whose annual expenses (US$ 6,2 thousand per inhabitant) was twenty times higher than Brazilian ones. Although Brazil has the bigger economy in South America, three countries in the continent did better: Argentina, Uruguay and Chile ( Barrocal & Nassif, 2011 )

The expenses designated to health care in 2010 represent approximately 8,5% of the gross domestic product (DATASUS, July 2014), much lower level than the presented by the same group of countries. It is much lower from the average when we consider the size of Brazilian Government. Besides that, from 2000 to 2012, the proportion of public expenses designated to health care varied very little, reveling the absence of effort to prioritize the sector ( Niemeyer, 2014 ). However, we can question if these investment levels are lost by inadequate management or other loss sources, that revert in the non-delivery of value to the population.

The State of Rio Grande do Sul (RS), southernmost state of Brazil has, according to the 2012 survey, a total of 30.457 hospital beds (Brazil has 456.291), being 5.750 public (Brazil has 167.346), 20.048 philanthropist (Brazil has 141.027) and 4.659 private (Brazil has 147.818). The number of hospital internment in 2012, was 716.730, being, in all the territory, this number in 2012 was 11.002.903 ( DATASUS, 2014 ). The state counts on 21.585 health care establishments, 316 general hospitals and 17 specialized hospitals. According to the Complementary Law no. 141, of January 13th of 2013, that regulates paragraph 3rd from the article 198 of the Federal Constitution to dispose about the minimal amounts to be applied annually by the Union, States, Federal District and Cities in actions and public health care services, the percentage of individual revenue applied in public health in the first quarter of 2014, was 11,06%. The expenses in basic health care were approximately R$ 31,4 million, while the hospital and ambulatory assistance expenses were approximately R$ 434,6 million (SIOPS – Health Care Public Budget Information System, data from 04/29/2014). In 2010 the State’s GDP was R$ 252.482.597,5. 2010’s per capita was R$ 23.609,90. Even with these investments on RS State, public health care is short of attending the demand that corresponds to it.

Situations like the ones described in the previous paragraphs do hot happen exclusively in Brazil or in the State of Rio Grande do Sul, being a reality in other countries and a theme for permanent concern, for they deal with basic human life factors ( ONU, 2014 ). The United States invest around 17% of its GDP in health care and present, nevertheless, significant gaps in terms of medical care ( Kotler et al., 2010 ; Porter & Teisberg, 2007 ). Considering this situation, the World Health Organization ( OMS, 2002 , p. 33) warns that:

The predominant health care system all over the world are failing, for they are not being able to keep up with the decline tendencies of acute health problems and rise of chronic conditions. When the health problems are chronic, the attention mode to the acute conditions do not work.

According to Mendes (2011) , the rise of chronic tendencies is the determinant point of the contemporary crisis of Brazilian and global health care systems. Moreover, to overcome this crisis, we need deep changes in the public health care system.

A public health care system is typically a complex environment, where various factors interact in multiple ways, presenting an elevated number of connections ( Sterman, 2000 ; Forrester, 1968 ). Population, divided in patients and familiars, interact with emergency care units, clinics, health care units, hospitals, laboratories. These, in turn, interact with equipment, material and hospital services providers, doctors, nurses and technicians, health insurance and regulatory organisms in municipal, state, national and international levels. Resources are passed and the standard values are established by procedures, in function of national and international public health indicators, and budgets of the federal, state, municipal and national units, being, therefore, subject for politic elements out of time and not related to long term public politics ( Borba et al., 2009 ; André, 2010 ; Nita et al., 2010 ; Lanna, 2011 ).

Under the point of view of the system dynamics, problem solving in complex systems require the comprehension of the types of relationships between the actors of the system, as well as their influence and interaction cycles, aiming to identify forms of action in points that can promote interactions in the desirable manner. This vision is corroborated by Senge (2006) , which affirms that a large slice of the human problems, including public health care, is consequence on the lack of preparation and knowledge to manage the complex systems of the world. Senge (2006) suggests that many managers do not have the fifth discipline, the systemic thinking, and this prevent them to truly comprehend the process of their own jobs, limiting the decision making capacity as much as the learning process that can happen inside the organization. On the other side, what normally occurs, even in the health care sector, are solutions that can alleviate the symptoms, mas do not treat the cause of the problem. Consequently, the systems respond to re-balance itself in the states that are not the best than before the symptom happened, favoring repeated and mistaken corrections ( Walley et al., 2006 ).

Facing the situation, society can question: How will be the public health care in the future? How much the political issues will affect public health care? How will be the instruction of health professionals from now on? How much money will be spent on health investment? How will the population treat smoking and alcoholism? And what will be the reflection of these and other vices in the society and public health system? What will be the degree of concentration of the medical supply and the attendance to regional demands? How much will bed availability be? How will the SEDI behave - Social and Economic Development Index?

These matters refer to elements of uncertainty which its results depend on context and relationship structure factors between the health care value actors. According to Andrade et al. (2006) critical uncertainties are understood as forces that interfere in the significant behavior of a dynamic system and on which there is no clear tendency about the unfolding of events in the future. A form of comprehending the impact of these uncertainties is to create future scenarios. This process differs from the process of prediction, because it focuses on learning development on possible futures, creating awareness, in the present, about the potential effects of these forces on the future. To Andrade et al. (2006) , scenario planning do not intend to discover the real future, but instead, to build possible scenarios and from that to elaborate strong strategies in the present so that the scenario comes true. The scenario visualization practice in the Systemic Thinking (ST), that, according to these authors, culminates in the elaboration of strong strategies, can help to boost and catalyze the improvement on the quality of public health care system in the State of Rio Grande do Sul. The goal of this study, therefore, is to map the systemic relationship of health care’s services and actors and to analyze possible impacts of this system, focusing on the elaboration of plans of action that can serve as an alternative to improve the public health care on the state of RS.

The article is structured in the following manner: section 2 presents concepts about Systemic Thinking and System Dynamics. Section 3 presents considerations about correlated works on Public Health Care Systemic Thinking. Section 4 defines the method. Section 5 presents the prospective study in relation to the improvement on the quality of the public health care on RS. Section 6 approaches robust strategies, learning and study analysis. Lastly, the conclusion.

2 System dynamics, limited rationality and systemic thinking

The discipline denominated Systems Dynamics is based in the paradigm of complexity, assuming the model and human system controls, having as one of the important characteristics feedback and the possibility of simulating quality-quantity systems via softwares. The system dynamics have been developed as a way of describing complex systems in different areas of knowledge, such as epidemiology and global modeling ( Forrester, 1961 ; Pidd, 1998 ). Corroborating with the concept, Capra (2006) defines as holistic vision, or according to him, more appropriately talking, ecological vision of the world (in the broader and deeper sense, which incorporates the vital relation between living organisms and the nature), which sees the whole, or, the parts interconnected, synergistic, plain, united, so the human beings interact with the nature in a cyclical manner. To this systemic concept we add the organismic biology examples, the autopoiesis and basic circularity, psychosomatic network, and even the cybernetic complexity, quantum physics Poincaré’s visual mathematics, chaos, fractal geometry and complex numbers. When it comes to systems in which the human element is an actor, we can use the comprehension about system dynamics to describe behaviors and mental models. In this context, the Systemic Thinking, according to Senge (2006) , integrates and makes possible to merge into one organism theoretic concepts and practical exercises, to the point where the whole can be outdated, if all the parts add to themselves.

According to Senge (1990) , the systemic thinking is a technique practiced to help complex issues to be comprehended. In addition to it, Senge (1990) explains the systemic thinking needs to be considered in three different positions: practical, the principles and the essence. In these terms, all three aspects must be considered simultaneously. Systemic thinking allows the consideration of the whole, instead of individual elements, consists in a discipline that aims to see the whole, obtaining a comprehension of greater amplitude on the union of all factors that complement the situation ( Senge, 1990 ). Capra (2006) postulates that the terms “system” and “systemic thinking”, in 1940, were used by the scientific community. In this context, Capra (2006) complements that the concepts of Bertalanffy (1977) about an open system, along with the general theory of systems, were the main agents that established the systemic thinking with the first order science movement.

Systemic approaches gained force in the last decades by acting about intrinsic limitation of the human being of naturally seeing, understanding, mapping, retaining information, modeling and prospecting the complex world. These limitations are associated to cognition, that is the relation of the beings with the environment ( Capra, 2006 ), being a characteristic common between the human beings. Such limitation has as a consequence a selective approach and non abrangent of the real world (Hogarth, 1987 apud Sterman, 2000 ). In these terms, the limited rationality arises explained in two similar approaches (Simon, 1979, 1982 apud Lebcir, 2006 ). The first approach explains the human mind as limited concerning the information processing when confronted with the real world’s complexity ( Lebcir, 2006 ). The second approach refers to cognitive abilities and limitations of human memory. This limitation can contribute to mistaken decisions ( Lebcir, 2006 ) or divergent ones.

The systemic thinking is proposed as a possible methodological alternative to overcome natural barriers of limited rationality. It aims to sharpen the vision and the senses with the goal of acting ecologically, in the deeper sense of the word, as proposed by Capra (2006) .

3 Applications of the Systemic Thinking approach in public health care

Although the considerable amount of investments on public health care, in various places, Brazil being one of them, there is a perception that the systems of health still deliver the estimated value to the population, but generating dissatisfaction with the quality and volume of the services delivered ( Lebcir, 2006 ). An important factor, that explains these failures in the management of this sector consists in the inadequate use of tools and methods to analyze, project and implement effective actions and politics of managing character ( Lebcir, 2006 ). Given the complexity of the health care systems, heuristic or approaches developed towards the management of other sectors show themselves ineffective or making superficial decisions. It occurs that, in systems with elevated level of complexity, an isolated action inside the context of a part of the system can take down the balance of all the system. In this case, a more appropriate approach can be the employment of the Systemic Thinking and System Dynamics ( Lebcir, 2006 ; Koelling & Schwandt, 2005 ).

According to Koelling & Schwandt (2005) and Homer & Hirsch (2006) , the method of system dynamics is an adequate form of treating the complexity involved in the public health care systems. Such method involves the development of casualty as well as the support of advanced techniques and computer modeling ( Homer & Hirsch, 2006 ). To Royston et al. (1999 apud Lebcir, 2006 ), the systemic thinking allows advantages in terms of modeling and system analysis of the health sector, and because of this it have been greatly used to help management decisions. To Lebcir (2006) , the main interventions of the system dynamics, consist in the analysis of propagation mechanisms of infectious illnesses, in the concept of primary care systems and to find the main motives that result in the waiting lists. Many models were built to analyze the variables that can influence the waiting list in its dimension and duration. This theme contemplates a rich variety of arreas in which the system dynamics can perform a meaningful role in formulating health politics ( Lebcir, 2006 ).

The ability to see the hospital as a system can contribute to the practice of effective clinical documentation ( Rothschild et al., 2005 ). Other contribution is to the medical screening (procedures to detect an illness before it causes damage), with regards to the evaluation of medical, social and economic consequences of different strategies of screening ( Lebcir, 2006 ). Mehrjerdi (2012) , for example, used the system dynamics model to present the interconnections between the weight factor and health issues.

In short, the system dynamics modeling have been applied in population health issues, mainly in North America and Europe, including works in: (a) heart illnesses and diabetes, HIV/AIDS, cervix cancer, infection by chlamydia, dengue and pneumococcus, among others; (b) studies about addictive substances, like heroin, cocaine prevalence and tobacco reduction policies; (c) patient flow in emergency situations and extended medical assistance; (d) dental care, mental health and people affected by natural disasters or terrorist attacks; and (e) interactions between the capacity of public health care and the epidemiological profile of the population ( Homer & Hirsch, 2006 ). Generally, systemic thinking allows that the managers overcome their feeling of impotency when confronted with complex problems. Furthermore, it provides necessary tools to analyze, understand and influence the functioning of systems that they are trying to improve ( Lebcir, 2006 ).

Among various applications of system dynamics modeling in health care, Samuel et al. (2010) explain the whip effect in supply chain oriented to health services, in which the reduction of stages and adjusting the ability were strategic options. This work sparked a light in the strategic options to the chain dynamics strategy of supplies oriented to health services. The system dynamics modeling to prevent chronic illnesses aims to incorporate all basic elements of a modern ecological approach, including the illnesses’ evolution, health and risk behaviors, environmental factors and resources related to health and distribution systems. The study refers to health care provided by a prominent hospital in India.

In other context, Sterman (2000) presents a systemic structure applied to the propagation of infectious diseases. The epidemic case was studied in an English boarding school in 1978. One single infected student spread the flu virus, that initially spread slowly. As more students got sick, the virus became infectious, and the number of infected students increased exponentially ( Sterman, 2000 ).

Given potentially dramatic consequences of epidemic diseases, typically flexible by diffusion and absorption rates, infectious diseases gain highlights in public health care planning ( Nita et al., 2010 ). Typically entail economic and social consequences that can be damaging if a diffusion and absorption cycle get out of control from other balance cycles or controlling its effects, or preventing its diffusion. Examples in Brazil include the recent episode of H1N1 and dengue ( Brasil, 2014 ). To the contention and adequate prevention of epidemic illnesses it is necessary to develop effective contention policies, guaranteeing a better use of available resources like, for example, the modeling of HIV/AIDS virus, dengue virus and others ( Lebcir, 2006 ).

In the same way, drug use, even not being a natural illness, is considered a public health problem with epidemic characteristics. In the USA and other countries, cocaine use and other drugs increased significantly since the 80’. With the rising use propagation, many correlated problems followed such growth, among them: criminality, violence and health problems ( Sterman, 2000 ). Sterman (2000) illustrates, by systemic structure, the main relations of cause and effect related to the propagation of the drug use in the decade of 1980 in the USA, as well as some relevant effects.

In a broader perspective, Homer & Hirsch (2006) used system dynamics to analyze population health. This framework has been used in discussions about how the American Disease Control and Prevention Agency must go forward in a public health era with expanded goals and greater challenges. Its model of health community examines feedback typical in interactions between the states broadly defined of illness prevalence, adverse life conditions and the community’s ability to act. The community health model is relatively compact and was not developed based in any specific case. However, sensibility tests of the model in many possible communities and illnesses’ characteristics led to some conclusions about how different types of external help are susceptible of affecting a community in short and long term. For example, the model suggests that the focused external assistance in the construction of the ability of the community to act can be the most effective place to start to fight against an illness and poverty, and to guarantee the success on a long term basis in a way that other direct interventions could not work.

The examples quoted in this section do not aim to exhaust the existent applications, but to indicate potential benefits from the use of system dynamics and systemic thinking in the context of public health care. Hereafter it is presented the method used to elaborate this work.

4 Methodology

This work was developed from a prospective study about the Improvement of the Quality of the Public Health Care in the state of Rio Grande do Sul. The research was made in the period of August, 2012 to April, 2014.

The preliminary data prospective and initial modeling were made by a multidisciplinary group, constituted by 18 professionals of distinct knowledge areas (administration, accounting sciences, economy, clinical engineering/hospital technology, electrical engineering, mechanical engineering, mechatronics engineering, production engineering and mathematics). This preliminary stage had the goal of capturing secondary information existent in database of public domain, its analysis and identification of relevant events and of significant associations between events and variables. As an effect, a preliminary map was built of variables’ influences, which subsidized the stage of research with professionals of the sector.

The study proceeded with the construction of scenarios, aiming to establish future potentials ( Senge et al., 1995 ) and to identify relevant elements to support robust decisions in different future settings. In this approach, also supported by Schwartz (2004) , the essence is in confronting uncertainties macro-environmental, such as economics, social, ecological, demographic, legal, technological, health politics, human resources, as part of the process of oscillations and cycles, helping the organizations to become more adaptable and able to recognize unexpected events ( Salazar, 2001 ). According to Sterman (2000) , the modeling process can be made by the researcher and/or a multi-professional group, for further validation.

To criticize, contribute, consolidate and provide semantic validation on the results made in the previous stage, 8 specialist professionals of health management sector were chosen. These professionals have degrees in administration, economy, medicine, engineering, with postgraduate in health care management and average experience of 15 years in this sector, acting in the management of hospitals, clinics, laboratories, emergency care units, public health care system, health insurance policies operator, as well as consulting in the area and in technology supply. They act or acted in health organizations, profit and non-profit, both public and private, geographically sparse in metropolitan region and northern region of Rio Grande do Sul. The validation meetings happened between March and April of 2014. Four meeting were made, each one durating an average of 2 hours. The content of the meetings were recorded with the participants’ consents, being further transcribed and analyzed. Chart 1 describes the profiles of the participant specialists.

Chart 1 Profile of the participant specialists. 

Academic Education Quantity of Professionals by Education Experience in health care sector (in years) Role in health care sector Area
Administration 1 More than 20 Consultant Health consultory business
Economy 2 More than 15 Superintendent Private non-profit organization
More than 5 Superintendent Private non-profit organization
Engineering 2 More than 20 Manager Private non-profit organization
More than 20 Director Technology supply business
Medicine 3 More than 15 Director Public Health Care Service
More than 15 Superintendent Private non-profit organization
More than 20 Medical Manager of Quality Private for-profit organization

Source: Author.

Characteristics like the ones described previously, were preponderant factors to choose the systemic thinking as a tool to study prospectively the possibility of Improvement on the Quality of Public Health Care in the State of RS. According to Andrade et al. (2006) , the systemic tools reproduce the ways to build the stages needed to the systemic method in an efficient manner. From the application of this method, a prospective study was developed which the result is detailed in the next section.

5 Results presentation

5.1 Modeling

As a starting point, we established the guiding question in the study: How to improve the quality of the public health care system in the state of Rio Grande do Sul? Having as a horizon the analysis in the period of 1980 and 2030, the main issue of the study was followed by 4 guiding questions, aiming to direct the search for events and its respective variables:

    a. What are the main ways to the improvement on the public health care system in the state of Rio Grande do Sul?

    b. What are the limits to the execution of boosting actions to improve the public health care system in the state of Rio Grande do Sul?

    c. What are the future social and economic impacts to RS if the public health care system continues to deteriorate?

    d. What are the quality evaluation metrics on the public health care system?

Aiming to identify events and its respective variables, some of them were selected, basing in the referential literature and in technical publications and newspapers, with the objective of understanding the problem. The result contemplates a list of 50 initial events with 77 associated variables. To each event-variable selected it was also identified which criteria and/or practice that each binomial event-variable relates to, like it was illustrated in Chart 2 .

Chart 2 Events and variables. 

Events Variables Criteria and/or practice related to:
Onset of vaccination against childhood paralysis Children vaccinated against childhood paralysis Prevention; immunization of the infant community;
Creation of SUS (Health Unic System) Atendances made by SUS Regulamentation; public health care system
Creation of the Family Health Program/PSF Families attended by PSF Pro-active health attendance; primary attendance
Creation and End of the Provisional Contribution on Financial Transactions/CPMF Collected amount and left by CPMF Investment-sum to public health care
Laws that regulate health insurance policies Number of patients attended by health insurance Legislation/regulamentation of private health care systems

Source: Elaborated by the study group.

The second stage was focusing on identifying patterns of behavior to the listed variables, aiming to understand bases to identify associations and potential cause relations between factors, as well as the formulation of preliminary hypothesis about reciprocal influences. The main query source used was the DATASUS (2014) .

The third stage remained in the construction and analysis of the matrix correlation of the variables researched by the group. Analysis exercises were made and it was debated how the influences happened among the variables, or if they were only coincidences. After this stage, correlations were analyzed and judged more representative and important to the theme analyzed, serving as a base to the start of the systemic structure.

In the fourth stage we started to build the systemic structure, using the influences notation (black continuous arrows, indicating direct influences and red dashed arrows to indicate influences in the opposite direction) of impact from the pointed variables, one over another, when pertinent, due to the analysis of correlation established in the previous stage. The preliminary resulting map ( Figure 1 ) was constituted of the main identified relations between the variables pointed, from the vision of 4 independent subgroups of preliminary work. This map contemplated 50 interrelated variables. The use of the indication with the letters A and I in the map, was a form of pointing the relation between variables, when these are situated in distinct places or distant in the map, to avoid more arrows crossing through others and polluting the map. This way, it means the variable that has an indication with the letter A, for example, has an influence on other(s) variable(s) that have the same indication with the letter A and the same is applied to the letters up to I.

Figure 1 Systemic Maps for the proposed problem – Version 2. Source: Author.  

Many of the links presented in the map lead to different times to generate effects, thus distinct temporalities considered in the map. As premise to build the map we admitted that the ratings or cause-variable indexes were not to be altered in a significant way in the period of analysis, that is, while boosting actions are not adopted and alterations in the critical points in the map are made, the relations remain unaltered.

The development and analysis of the map proceeded with archetype analysis. Archetypes are predefined relationship structures and already studied that allows us to comprehend, in an abstract manner, how determined behavior arise on variables of interest, warning on possible effects, still hidden, in new situations ( Forrester, 1968 ). According to Senge et al. (1995) , the archetypes are helping tools to decide through hypothesis construction about the main forces that govern the systems. Furthermore, the archetypes favor elucidation on mental models existing in the system ( Senge et al., 1995 ). To Andrade et al. (2006) , archetypes indicate systemic behavior, being able to define structures generically. From the use of these structures, we seek to identify, in the systemic map developed, relevant relations that could be represented in the form of archetypes. Three different types of archetypes in the model. The types of archetypes found were: “reinforcing links”, “balancing links” and “limits to growth”.

5.2 Identification of actors and mental models

Mental models are built by people and exercise significant influence over the way each individual perceive (and consequently, interacts with) the system’s reality. Sterman (2000) describes mental models as responsible for influencing important inputs received from the environment, even in the form of not perceiving relevant and useful information, directing the attention to information of minor importance. It is necessary to understand them to, thus, modify them, providing restructuring actions of greater depth ( Andrade et al., 2006 ). To Senge et al. (1995) mental models are visualizations, preconceptions and mental memories, that we have from the world and everything that compose it. Besides that, mental models are lenses that can minimally blur the reality and set what we perceive ( Senge et al., 1995 ). Sterman (2000) affirms that the concept of mental model have been fundamental to understand System Dynamics. Forrester (1961 apud Sterman, 2000) emphasize that mental models include what we believe concerning cause relations that map the system’s functioning. To Sterman (2000) , great part of people do not perceive the invisibility of mental models, believing, naively, that our senses can reveal the world as it is, like it is outlined in Figure 2 .

Figure 2 Learning about complex systems. Source: Sterman (2000) .  

To generate profound changes in reality, it is necessary to identify how mental models generate or influence the structures so that they can be comprehended and modified. So, changing an organizational reality passes through changing the way of how the individuals think and interact. A possible approach to such objective is based in identifying mental models that generate the reality and comprehending how the systemic relationships between these models create reinforcing or balancing links on the system ( Forrester, 1968 ). From that, the study proceeded to search identification of the mental models of the actors considered relevant to the analyzed context.

The actors identified by the group were: (i) the hospitals and health care service provider organizations; (ii) the health insurance companies; (iii) the medical sector; (iv) the pharmaceutical companies; (v) the government; (vi) federal government; (vii) municipal government; (viii) press and media; (ix) the health professionals training organizations; and (x) civil society. From that, we identified mental models of these actors in the systemic from the impact of the variables, considering the following questions: 1. How does the actor perceive the quality of the public system?; 2. What is my responsibility with the quality of the public system?; 3. What are the difficulties of the improvement in the quality of the health care public system?. In a second round, the mental models were revised to consolidate the boosting actions in subsequent stages of the method. The mental models presented in Chart 3 were used during the modeling process and analyzing the forces on the systemic structure.

Chart 3 Mental models of the actors. 

Federal government “There is a need for improvement, although the budget’s resources are not enough to fix it. There is also other priorities that are more important than health care”; “The problems already exist since its creation and for this fact it is difficult to fix immediately”; “We do not have the budget to provide the expenses by ourselves”; “The tax reform is vital in the country”; “I don’t know exactly the cost of health care”.
State Government “It’s not possible to do more for public health without the support of the federal government and the cities”; “it could be better if we had transfer of funds more frequently”; “The investments done by the federal government in health are not enough to attend all the demand of the state”; “There is a lack of professionals of the health sector”; “The cost of the state public machine prevents that more resources for health exist”; “State debt (decreased the resources for investment and maintenance)”; “The centralization of resources in the federal government makes the quality improvement difficult”.
Health care sector workers “SUS is very bad, we work a lot and earn a little”; “The salaries and emotional distress verified in the public health service makes the service unattractive for the carrier”; “As soon as I get more private clients, I’ll de-accredit from SUS”; “We could have more resources to work. Better salaries and gratifications”; “The amount of procedures paid to the professionals are impediments to do a good job”; “There are a lot of infrastructure problems”.
Health Insurance Administrators “The access to a quality health care system is only possible through private companies”; “And the worse the quality of public health is, the bigger our consumer market will be”; “Providing access to a quality health system and in counterpart the weak service of public health care”; “The federal and state governments do not invest enough to provide a good public health system”; “The norms of ANS (National Health Agency) makes it less lucrative to the Insurance Company”.
Press/Media “Public health is chaos, it does not improve because the enters government and leaves government and the do not give proper attention to this issue”; “State government does not apply the fixed budget in public health”; “The quality of public health is concerning because people die in waiting line for transplants, medications, treatments”; “Leave society informed”; “Lack of effort from governors in putting ideas in practice”; “The focus of the government is not public health”; “Diversion of funds that should be destined to health and the lack of population’s awareness concerning their rights and responsibilities”.
Health Training Organization “We need to guarantee the quality training to improve the quality of the public system”; “we train capable professionals, but we are not responsible for the bad infrastructure offered by government”; “Training professionals capable of attending the population”; “Government does not present long term projects, that would involve the training organizations too, but focus on visible actions that can be seen in the 4 years of government”.
Civil Society “The situation is very concerning, since there is a lack of financial resources to improve the infrastructure, like hospitals, health care units, equipment to do exams”; “The doctors take too long to attend, and when they attend, do it in a few minutes”; “There is lack of medication, for free distribution”; “Government does not invest properly with the taxes we pay”; “My responsibility is to charge public power to improve public health”; “I have to elect candidates that worry about the quality on the public health system”; “There is misappropriation of budget that should be destined to public health”; “There is a lack of specialty doctors”; “It is difficult to access routine exams and more sophisticated ones in a reasonable amount of time”.
Pharmaceutical Companies “There is a lack of medication, which causes the delay/suspension on treatment by lack of investment in public health”; “To do more deals with the government to distribute medication”; “To look for innovation in the development of pharmaceutical industry in order to reduce the price of medication and equipment”; “Too much bureaucracy of the health system”; “Why does the government do not exempt the pharmaceutical industry of taxes like it happens with the automotive industry?”
Hospital Administrators “Government does not invest appropriately in infrastructure and training professionals”; “We do what is possible with the available resources, do not ask us more than that”; “Giving conditions for the professionals to save the biggest number of lives, using available resources”; “Federal and state government do not pass along all the budget destined to health care by law”; “I need to work the least of necessary resources”; “The values on the SUS table are very below the cost”.

Source: Elaborated by the study group.

These models can set restrictions or barriers to the implantation of robust strategies and improvements for the health system. According to Senge & Sterman (2012) , a vision is built from mental models, which are formed by beliefs and preconceptions of the individual about reality. Said that, a new form of thinking must help to map, defy and improve mental models, focusing on more effective actions in reality, from a new way of viewing the system, which is the public health system.

5.3 Scenario analysis

The study proceeded with the construction and analysis of scenarios, which has the following stages, according to Schwartz (2004) and Andrade et al. (2006): (i) Learning about the interest focus; (ii) Defining the driving force; (iii) Defining critic uncertainties; (iv) Prioritizing critic uncertainties; (v) Forming scenarios; (vi) Characterizing scenarios; (vii) Experimenting and learning; (viii) Identifying flags; (ix) Taking decisions or defining the plan of action; (x) Monitoring the scenarios through flags.

To that, it were identified driving forces, understood as forces that act in reality and are fundamental for the decision making, being generally “external” forces. Also, the driving forces can be predetermined tendencies about which can obtain a clear vision of how they will unravel in the future. Alternatively, driving forces can’t have clear tendencies of evolution, being in this case, characterized as critic uncertainties. In the critic uncertainties lies the possibility of various futures about which the system and its actors can behave in diverse forms. Thus, the analysis of driving forces, integrates in the sense of understanding what are their impacts on the systemic structure built ( Schwartz, 2004 ).

The driving forces identified were: epidemics, changing of government, premature labors, population, pharmaceutical industry and equipment technological development, life expectancy, preventive policies, new techniques of medical procedures, mental health of the population. From those, 8 were identified as critic uncertainties, them being: relation between public and private in health administration, health professionals training, investment in health, smoking/alcoholism, concentration of medical offer, hospital beds availability, quality in attendance and IDESE influence.

According to the data collection of forces and uncertainties, we passed to the identification of scenarios to be analyzed. To do so, two critic uncertainties were chosen, investment in health and IDESE influence, and extreme levels were established for these uncertainties as represented in Figure 3 , giving rise to four interest scenarios. These critic uncertainties were elected by considering that social-economic variables are determinant and representative in public health system.

Figure 3 Scenarios analyzed, based in two key-variables selected and its respective levels. Source: Elaborated by the study group.  

Scenario analysis was made, again, by four independent groups in the team of the project, which had as a task to construction and description of the four scenarios defined. To build the scenarios each group tried to project themselves in the reality established by the scenario, as recommended by Schwartz (2004) , responding to questions like: (a) What name best defines the scenario? (b) What would be necessary for the scenario to be set? What are the flags to these happenings? (c) Once implemented, what are its characteristics? (d) What happens with the driving forces in this scenario? (e) What happens with the society, technology, environment, economy, politics? (f) What are the consequences to the society? (g) What are the creative strategies before the scenario implements itself? How to be prepared for the scenario, in the interested perspectives? (h) What are the conditional strategies, only taken once the scenario is implemented, in the parts interested perspectives?

The results obtained by the group were, then, compiled in qualitative form, aiming an integrated vision of the data collected. Chart 4 presents the profiles imagined by the subgroups, if the scenarios were implemented, as well as which would be the economical and social impacts in each of them. According to the methodology adopted, each scenario was baptized to present, emphatically, its contents.

Chart 4 Scenarios. 

SCENARIO 1
“High investment × low IDESE”
• GPD: Elevated
• Financial resources for public health
• Inadequate investment distribution
• Incompetence/corruption
• Social inequality
• High levels of illiteracy, low development of HR
• Low investment in prevention, education, sanitation
• Low social and economic development
SCENARIO 2
“High investment × high IDESE”
• GPD: Elevated
• Financial resources for public health
• Adequate investment distribution (probable)
• Incompetence/corruption (unfortunately in Brazil)
• Social inequality must improve
• Decreasing levels of illiteracy, conditions for the development of HR
• Investment in prevention, education, sanitation
SCENARIO 3
“Low investment × low IDESE”
• GPD: Low
• Drastic reduction of financial resources for public health
• Inadequate investment distribution
• Incompetence/corruption
• High levels of social inequality
• High levels of illiteracy, low development of HR
• Low investment in prevention, education, sanitation
• Low social and economic development
• Society goes into crisis
SCENARIO 4
“Low investment × high IDESE”
• GPD: Low
• Reduction of financial resources for public health
• Incompetence/corruption
• High levels of social inequality
• Low levels of illiteracy, development of HR
• Low investment in prevention, education, sanitation
• Low social and economic development

Source: Elaborated by the study group.

5.4 Actions identification and suggestion

After analyzing the behavior of the metrics selected to represent the performance of the model in different scenarios identified with the help of simulation, the group identified boosting points to the system studied. Boosting points, according to Andrade et al. (2006) , are areas of the systemic map that contain variables, relations of cause and effect, archetypes and mental models and allow to potentialize the application of actions and changes in the relationship structures, and can bring meaningful and lasting results. The boosting points identified to the studied system were: (i) preventive actions; (ii) investment in health; (iii) infrastructure (sanitation and security); (iv) social awareness (concerning the importance of the preventive medicine and self-care); and (v) training of health professionals. These points were selected, presenting agreement with the studies of Lebcir (2006) and Homer & Hirsch (2006) . The boosting points were validated by the specialists of health administration, which represented 50% of identified actors.

The analysis proceeded listing the probable limits to the execution of the boosting actions to improve the quality of public health care, them being: (i) decrease of GDP, job reduction and investments; (ii) public budget destined to health and health policies; (iii) low investment in infrastructure of hospitals; (iv) low investment in health budget and high expenses; (v) low investment in illnesses prevention and immunization; (vi) inadequate policy of investment distribution for the health sector; (vii) incompetence and corruption; (viii) low development of human resources, with doctors, nurses and health professionals; and (ix) low or inadequate investment in pharmaceutical industry development. We also identified the probable three bigger limits to the execution of boosting actions: (i) messy or not integrated mental models; (ii) bad administration; and (iii) inadequate and or/lack of economic resources. These limits are restrictions to the boosting actions. They can make it difficult to potentialize the application of changing actions in the relation structure of the systemic map, and with that prevent meaningful and lasting results on improving the health system, as corroborate Borba et al. (2009) , André (2010) , Nita et al. (2010) and Lanna (2011) .

Identified and defined the principal boosting points, we built a proposal of boosting actions, arriving at a group of robust strategies to the project. The strategies were classifies as Short Term (a year) and Medium/Long Term (from 3 to 5 years), according to Chart 5 .

Chart 5 Boosting actions identified in the study. 

Suggested actions focusing in Short Term Suggested actions focusing in Medium/Long Term
• Mapping the main incidental illnesses
• Analyzing effectiveness/closing of diagnosis/medical treatments
• Contention plan definition/chronicle illnesses control
• Integration between public and private organizations
• Stratification of the actions in health area by social class, region, age and epidemiological profile
• Analysis of the necessities of investment in the areas of health/sanitation
• Incentive to the training of new professionals with subside to the courses
• Integration of the school with social problems, mainly in courses of the education, health and engineering fields
• political reform aiming to avoid radical alterations in investment plans in each govern
• Investment actions to amplify the sanitation system

Source: Elaborated by the study group.

To consolidate boosting actions in different perspectives, the strategic analysis map was elaborated and it is presented in Figure 4 . This map was obtained through consideration of the main routes to improve the quality of the public health system. It is composed by Government, Health System (public-private relations), Education, Health Organizations and Society perspectives. The actions were grouped by adherence in each previous perspective. They emerged from the stages of the prospective study. The proposition of the map is that these action can contribute to improve the quality of health for society.

Figure 4 Map of strategic analysis. Source: Author. 

5.5 Analysis and semantic validation

The final stage of this study was doing rounds of evaluation and semantic validation of the artifacts generated by 8 specialists, as described in the section Methodology. In general terms, the structure presented was validated by specialists, considering its capacity of description and representation of the health system in focus.

From the point of view of scenario evaluation and proposed actions, the specialists declared conviction that health becomes less onerous when based in the community family agent, making the hospital service a “backup” when it is necessary to do corrective services and hospitalization. However, we observe that we must focus on health prevention, agreeing with the emergent results from the modeling made. They believe that networking, acting effectively in different stages, since doctor/health of the family, passing through basic attention, UPAs (Emergency Care Units), until arriving to the tertiary services and more complex attendances is a system that can work. But there is a necessity of creating patient managing units, to manage and reference the patients to the correct level of attendance that they need in health service.

6 Conclusion

This article touch the performance problems of the public health systems existing in Brazil, considering an analysis unit the public health system in the state of Rio Grande do Sul. Systems like health are dynamic and complex, and, as a consequence, tend to be resistant to solutions obtained by heuristic methods that can’t contemplate appropriately the effects of actors, variables and interrelationships that exist in them. In this context, the approach used in this article presented itself as an alternative, gaining space in public administration in all over the world ( Jackson, 2013 ). However, there is still many opportunities to a greater use of the principles of systemic thinking to improve public health system.

From the application point of view, we understand that this study can base a better comprehension of the reality of the analyzed unity and that the proposed actions in the strategic map can contribute to the improvement of the system quality in the state of Rio Grande do Sul. The study also allowed, to name some of the possible evaluation metrics in public health system of Rio Grande do Sul, aiming to have more tangible measures of quality of the public system. Still, the limits were considered, the economical and social future impacts to the State, if the prospected scenarios implement themselves, serving as a reference and warning so that we take strategic decisions and administrators of this system can prepare to fight them.

Other restriction or limit can be on the political form of understanding health systems (partially derived from actors’ mental models), which decisions can not be adhered for the reason of being public health, which is maintaining the population healthy, preventive or reactively, mitigating hospitalizations, corrective procedures or acting only in advanced stages of the illness. Finally, the methodology used allows to improve the system and interrelations, obtaining knowledge, knowing mental models of the main actors of the system, as well as the restrictions that these actors can generate in the system and possibly breaking the preconceptions and beliefs of the actors aiming to search the more adequate system to the population’s reality over the years.

Future research alternatives with the same varied method employment to evaluate the correlation of the variables that based the construction of the systemic map, are suggested, as indicated by health specialists. We also suggest to refine the model that touch the cause links and mental models of main actors, so that the boosting point become clearer and the acting form on them, so that the system studied improves. The future impacts in life expectancy of the population can also be opportunities to new works. Other opportunities of future researches are related to the investigation of administration models of the health system, that can give robustness and consolidate actions in an integrated and structured way, considering the tactical and operational perspectives. And so avoiding that ruptures or deviation on the orientation conduction happens, avoiding, thus, undermine the ability of generating value to the health system. Also, innovative models and criteria of administration and service operation of great provider actors from health services can be explored to the light of systemic modeling. Supplementary health plan operators, hospitals and doctors are action conductors of the health system and serve as access to health attendance, and, so, delivering this value to the population.

How to cite: Marciano, M. A., Vaccaro, G., & Scavarda, A. (2019). Quality of the public health system: a systemic comprehension in Brazilian southern region. Gestão & Produção, 26(1), e1626. https://doi.org/10.1590/0104-530X1626-19

Financial support: None.

REFERENCES

Andrade, A. L., Seleme, A., Rodrigues, L. H., & Souto, R. (2006). Pensamento Sistêmico: caderno de Campo: o desafio da mudança sustentada nas organizações e na sociedade. Porto Alegre: Bookman. 488 p. [ Links ]

André, A. M. (2010). Gestão estratégica de clínicas e hospitais. São Paulo: Atheneu. [ Links ]

Barrocal, A., & Nassif, M. I. (2011). 7º PIB, Brasil é 72º no ranking da OMS de gasto per capita em saúde. Carta Maior. Recuperado em 29 de julho de 2014, de http://www.cartamaior.com.br/?/Editoria/Politica/7%B0-PIB-Brasil-e-72%B0-no-ranking-da-OMS-de-gasto-per-capita-em-saude-/4/17653 [ Links ]

Bertalanffy, L. (1977). Teoria geral dos sistemas. Petrópolis: Vozes. [ Links ]

Borba, V. R., Lisboa, T. C., & Ulhôa, W. M. M. (2009). Gestão administrativa e financeira de organizações de saúde. São Paulo: Atlas. [ Links ]

Brasil. Ministério da Saúde. Portal da Saúde. (2014). SIOPS . Brasília. Recuperado em 20 de julho de 2014, de http://portalsaude.saude.gov.br/index.php/o-ministerio/principal/siops [ Links ]

Capra, A. (2006). Teia da vida: uma nova compreensão científica dos sistemas vivos. São Paulo: Cultrix. [ Links ]

Departamento de Informática do Sistema Único de Saúde – DATASUS. (2014). Brasília. Recuperado em 20 de julho de 2014, de http://www2.datasus.gov.br/DATASUS/index.php [ Links ]

Forrester, J. (1968). Principles os systems. Cambridge: Writh-Allen Press. [ Links ]

Forrester, J. W. (1961). Industrial dynamics. Cambridge: MIT Press. [ Links ]

Homer, J. B., & Hirsch, G. B. (2006). System dynamics modeling for public health: background and opportunities. American Journal of Public Health, 96(3), 452-458. http://dx.doi.org/10.2105/AJPH.2005.062059. PMid:16449591. [ Links ]

Jackson, M. C. (2013). Pensamento sistêmico holístico. MPM Project Management, 9(50), 25-26. [ Links ]

Koelling, P., & Schwandt, J. (2005). Health systems: a dynamic system: benefits from system dynamics. Virginia: Grado Department of Industrial and Systems Engineering. [ Links ]

Kotler, P., Shalowitz, J., & Stevens, R. J. (2010). Marketing estratégico para a Área de Saúde: a contrução de um sistema de saúde voltado ao paciente. Porto Alegre: Bookman. [ Links ]

Lanna, E. C. (2011). Estratégias e práticas para um gerenciamento logístico eficiente na área hospitalar. Perspectivas Online, 5(17), 1-15. [ Links ]

Lebcir, M. R. (2006). Health care management: the contribution of systems thinking (Business School Working Papers). Hertfordshire: University of Hertfordshire. [ Links ]

Mehrjerdi, Y. Z. (2012). A system dynamics approach to healthcare cost control. International Journal of Industrial Engineering & Production Research, 23, 175-185. [ Links ]

Mendes, E. V. (2011). As redes de atenção a saúde . Brasília: Organização Pan-Americana da Saúde. [ Links ]

Niemeyer, L. (2014). O governo brasileiro gasta pouco em saúde? O Agente Principal. Recuperado em 20 de julho de 2014, de http://oagenteprincipal.wordpress.com/2014/07/11/o-governo-brasileiro-gasta-pouco-em-saude/ [ Links ]

Nita, M. E., Campino, A. C., Secoli, S., Sarti, F. M., Nobre, M., Costa, A. M., Ono-Nita, S. K., & Carrilho, F. J. (2010). Avaliação de tecnologias em saúde: evidência clínica, análise econômica e análise de decisão. Porto Alegre: Artmed. [ Links ]

Organização das Nações Unidas – ONU. (2014). Recuperado em 20 de julho de 2014, de http://www.onu.org.br Links ]

Organização Mundial da Saúde – OMS. (2002). Cuidados inovadores para condições crônicas: componentes estruturais de ação: relatório mundial. Brasília: OMS. Recuperado em 20 de julho de 2014, de http://www.who.int/chp/knowledge/publications/icccportuguese.pdf [ Links ]

Pidd, M. (1998). Modelagem empresarial: ferramentas para tomada de decisão . Porto Alegre: Artes Médicas. [ Links ]

Porter, M. E., & Teisberg, E. O. (2007). Repensando a saúde . Porto Alegre: Bookman. [ Links ]

Rothschild, A. S., Dietrich, L., Ball, M. J., Wurtz, H., Farish-Hunt, H., & Cortes-Comerer, N. (2005). Leveraging systems thinking to design patient-centered clinical documentation systems. International Journal of Medical Informatics, 74(5), 395-398. http://dx.doi.org/10.1016/j.ijmedinf.2005.03.011. PMid:15893262. [ Links ]

Salazar, J. N. A. (2001). Gestão estratégica de negócios . São Paulo: Pioneira. [ Links ]

Samuel, C., Gonapa, K., Chaudhary, P. K., & Mishra, A. (2010). Supply chain dynamics in healthcare services. International Journal of Health Care Quality Assurance , 23(7), 631-642. http://dx.doi.org/10.1108/09526861011071562. PMid:21125959. [ Links ]

Schwartz, P. (2004). A arte da visão de longo prazo. São Paulo: Best Seller. [ Links ]

Senge, P. (1990). A quinta disciplina: teoria e prática da organização de aprendizagem. São Paulo: Best Seller. [ Links ]

Senge, P. (2006). A quinta disciplina (21. ed.). Rio de Janeiro: BestSeller. [ Links ]

Senge, P., & Sterman, J. D. (2012). Systems thinking and organizational learning: acting locally and thinking globally in the organizations of the future. In P. Senge (Ed.), A quinta disciplina: arte e prática da organização que aprende . Rio de Janeiro: Best Seller. [ Links ]

Senge, P., Kleiner, A., Roberts, C., Ross, R., & Smith, B. J. (1995). A quinta disciplina: caderno de campo. Rio de Janeiro, Qualitymark. [ Links ]

Sterman, J. D. (2000). Business dynamics: systems thinking and modeling for a complex world. New York: McGraw Hill. [ Links ]

Walley, P., Silvester, K., & Mountford, S. (2006). Health-care process improvement decisions: a systems perspective. International Journal of Health Care Quality Assurance, 19(1), 93-104. http://dx.doi.org/10.1108/09526860610642618. PMid:16548403. [ Links ]

Received: July 31, 2017; Accepted: April 15, 2018

Creative Commons License  Este é um artigo publicado em acesso aberto (Open Access) sob a licença Creative Commons Attribution, que permite uso, distribuição e reprodução em qualquer meio, sem restrições desde que o trabalho original seja corretamente citado.