Impact assessment of emergency care units on hospitalizations for respiratory system diseases in Brazil

de Saúde, Políticas Públicas


Introduction
The National Policy for Emergency and Urgent Care (PNAU), established by the Ministry of Health in the early 2000s, has created the Mobile Emergency Care Service (SAMU) and regulated, nationwide, Non-Hospital Emergency Care Units, which today are called 24-hour Emergency Care Units (UPAs). Implemented in 2008 to promote the decentralization of less complex emergency services, the UPAs aim to prevent referring such cases to hospital units 1 . Their role is to expedite emergency services that can be handled within 24 hours, reducing the number of occupied beds in hospitals, due to the admission of cases that could be completely solved in those units.
The implementation of UPAs is an expanding public health policy in Brazil. According to O'Dwyer et al. 2 , their number grew more than 400% between 2011 and 2016. August 2020 data from the Ministry of Health show 750 Emergency Care Units in the country, and 332 under construction or inactive.
However, evaluation researches focusing on UPAs are scare. Rocha and Fernandes 3 were the first authors to develop an econometric assessment of UPAs' impact on health indicators. They identified the effect of UPAs on mortality rates by city of residence, place of occurrence, and death cause. From the estimation of models with fixed effects of city and time, results indicated that UPAs had a negative, but not significant, effect on the general mortality rate. Nevertheless, there were significant effects on the mortality rate in specific contexts. Another study, conducted by Santos 4 , measured the impact of UPAs on the mortality rate from Acute Myocardial Infarction in Brazilian capitals and metropolitan regions. The study used the differences-in-differences method with fixed effects of observational and time units, from municipal panel data for the period 2001-2012. There was evidence of a positive and statistically significant impact, indicating a reduction in the mortality rate from Acute Myocardial Infarction due to UPAs.
Hence, those studies have measured the impact of UPAs on mortality rates. However, mortality reduction does not suggest the decrease of hospital emergencies, nor the number of hospitalizations. The only study found in the empirical literature that assesses the effect of UPA's on hospital admissions was carried out by Medeiros et al. 5 . The authors estimateed the effects of UPAs on hospitalizations for conditions sensitive to primary care in the cities of the State of Rio de Janeiro. Using the difference-in-differences methodology as an identification strategy, the authors found that the UPAs had a negative and significant effect on hospitalizations.
In this context, the paper intends to contribute to the field's state of the art by tackling the following question: Which is the impact of UPAs on the rate of hospitalizations for respiratory system diseases in Brazil?
In order to answer this question, it is necessary to isolate the cases that these units can fully solve, without later referral to hospitals. Cases where patients go to hospitals after their stabilization at UPAs should not be considered. For this reason, among the illnesses treated by those units, this evaluation has adopted the respiratory diseases as its research focus.
To achieve the proposed goal, we explored the fact that there are no UPAs in all Brazilian cities, and, based on the machine learning methodology, built the counterfactual to identify causality. In addition, we considered all 310 cities over 100 thousand inhabitants, in order to expand the study's contribution.
The managerial implications of the paper regard the evaluation as part of an information system that supports the process of formulation and adjustment of public policies. Thus, the use of our results by public managers working in the health system can support the decision process related to UPA's expansion policies. Finally, as expected for the theoretical contributions, practical implications may go beyond the Brazilian reality, leading to policies in other countries, especially where population growth coexists with a non-proportional increase, or even the decrease in the number of hospitals beds 6 . method It is an ecological study of multiple groups that includes 310 Brazilian municipalities. The data collected and analyzed in this research are described in a later section. To assess the impact of an intervention, action, or public policy, there must be an impact relationship between the assessment object and the results observed after this action [7][8][9] . It should be determined that external factors did not cause those effects. Therefore, the main objective of impact inference is to reach conclusions on the changes that occurred in the variable of interest and conclude that the policy implemented caused such changes.
The empirical strategy of this research sought to estimate the average treatment effect of the Emergency Care Units, considering their sizes, on the hospitalization rates for respiratory diseases. To analyze UPAs' size, we considered the workload of doctors in these units as a proxy. We assessed hospitalizations for diseases of Chapter X -Diseases of the respiratory system (J00-J99), of the International Classification of Diseases 10.
To examine the impact aspect, following the logical model by Wager and Athey 10 , we assumed a binary variable (Di), here simplified only by the presence or absence of UPAs in municipality i. This variable can take two values: Di=1 (presence of UPAs) or Di=0 (absence of UPAs). Thus, Y i (1) is the potential result (here represented by the Rate of Hospitalizations for Respiratory System Diseases -RHDRS) when municipality i underwent the intervention (Di=1). Y i (0) is the result of interest (RHDRS) when the same city i did not implement the policy, that is, Di=0. Thus, the effect of UPA's presence on the result Y for municipality i can be expressed by: To measure UPAs' influence on the hospitalization rate of municipality i, it would be necessary to compare the results of municipality i with functioning UPAs in a given period, and the results of the same municipality i without UPAs, for the same period. Thus, we faced a fundamental problem of causal inference: it is never possible to observe the same individual in both situations, that is, with treatment and without treatment 11 . Therefore, one of the two situations must be estimated in order to compare them, since, in a given period, the municipality used or did not use a public policy. We call the event that needs estimation counterfactual 12 .
The average treatment effect of the intervention is the estimation of the effect, considering all individuals 11 . We cannot estimate this effect without restrictive hypotheses, such as the homogeneity of the effects. The main alternative for building the counterfactual is to try identifying the mechanism for defining treated municipalities from other observable characteristics 13 . Vector Xi represents these attributes, and it is built from independent variables for each city. Thus, we can estimate the conditional average treatment effect on the variables that make up vector Xi, given by: However, estimation of equation 2 also brings challenges. There is a need for the conditional independence hypothesis to be valid; that is, given vector X i , interventions are random among municipalities. In addition, there is a large amount of information and potential relationships among the variables, which can be used to build vector X i . However, 'machine learning' methods have proven to be an efficient way to get around these traditional problems of causal inference [14][15][16] .
Once we understand the logical model by Wager and Athey 10 , this same argument can be extended to intensity variables (treatment levels), and not just binary ones. This is the case of this research, where we did not just analyze the presence or absence of UPAs in the cities, but their size and service capacity. To do that, we estimated the average dose-response function (ADRF) of UPAs on hospitalizations, according to equation 3: Where t is the treatment dose and Y i is the result 17 .
In order to estimate ADRF, it is important to consider that the implementation of attention points is not random. Economic, demographic, and epidemiological issues can influence the choice of municipalities for UPAs implementation. Some of these issues can also affect the hospitalization rate for respiratory diseases. Thus, they need to be controlled, to reduce the analysis bias. Bidirectional causation is another potential source of bias, and also needs to be controlled, in order to ensure that we analyze the impact of these issues on UPAs' implementation and not the opposite -UPAs' impact on these issues. Hence, we used data prior to the year chosen for analysis, which took place in 2017, in order to establish the rate of medical hours at UPAS. We did not use this logic of precedence for the control between outcome and treatment, as we assumed that UPAs' impact on hospitalizations is short.

machine learning for the estimation of counterfactuals
Traditional methods of causality inference seek to estimate counterfactuals with non-treated units of similar characteristics. A well-known method that attempts to reduce the bias of confounding variables is the Propensity Score Mathing, in which the comparison group is constructed artificially by matching observations of treated units and control units with similar char-Chart 1. Data Source: presents all data used in the research, along with their sources, for each.  18 . To assess the effect of participation in the "Pacto pela Saúde" program, for example, Kroth and Guimarães 19 estimated dose-response models with pairing of treated municipalities and control by generalized propensity score. In the case of our study, the important characteristics for estimating the counterfactual are those related to the likelihood of cities having implemented or not the UPA.

Data by municipalities source
However, PSM methods have several limitations, such as misspecifications of the functional form of the model, possibilities of non-linear relationships, use of categorical variables with more than two levels, and difficulties in dealing with missing data 20 . For these reasons, machine learning strategies are increasingly being considered in causal inference applications and have shown more accurate results 21 .
In the "machine learning" methods based on trees, although they also seek to find observations with similar characteristics, a decision tree built from the X i vector defines the proximity 10 . This vector represents the observable attributes, which may or may not determine the presence of UPAs in the city.
For each chosen decision, those groups that fall into the same "leaf node" of the tree establish counterfactuals; that is, the last level of the decision tree, when there are no more separations to make 12 .

Bayesian Additive Regression trees (BARt)
This study applies a "machine learning" strategy using the Bayesian Additive Regression Tree (BART) as algorithm, as proposed by Chipman et al. 22 . This method allows an efficient non-parametric Bayesian measurement of the heterogeneous effects of treatment, based on the relationship of observable variables X i . Unlike other tree-based learning methods, BART proposes the sum of decision trees ("sum of trees"), using an estimation based on a Bayesian probability model. The use of Bayesian modeling approach has the advantage of not assuming distributions of parameters, simplifying the parameterization of the model. In the case of BART, the Bayesian distributions of the parameters are calculated using an interactive method that approximates them to the non-parametric model 23 . Also, BART has been shown to be efficient in dealing with high-dimensional data and reducing the risk of overfitting 24 .
As demonstrated by Kapelner and Bleich 25 , models composed by sums of regression trees have a greater capacity compared to single tree models to capture interactions and nonlinearities, generating counterfactual groups more similar to the treated group. Thus, the BART method has been used for a variety of healthcare applications such as covid incidence 24 , community mental health outcomes 26 and predictability of pressure ulcers 27 . The application of the model was performed using the statistical software R using a package with the same name as the method.

Data
The variable of interest was the Hospitalization Rate for Respiratory System Diseases (RH-DRS). RHDRS is achieved by the number of hospitalizations for this type of disease in each Brazilian municipality (patient's place of residence), per 1,000 inhabitants. We measured the intensity of UPAs' treatment by the rate of working hours/doctor at UPAs, per municipality, per 1,000 inhabitants. Thus, it is possible to consider not only the presence of these units in the cities, but also their size and the number of services provided.
The X i vector, previously mentioned, incorporates variables that represent attributes of the social and health conditions of the municipalities. To compare cities with similar conditions, we used the following variables (Chart 1): proportion of the elderly population; proportion of the population under 5 years old; proportion of figure 1. Directed acyclic graph of the relationship between UPAs and hospitalization rate. the population that has health plans; GDP per capita; care rates in hospital emergency rooms and in Basic Health Units (UBSs), as well as data from the year before the analysis (2016), in order to include the initial health condition of the cities.
To minimize confusion biases and bidirectional causation mentioned above, we organized the analysis model with such variables, as shown in Figure 1.
We made the assessment for the year 2017, due to the availability and consistency of data for that year. Initially, we tried to get data on the productivity of UPAs and UBSs from the SUS Outpatient Information System (SIA/DATA-SUS), but we found many discrepancies between this databank information and administrative records of municipal secretariats, in addition to filling faults. Therefore, we used data from the National Register of Health Establishments (CNES/ DATASUS) as an alternative. The year 2017 was the most recent year with more recorded months. Data on the proportion of people over 60 years old and under 5 years old are from 2010, since only IBGE's censuses get this information by municipality.
We only included in the study the 310 cities with population over 100 thousand inhabitants (according to 2017 IBGE estimates). This is because UPAs, in general, are present in larger municipalities, and we sought to build a more homogeneous sample. In very small cities, the rates per 1,000 inhabitants are very volatile, which could lead to a biased result. In Brazil, approximately 1.2 million hospitalizations for diseases of the respiratory system were observed in 2017. For the 310 municipalities considered, this number was 511,595.
There are UPAs of different sizes, with distinct service capacities. To consider them in the study, we used data regarding the medical workload at UPAs, differentiating those with little availability of professionals/hour from those with more availability. To do this, we considered five levels of treatment, as detailed in Chart 2.
We made the division only up to the fifth level, given that about 75% of the cities that established UPAs showed numbers below 5 working hours/doctor per 1,000 inhabitants. ADRF is the estimated average of hospitalization rates for diseases of the respiratory system, at each level of treatment intensity proposed in the study. Likewise, we achieved the differences in hospitalization rates between levels 1 to 5 and level 0, which represent UPAs' impact. To test the statistical significance of the estimates, we built 95% confidence intervals for ADRF differences. The impact analysis of this study is based on the differences between dose-responses at each level and the dose-response at 0 level, as suggested by Galagate 17 . The 0 level represents UPAs' absence.
We got the confidence interval from the resampling distribution using bias-corrected and accelerated bootstrap 28 . For 1,000 bootstrap resamples of the average difference, we considered the 25 th and the 975 th values of the differences as limits of the 95% confidence interval. This captures the central 95% of the distribution.

Results
We carried out the analysis for 310 Brazilian municipalities with over 100 thousand inhabitants. Table 1 shows the results of the BART estimations, considering the cities analyzed and the five levels of treatment.
The column ADRF, which are the effects by comparing potential results, shows the point estimate of hospitalizations for each level of working hours/doctor at UPAs. In the absence of this type of healthcare unit, that is, at the 0 level, the point estimate indicates, on average, 4.585 hospitalizations for respiratory diseases per 1,000 inhabitants in Brazil. At level 1, adding one working hour/doctor at UPA, the number of hospitalizations drops, on average, to 4.232. Proceeding successively to level 5, the point estimate of the number of hospitalizations, with 5 working hours/doctor per 1,000 inhabitants, is 3.19, given the dose-response function.
The column "impact (differences)" is the result of impact assessment for Brazil. The interpretation of the presented values is as follows: assuming a proper estimation of the counterfactual, the increase of 1 working hour/doctor at UPAs per 1,000 inhabitants impacts, on average, a decrease of 0.353 (4.232 -4.585 = -0.353) hospitalizations due to respiratory diseases per 1,000 inhabitants in Brazil, compared to the absence of UPAs. Likewise, with an increase of 5 working hours/doctor at UPAs, there is a reduction of 1.389 (3.196 -4.585 = -1.389) hospitalizations for respiratory diseases per 1,000 inhabitants, in relation to the 0 level. Another important result is that the addition of two hours of working hour/doctor at UPA generates an even greater result, proportionally, than the first hour. This is evidence of possible gains of scale in UPA care. From the second hour of work per 1000 inhabitants, however, estimates suggest that the contributions of additional hours decrease. All estimated differences are statistically significant since the limits of the confidence interval are negative. The behavior of the dose-response function may be better analyzed in a figure that shows the impacts of the additional working hours/doctor at UPAs. Figure 2 shows UPAs' impact on RHDRS in Brazil. As discussed earlier, higher levels of treatment reduce the rate of hospitalization. We notice more clearly, however, that the graphic slope decreases as working hours at UPAs increase. In other words, the impact of UPAs tends to decline as treatment levels rise. Confidence interval limits are narrow due to the sample size and the bootstrap procedure used.

Discussion and conclusion
Our evaluation of UPAs' impact on the rate of hospitalization for diseases of the respiratory system (RHDRS) led to important results for policymakers. Our findings point out that, in Brazil, the reduction of RHDRS was due to patients' care at UPAs, at a 95% confidence level. Therefore, there is evidence that UPAs have impact on on reducing that rate. For example, one hour/physician UPAs Impact 95% Confidence Interval in the UPA generates an estimated reduction of 0.353 in the hospitalization rate, compared to the absence of hours. This is equivalent to a reduction of approximately 8%. In addition, we notice that the reduction of RHDRS is lower when working hours/doctor in these units increase, as detailed previously. The findings suggest that scale gains occur up to the addition of 2 working hour/doctor at UPAs per 1,000 inhabitants. Still, an additional 5 hours reduces, on average, the rate by 30% when compared with the case without UPA. Thus, this study has important findings that can contribute to both the management of public health and UPA in Brazil. Evaluation is an essential part of the public policy cycle. After implementation, there must be evaluation and monitoring, so that the responsible managers can adjust it, change it, or even extinguish it, according to information resulting from the evaluation process. The main objective of the assessment is to add knowledge to the decision-making process, in order to improve public policies 29 . In view of the relevance of public health policies, strengthened in the year 2020 due to COVID-19 pandemic, the main objective of this study was to evaluate the impact of the 24hour Emergency Care Units (UPA) on hospitalizations for respiratory diseases in Brazil.
According to the guidelines of the National Policy for Emergency and Urgent Care, (PNAU), Emergency Care Units aim to provide urgent and emergency solution for clinical cases. Therefore, UPAs seek to prevent urgent cases of medium and low complexity from being referred to hospital units, and one of the expected effects is to reduce hospital overcrowding 1 . Research results show a decrease in hospitalization rates (RH-DRS) due to UPAs, taking into account the hypotheses of the presented model. This indicates that this strategy by SUS -the Brazilian public and free health system -, fulfills its objective of reducing the number of occupied hospital beds.
This study serves as input for decision-making by those responsible for public policies. Evaluation is part of the public policy cycle, and its results should support future decisions 29 . Previous studies have already shown that the presence of UPAs in municipalities has an impact on mortality rates and hospitalizations, therefore, are relevant for public health [3][4][5] . With our findings, it became evident that UPA is a responsive strategy that adds value to society, and should be adopted by public managers of any country with public health policies. Given the resources available and other society demands, managers may decide to implement, expand, or reduce investments in UPAs, considering the effect that each of these actions may have on the rates of hospitalization for respiratory diseases.
Evaluating the results of a government action also serves as an instrument of accountability to society. Thus, from the perspective of the New Public Service, public policy assessments show to the population the relevance of public action 30 . In this sense, this evaluation depicts the effect of UPAs on the health system, showing the relevance of this public policy.
Although our results regard the average of cities in Brazil, local specificities should be considered when analyzing them. Brazil is a broad and heterogeneous country, with different realities at each location. Therefore, the effect of UPAs on each municipality may be different from the Brazilian average. Likewise, government officials from other countries can use this study and adapt it to their reality.
The paper contributes to the literature on the subject and fills some gaps. Rocha and Fernandes 3 carried out an impact assessment of UPAs on mortality rates in the state of Rio de Janeiro, and Santos 4 evaluated UPAs' effect on mortality from infarction in Brazilian capitals and metropolitan regions. The results showed a negative effect of UPAs on mortality rates in both cases. However, these authors did not analyze the impact on hospital admissions, neither considered UPAs' size. According to the Ministry of Health guidelines, UPAs aim to relieve hospital overcrowding. Although Medeiro et al. 5 have found evidence of the impact of UPAs on hospitalizations, they only analyzed primary care-sensitive diseases and also did not consider the size of the health unit. Thus, our study fills these gaps, by assessing the effect of this component on hospitalizations for diseases of the respiratory system. In addition, we considered the size of the units, dividing the treatment effect into 5 levels, according to the working hours/doctor at UPAs.
Evidence found shows that the UPAs impact on hospitalizations for diseases of the respiratory system. The relationship was decreasing and non-linear, steeper at low levels of working hour/ doctor at UPA. This finding can be observed in the estimated dose-response curve, which follow the standard shape with decreasing benefits by treatment levels 31 . That is, considering UPAs' sizes, the benefits achieved by each additional unit seem to be smaller. Therefore, these results can support government managers in their decisions to allocate resources to implement or not Emergency Care Units, or to expand or reduce the units in operation.
During the research, we faced some limits that should be considered for future studies. We only analyzed hospitalizations for diseases of the respiratory system, but did not measure the impact on admissions for other types of diseases, which UPAs could also treat, or even the impact on the total number of hospital admissions. In addition, the X i vector is composed only of observable characteristics; however, there are unobservable attributes that could determine a city's decision on establishing or not a UPA. An example is the level of work, or the personality of a Health Secretary, which can affect the reality of a municipality's public health. In addition, one way to complement this research is to include a qualitative assessment of the impact of these units on the lives of its beneficiaries. Another limitation faced by this study is that the estimation of machine learning models is computationally complex and, therefore, we do not perform alternative strategies as robustness checking.
There are many ways to address this subject, in order to create knowledge to improve public policies. Several authors emphasize the relevance of evaluation research [32][33][34] , hence, we expect that the effort put into this evaluation study will serve for public managers' decision-making.
We expect that the evidence found will improve public health policies, especially in middle or low-income countries, where resources are scarce and health is more precarious. Many people die from preventable causes, and emergency pre-hospital units, such as UPAs, can help in such cases.

Collaborations
MV Wink Junior: conceptualization, methodology, writing -original draft preparation, writing -reviewing and editing, and supervision. FL Santos: conceptualization, writing -original draft preparation. MG Hoffmann: conceptualization, supervision, writing -reviewing and editing. LP Garcia: conceptualization, methodology, data curation, software and validation. All authors read and approved the final manuscript.