Analysis of the distribution of health services in the State of São Paulo: focus on Big Data

Objective  To develop a process (pipeline) for extracting, processing, and analyzing data from the National Registry of Health Establishments in the State of São Paulo, Brazil, to evaluate the distributions of health professionals and services throughout this state.

Methods  Big Data resources were used to acquire, process, and aggregate health-related information, with the creation of a relational database for the local storage of processed data. The entire process is implemented using a framework that enables the creation and editing of a series of instructions for performing specific tasks (scripts) in different languages. For the end user interface, an interactive panel of information, metrics, graphs, and numerical indicators was created to show the distribution of health professionals and services across the municipalities of the State of São Paulo.

Results  The developed tool generates a process for monthly updating of a database, producing a dynamic information report that allows users to perform queries on quantitative and qualitative indicators, providing a general overview of health establishments, and furnishing information on specific health professionals.

Conclusion  The preliminary results indicated that the developed tool is potentially scalable and could contribute to the identification of regions of the state requiring action from public authorities, helping to optimize the hiring of doctors and community health agents, among others.

Big Data; Health services accessibility; Geographic locations; Health facilities; Health personnel; Power BI; Dashboard systems; Apache Spark; CNES; PostgreSQL; Database; Public health

■ The extract, transform, and load algorithm automates data extraction and updates in the PostgreSQL database.

■ The interactive dashboard enables detailed queries on establishments and professionals.

■ Results highlight medical doctor deficit in 22 cities and a shortage of community agents.

■ The tool can scale and assist managers in resource allocation for healthcare.

A tool was developed to extract, process, and analyze data from the National Registry of Health Establishments of São Paulo, creating a relational database and an interactive panel. This tool enables queries about health professionals and services, generating dynamic reports that assist in the optimized allocation of resources and public management.