Proposed UNGASS indicators and the monitoring of the AIDS epidemic in Brazil

Abstracts

The objective of this study was to analyze, on the national level, the process of monitoring the proposed UNGASS indicators through the use of the Brazilian National Program for STD/AIDS indicators. Two groups of proposed indicators were analyzed in 2002 and 2005 respectively, as part of the monitoring of the progress of the UNGASS Declaration of Commitment. The availability of information and limitations in calculating the proposed indicators in Brazil were analyzed and the appropriateness of the indicators for monitoring the epidemic in Brazil was discussed. Of the 13 quantitative indicators originally proposed by UNGASS, five were not included in the National Program. One was not included due to its qualitative nature. Two of the indicators were considered to be of little use and two were not included due to the lack of available data needed for their calculation. As the epidemic in Brazil is characterized as being concentrated, within the second group of proposed UNGASS indicators those that refer to the accompaniment of epidemic among high-risk population groups were prioritized. The study highlights that the National Program concentrates its efforts in the development, adaptation, and sharing of sampling methodologies for hard to reach populations. Such activities are geared towards estimating the size of vulnerable population groups, as well as obtaining more information regarding their knowledge, attitudes, and practices. The study concludes that by creating the possibility of international comparisons between advances achieved, the proposal of supranational indicators stimulates countries to discuss and make their construction viable. In a complementary way, the national monitoring systems should focus on program improvement by covering areas that permit the evaluation of specific control and intervention actions.

HIV; Acquired immunodeficiency syndrome; Brazil; Program evaluation; National health programs; Outcome assessment (health care); Health status monitoring; Monitoring of epidemiological information; Indicators of health services; Health status indicators

O objetivo do estudo foi analisar o processo de monitoramento de indicadores centrais propostos pela UNGASS, em nível nacional, vis-à-vis aos indicadores do Programa Nacional de DST e Aids. Foram avaliados os dois conjuntos de indicadores propostos, respectivamente, em 2002 e 2005, para o monitoramento de progresso da Declaração de Compromisso da UNGASS. Foram analisadas a disponibilidade de informações e as limitações para o cálculo dos indicadores propostos no Brasil, incluindo discussão sobre a adequação dos indicadores propostos para o monitoramento da epidemia brasileira. Dos 13 indicadores quantitativos inicialmente propostos pela UNGASS, cinco não estão incluídos no sistema de monitoramento do Programa Pacional. Um deles não foi incluído por ser um indicador qualitativo. Dois dos indicadores foram considerados de pouca utilidade e dois não foram contemplados pela falta de dados disponíveis para o seu cálculo. Como o Brasil é caracterizado por ter uma epidemia concentrada, no segundo conjunto de indicadores propostos pela UNGASS, prioriza-se o acompanhamento da epidemia entre as populações sob maior risco. Destacou-se que o Programa Nacional centra seus esforços no desenvolvimento, adaptação e transferência de metodologias de amostragem em populações de difícil acesso. Tais ações são voltadas para a estimação do tamanho dos grupos vulneráveis, bem como para obter mais informações sobre seu comportamento, atitudes e práticas. Concluiu-se que, pela possibilidade de comparações internacionais dos avanços conseguidos, a proposição de indicadores supranacionais estimula os países a discutir e viabilizar sua construção. De maneira complementar, os sistemas nacionais de monitoramento devem ser focados na melhoria do programa, cobrindo áreas que permitam avaliar as ações de controle e intervenções específicas.

HIV; Síndrome de imunodeficiência adquirida; Brasil; Avaliação de programas; Programas nacionais de saúde; Avaliação de resultados (cuidados de saúde); Monitoramento do estado de saúde; Monitoramento de informação epidemiológica; Indicadores de serviços; Indicadores de saúde

ORIGINAL ARTICLES

Proposed UNGASS indicators and the monitoring of the AIDS epidemic in Brazil

Aristides Barbosa Junior^I; Ana Roberta Pati Pascom^I; Célia Landmann Szwarcwald^II; Carmen de Barros Correia Dhalia^I; Leandro Monteiro^I; Mariângela Batista Galvão Simão^I

^IPrograma Nacional de DST e Aids. Ministério da Saúde. Brasília, DF, Brasil

^IIDepartamento de Informação em Saúde. Centro de Informação Científica e Tecnológica. Fundação Oswaldo Cruz. Rio de Janeiro, RJ, Brasil

^{Correspondence}

ABSTRACT

The objective of this study was to analyze, on the national level, the process of monitoring the proposed UNGASS indicators through the use of the Brazilian National Program for STD/AIDS's indicators. Two groups of proposed indicators were analyzed in 2002 and 2005 respectively, as part of the monitoring of the progress of the UNGASS Declaration of Commitment. The availability of information and limitations in calculating the proposed indicators in Brazil were analyzed and the appropriateness of the indicators for monitoring the epidemic in Brazil was discussed. Of the 13 quantitative indicators originally proposed by UNGASS, five were not included in the National Program. One was not included due to its qualitative nature. Two of the indicators were considered to be of little use and two were not included due to the lack of available data needed for their calculation. As the epidemic in Brazil is characterized as being concentrated, within the second group of proposed UNGASS indicators those that refer to the accompaniment of epidemic among high-risk population groups were prioritized. The study highlights that the National Program concentrates its efforts in the development, adaptation, and sharing of sampling methodologies for hard to reach populations. Such activities are geared towards estimating the size of vulnerable population groups, as well as obtaining more information regarding their knowledge, attitudes, and practices. The study concludes that by creating the possibility of international comparisons between advances achieved, the proposal of supranational indicators stimulates countries to discuss and make their construction viable. In a complementary way, the national monitoring systems should focus on program improvement by covering areas that permit the evaluation of specific control and intervention actions.

Keywords: HIV. Acquired immunodeficiency syndrome. Brazil. Program evaluation. National health programs. Outcome assessment (health care). Health status monitoring. Monitoring of epidemiological information. Indicators of health services. Health status indicators.

INTRODUCTION

Monitoring and evaluation of health programs are increasing in importance among health sector administrators and officials. These activities constitute an essential stage in the planning of actions geared towards guaranteeing quality in health care attention.⁵

Monitoring and evaluation directed at management are focused on the production and use of information to improve the programs evaluated by integrating purely analytic and management activities.³ The main focus is to characterize a condition using measures that can be quantified and repeated. The quantitative methodology is predominant, although instruments with a qualitative approach that can be standardized may also be used.⁶

As such, health program evaluation requires choosing relevant characteristics or attributes to ensure that the approach to the problem is viable from a methodological point of view.¹³ In this process, there are important stages involving the definition and construction of indicators that allow the monitoring of programmatic actions and subsidize decision-making. The objective of evaluation is to propose the establishment of criteria or norms to be used as indicators to monitor the object being evaluated.¹

When the indicators are well formulated, are part of a monitoring and evaluation plan, and are interpreted within the external contexts and organizational realities they can relevantly contribute to the establishment of an efficient and effective Program. The efficiency consists in the capacity of the program to reach its proposed results with the smallest amount of resources, and the effectiveness, in the capacity of the program to reach its results.7,¹

In June 2001, 189 countries, including Brazil, adopted a Declaration of Commitment about HIV/AIDS, agreed upon during the 26^th Special Session of the General Assembly of the United Nations (UNGASS). This declaration reflects the global consensus regarding the slowing down of the AIDS epidemic by 2015.¹² In an effort to monitor the progress of the Declaration of Commitment, UNAIDS proposed a group of indicators at the national and supranational level that should be filled out, periodically, by the countries who signed the declaration.

The objective of the present study was to describe the process of monitoring the indicators proposed by UNGASS at the national level, relating them to the indicators of the Brazilian National Program for STD/AIDS.

SYSTEM FOR THE MONITORING THE (PN-DST/AIDS) NATIONAL PROGRAM INDICATORS: MONITORAIDS

The revitalization process of the monitoring and evaluation activities within the National Program was incentivized, especially, by the third loan agreement (AIDS III) between Brazil and the World Bank. During this process, it was understood that a fundamental strategy would be the development of a group of indicators to monitor the epidemic and Brazilian response to HIV/AIDS.

In this context, the National Program developed an Indicator Monitor System of the National DST/AIDS Program (MONITORAIDS). The system was developed in a partnership initiative with the Departamento de Informações em Saúde do Centro de Informação Científica e Tecnológica [Department of Health Information in the Scientific and Technology Center] (DIS/CICT) of the Fundação Oswaldo Cruz and the Centers for Disease Control and Prevention, Global AIDS Program Brazil (CDC/GAP-Brazil). This system meets the promise of the National Program to offer partners and society as a whole, helpful information that facilitates the accompaniment to the Brazilian response to control AIDS and other sexually transmitted diseases (STD). MONITORAIDS is a system of information available on the PN-DST/AIDS website in Portuguese, English, and Spanish.²

The development of the MONITORAIDS indicators followed the following principles: be relevant to monitor the HIV/AIDS and other STD epidemics; be useful in the evaluation of programmatic actives of the National Program; and suggest aspects to be evaluated in the future.

Social factors constitute fundamental elements in the determination of the morbity-mortality population standards. In addition to this, a large difference in socioeconomic classes also exists in relation to the utilization of health services in Brazil. Keeping these aspects in mind, it is necessary to consider the equity question as one of the central principles of the monitoring systems.

From a monitoring and evaluation standpoint, the system is the most important structural component of the Plano Nacional de Avaliação (PNA) [National Evaluation Plan] (NEP). It adds information not only about the technical components of the program, but also gives a dimension to the equity component and integral nature of the Brazil Unified Health System (SUS). The analysis of these indicators constitutes an indispensable strategy to making adjustments for the improvement of the Program.

Currently, MONITORAIDS presents 95 indicators. Depending on the availability of information, the indicators are broken down by: Region, Federal Units, age, sex, and a variable that characterizes socioeconomic class, which allows the analysis of these inequalities. For each indicator, there is a corresponding indicator profile, directed at the following aspects: definition/interpretation; objective; classification according to the monitoring categories; classification according to the PNA categories; calculation method; data source; possibility of analyzing the equity dimension; limitations; period of time; date of revision.

With regards to monitoring, the indicators are categorized in three dimensions: 1) contextual indicators; 2) indicators related to the program; 3) impact indicators (Table 1). The first dimension is established by the context in which the AIDS epidemic occurs in the country, represented by demographic characteristics and socioeconomic characteristics of the population, as well as the national health system indicators. The second is divided into 14 sub-areas related to the PN-STD/AIDS, which include individual vulnerabilities, prevention strategies, care given, AIDS vigilance, and STD prevention and control. The third dimension allows for the analysis of impact of the developed actions in terms of morbidity-mortality due to AIDS and other STDs.

The indicators classified as monitoring can also be categorized in accordance with the proposed institutions, such as the World Bank and UNGASS, institutions with whom the National Program signed an agreement to monitor indicators.

From an evaluation point of view, the indicators are classified according to the PNA.³ Within this logic, the indicators are first classified along evaluation guidelines, which include: external context, organizational context; implementation; performance; and impact. In this classification, the indicators are grouped into clusters of implementation (access and quality) and performance (effectiveness and user satisfaction). In the analysis, they are still classified according to technical components of the Program that includes epidemiological vigiliance, prevention, and care. In addition to these, the innovative component should also be included and reserved to adjust for unexpected operational program realities.

INDICATORS PROPOSED BY UNGASS

A group of indicators was developed in 2002 with the goal of establishing the progress of the implementation of the UNGASS Declaration of Commitment. Initially, they were grouped into three categories: 1) national commitment and action, focused on political, strategic, and financial inputs for the prevention of HIV infection; 2) national behavior and knowledge, with indicators related to the program products, results and reach; 3) national impact, with the goal of expressing the success of each program in reducing HIV infection rates.¹¹Table 2 shows a list of indicators which are central to UNGASS.

Of these 13 indicators initially proposed by UNGASS, five are not included in MONITORAIDS:

•

"National Composite Policy Index" - Because it is a qualitative indicator, composed of a series of questions related to the national policies related to the control of STD and AIDS.

• "Percentage of large businesses that have HIV/AIDS workplace policies." - To calculate this indicator, the businesses that develop a group of activities including the distribution of antiretroviral medications and provision of anonymous HIV/AIDS testing in the workplace are included. In Brazil, monitoring of this is not relevant as antiretroviral medications are provided through the public health system. In addition, it is not a government policy to stimulate testing at the workplace. •

"Percentage of patients with STDs at health clinics appropriately diagnosed, treated, and counseled" - This is because care provided to STD patients is decentralized and occurs, primarily, in the basic health care clinics. Additionally, technical and operational difficulties exist in the construction of this indicator, in terms of estimating the total number of patients with STDs and the number of patients with STDs that were appropriately diagnosed, treated, and counseled.

•

"Ratio of current school attendance among orphans and non-orphans aged 10-14 years old" - This index was not prioritized. The main reason is due to the fact that the epidemic in Brazil is concentrated, and presents an estimated prevalence rate of 0.41%¹⁰ among women in reproductive age and 5.8 deaths due to AIDS for every 100 thousand women between 15 and 49. To construct this indicator, a population-based household survey would be necessary with a large sample size, which would implicate excessive costs for little utility to monitor the Brazilian epidemic.

•

"Percentage of people with advanced HIV infection receiving antiretroviral combination therapy" - This is a relevant indicator. Nonetheless, it was not included because in accordance with UNGASS guidelines, 15% of the people infected with HIV are in advanced stages of infection. Remembering that 600,000 people are infected with HIV in Brazil¹⁰ and of these, 166,500 receive antiretroviral therapy (ARV), the percent of people in Brazil would be over 100%. Still, it is important to emphasize the percentage of patients that need to receive treatment in Brazil, but are not in ARV therapy, such as those who do not have symptoms and were never tested for HIV infection.

Some indicators proposed by UNGASS are relevant for the monitoring of the AIDS epidemic in Brazil, but are not contemplated in MONITORAIDS due to the lack of data available to calculate the indicator. One example is "the percentage of injecting drug users who adopted behaviors to reduce the risk of HIV transmission." The unavailability of this data is a result of the difficulty of conducting studies in populations that are difficult to access, such as injecting drug users (IDU). As an indicator of a change in risk behavior, the indicator, "percent of IDUs that do not share needles," is available, although the data comes from a specific study and does not represent all of Brazil.

The indicator, "Percentage of young people aged 15-24 that HIV-infected," was substituted in MONITORAIDS for two similar ones, "Percentage of individuals between 15-49 infected with HIV," and "Percentage of young men between 17-21 infected with HIV." These two were constructed from studies with representative samples of the country. To construct the indicator proposed by UNGASS, it would necessary to significantly increase the sample used in the studies of sero-prevalence.

The most recent estimates of the six common indicators and both monitoring systems are presented in Table 3.

The indicators proposed by UNGASS in 2002 were more relevant to generalized epidemics and of restricted utility for counties with concentrated epidemics. To address this problem, in July 2005, a new version of the indicators was made available, which resulted in a revision of the list of indicators previously proposed.¹² In this new version, two distinct groups of indicators were proposed in accordance with the nature of the epidemic in the country: concentrated or generalized. In the case of Brazil, which is characterized by a concentrated epidemic, the indicators aim to accompany high-risk populations. The high-risk populations are determined by each country, taking the characteristics of their epidemic into consideration. These indicators are listed in Table 4.

The National Program recognizes that these new indicators are relevant to monitor the epidemic in the country. However, their construction depends on the realization of a several studies with the important vulnerable populations in the Brazilian epidemic.

FINAL CONSIDERATIONS

The initiative to construct a supranational monitoring system based on standardized indicators allows the comparison of performance of various national HIV/AIDS control programs.¹² It also identifies regions where greater efforts and investments are needed to control the epidemic.

However, international comparisons have, at times, important limitations. First, due to the fact that countries are in different stages of the epidemic. In addition, there are enormous variations between nations, determined by distinct political, economic, social, and cultural contexts as well as organizational aspects of national health systems.

Another question is whether the indicators proposed by UNGASS, taken by themselves, have sufficient reach to be management tools and assist in decision making. This is due to the fact that these indicators do not attend to all the specificities and priorities of each country.⁴ Taking Brazil as an example, the reach of testing for HIV during pregnancy is an indicator that has been shown to be extremely relevant in monitoring prevention activities.⁸

An important restriction found in the first version of the UNGASS indicators was the group of indicators proposed which were of little importance for countries with concentrated epidemics, or with difficulties to calculate. For example, to calculate the prevalence rate of HIV among young people between 15-24 very expensive studies with extremely large sample sizes are necessary, limitations recognized by the group that coordinated UNGASS monitoring. In July 2005, the new version of indicators that differentiated between forms of the epidemic was disseminated (concentrated or generalized).

This new version totally moved the focus of the monitoring proposed in countries with concentrated epidemics. If one hand, the indicators are more relevant for vulnerable subgroups, on the other hand, those relevant to the general population are neglected, which may generate an interruption in a historic process.

Furthermore, in case of countries with concentrated epidemics, there is always the possibility that an epidemic may become generalized, and as such, there is a need to monitor indicators relevant to the general population. In the case of Brazil, epidemiological data indicate that socioeconomic class has progressively become a vulnerability factor,^2,9 noting an occurrence of high risk behaviors among the poor and in regions with lower social development.⁵

Regardless of the limitations pointed out in the present study, the change of the focus of UNGASS broadens the options of possible indicators, which is important for monitoring the dynamic of the spread of the epidemic in Brazil. In the case of countries with concentrated epidemics like Brazil, the current focus is on vulnerable populations, which facilitates participation in international scientific debates regarding methodologies to study these population groups. The importance of estimating the size of vulnerable population groups and obtaining information regarding their attitudes, behaviors, and practices should be recognized. However, difficulties exist in producing reliable estimates of these groups in population based studies. To respond to this situation, Brazil is currently focusing its efforts on developing, adapting, and sharing alternative sampling methodologies for hard to reach populations.

In conclusion, the proposed supranational indicators to monitor the epidemic permit international comparisons of advancements achieved and stimulate countries to discuss and make their (indicators) construction viable. In a complementary way, the national monitoring systems should focus on program improvement by covering areas that permit the evaluation of specific control and intervention actions. Independent of efforts to design monitoring systems which are adequate reflections of each country, the limitations imposed by the lack of data and/or the high costs involved in obtaining it, especially in developing countries, continues to be a challenge.

REFERENCES

Received: 3/4/2006.

• 1. Donabedian A. Criteria and standards for quality assessment and monitoring. QRB Qual Rev Bull. 1986;12:99-108.
• 2. Fonseca MGP, Travassos C, Bastos FI, Silva NV, Szwarcwald CL. Distribuição social da Aids no Brasil, segundo participação no mercado de trabalho, ocupação e status socioeconômico dos casos de 1987 a 1998. Cad Saúde Pública. 2003;19(5):1351-63.
• 3. Hartz ZMA. Institutionalizing the evalution of health programs and policies in France: cusine internationale over fast food and sur mesure over ready-made. Cad Saúde Pública. 1999;15:229-59.
• ⁴
  Ministério da Saúde - PN-DST/AIDS (Brasil). Manual da Oficina de Capacitação em Avaliação com Foco na Melhoria do Programa – Caderno do Professor. Brasília (DF): Ministério da Saúde, Programa Nacional de DST e Aids; 2005.
• 5. Ministério da Saúde - PN-DST/AIDS (Brasil). Pesquisa de Conhecimentos, Atitudes e Práticas na População Brasileira 2004. Brasília (DF): Ministério da Saúde, Programa Nacional de DST e Aids; 2006.
• 6. Novaes HMD. Avaliação em Saúde. Rev Saúde Pública. 2000;34(5):547-59.
• 7. Rugg DL, Heitgerd JL, Cotton DA, Broyles S, Freeman A, Lopez-Gomez AM et al. CDC HIV prevention indicators: monitoring and evaluating HIV prevention in the USA. AIDS. 2000;14:2003-13.
• 8. Souza-Júnior PRB, Szwarcwald CL, Barbosa-Junior A, Carvalho MF, Castilho EA. Detecção da infecção pelo HIV durante a gestação: resultados do Estudo-Sentinela Parturiente, Brasil, 2002. Rev Saúde Pública. 2004;38:764-72.
• 9. Szwarcwald CL, Castilho EA, Barbosa-Jr A, Gomes MRO, Costa EAMM, Malleta BV et al. Comportamento de risco dos conscritos do Exército Brasileiro, 1998: uma apreciação da infecção pelo HIV segundo diferenciais socioeconômicos. Cad Saúde Pública. 2000;16(Supl 1):113-28.
• 10. Szwarcwald CL, Barbosa Junior A, Souza Júnior PR, Pascom ARP, Esteves MA. Situação da Aids no Brasil: uma análise dos indicadores de monitoramento. In: Ministério da Saúde, Programa Nacional de DST e Aids. MONITORAIDS - Sistema de Monitoramento de Indicadores do Programa Nacional de DST e Aids versão 2.0. Brasília (DF): Ministério da Saúde, Programa Nacional de Aids; 2004. p. 68-100.
• 11. UNAIDS. Monitoring the Declaration of Commitment on HIV/AIDS: Guidelines on Constructions of Core Indicators. Geneva: UNAIDS; 2002.
• 12. UNAIDS. Monitoring the Declaration of Commitment on HIV/AIDS: Guidelines on Constructions of Core Indicators. Geneva: UNAIDS; 2005.
• 13. Vieira-da-Silva LM. The field of evaluation and the "sur mesure" strategy. Cad Saúde Pública. 1999;15(2):247-8.

1

Barbosa-Junior A. Conhecimento, práticas e comportamentos de vulnerabilidade relacionados à infecção pelo HIV e outras doenças sexualmente transmissíveis na população brasileira [anteprojeto de tese de doutorado]. Rio de Janeiro (RJ): Escola Nacional de Saúde Pública; 2004. Dados inéditos.

2

Ministério da Saúde. Sistema de Monitoramento de Indicadores do Programa Nacional de DST/Aids. Disponível em

http://www.aids.gov.br/monitoraids [acesso em 4 abr 2006]

3

Santos EM, Barbosa-Junior A, Pascom ARP, Dhalia CBC, Monteiro L. Plano Nacional de Monitoramento e Avaliação do Programa Nacional de DST e Aids. Disponível em

http://www.aids.gov.br/data/documents/storedDocuments/%7BB8EF5DAF-23AE-4891 -AD36-1903553A3174%7D /%7BB5D12E55-1000-40D9-B3F2-E73D4DE872B8 %7D/PNMeA _site.pdf [acesso em 22 fev 2006]

Publication Dates

• Publication in this collection
  14 Sept 2006
• Date of issue
  Apr 2006

History

• Received
  03 Apr 2006