External quality assessment in the voluntary counseling and testing centers in the Brazilian Amazon using dried tube specimens: results of an effectiveness evaluation

Abstract

INTRODUCTION

: In 2011, the Brazilian Ministry of Health rolled out a program for the external quality assessment of rapid human immunodeficiency virus (HIV) tests using the dried tube specimen (DTS) method (EQA-RT/DTS-HIV). Our objective was to evaluate the implementation of this program at 71 voluntary counseling and testing centers (VCTCs) in the Brazilian Legal Amazonian area one year after its introduction.

METHODS

: Quantitative and qualitative study that analyzed secondary data and interviews with healthcare workers (HCWs) (n=39) and VCTC coordinators (n=32) were performed. The assessment used 18 key indicators to evaluate the three dimensions of the program's logical framework: structure, process, and result. Each indicator was scored from 1-4, and the aggregate results corresponding to the dimensions were expressed as proportions. The results were compared to the perceptions of the HCWs and coordinators regarding the EQA-RT/DTS-HIV program.

RESULTS:

The aggregate scores for the three dimensions of structure, process, and result were 91.7%, 78.6%, and 95%, respectively. The lowest score in each dimension corresponded to a different indicator: access to Quali-TR online system 39% (structure), registration in Quali-TR online system 38.7% (process), and VCTC completed the full process in the program's first round 63.4% (result). Approximately 36% of the HCWs and 52% of the coordinators reported enhanced trust in the program for its rapid HIV testing performance.

CONCLUSIONS:

All three program dimensions exhibited satisfactory results (>75%). Nevertheless, the study findings highlight the need to improve certain program components. Additionally, long-term follow-ups is needed to provide a more thorough picture of the process for external quality assessment.

Voluntary counseling and testing centers; External quality assessment; Rapid tests; HIV; Dried tube specimens

INTRODUCTION

Voluntary counseling and testing centers (VCTCs) were established in Brazil in 1988 and are strategic for the promotion of equitable access to the serologic diagnosis of infections by the human immunodeficiency virus (HIV), syphilis, and hepatitis B and C. VCTCs also provide counseling before and after testing and offer resources for the prevention of sexually transmitted infections (STIs), among other services⁽¹⁾. The use of rapid HIV testing at VCTCs began following the publication of Municipal Health Secretary/Ministry of Health (MoH) resolution no. 151 in 2009, which standardized the sequence and basic logarithm for the laboratory diagnosis of HIV infections in individuals older than 18 months and in special situations⁽²⁾.

According to the Centers for Disease Control and Prevention (CDC), every facility that performs rapid HIV testing must be able to document and demonstrate its level of performance quality of serologic tests for HIV⁽³⁾. In 2005, resolution no. 302 of the Brazilian Health Surveillance Agency [Agência Nacional de Vigilância Sanitária (ANVISA)] Collegiate Board established that all laboratories conducting HIV rapid tests must be evaluated through internal and external quality assessments (EQAs)⁽⁴⁾.

External quality assessments is a method for objectively evaluating laboratory and health care professionals' performance and allows for the evaluation of inter-laboratory proficiency tests and the identification of related problems. Thus, it provides grounds for corrective and preventive actions on a regular basis⁽⁵⁾. After the successful scale-up of the rapid HIV testing in Brazil, several studies drew attention to the urgent need for comprehensive EQA programs to ensure that HIV tests were being performed accurately^{(6) (7) (8)}.

Because of the internationally and nationally expressed need for an established EQA system, the Department of Sexually Transmitted Diseases (STDs), acquired immunodeficiency syndrome (AIDS) and viral hepatitis of the MoH launched the external quality assessment of rapid HIV tests using the dried tube specimen (DTS) method (EQA-RT/DTS-HIV) Program at the VCTCs in 2011. This program aims to assess the level of technical performance of healthcare workers (HCWs) when using rapid HIV testing⁽⁹⁾.

The DTS method is based on the use of panels containing dry serum or plasma samples for testing the performance of workers carrying out the EQA for HIV without prior knowledge of the positive or negative status of the sample. DTS panels are stable at room temperature and can offer reliable results even in the basic VCTCs located in the remote interiors of the Amazon, which is characterized by its extreme temperatures and high humidity⁽¹⁰⁾. In Tanzania, which has rural health facilities in similar conditions as those in the Amazon, using DTS for EQA gave rise to 99.5% agreement between DTS and the rapid HIV test results⁽¹¹⁾. The EQA program using DTS was a useful and robust tool to identify HCWs who needed further training and triggered corrective actions to ensure the ability of the HCWs to correctly perform the test.

The EQA-RT/DTS-HIV program in Brazil includes the Quali-TR online system as a database. This software facilitates the process execution by generating management reports and providing information on the reception, assessment, and return of the DTS samples to the system manager⁽¹²⁾.

The introduction of a new health program is crucial to better understand interventions and the factors that account for discrepancies between planning and executing actions by examining the factors that facilitate or hinder the implementation of a new program^{(13) (14)}.

The aforementioned reasons account for the need to conduct the present study. We assessed the implementation of the EQA-RT/DTS-HIV program at the VCTCs located in the Brazilian Legal Amazonian area by comparing the information provided by HCWs and the data in the Quali-TR online system, following the program's first round in 2012.

METHODS

The study was conducted at the VCTCs located in the Brazilian Legal Amazonia. This region covers nine Brazilian States: Acre; Amapá; Amazonas; Pará; Rondônia; Roraima; and part of Mato Grosso, Tocantins and Maranhão. The area includes large sections of rainforest and covers approximately 5,217,423km². Although the area corresponds to 61% of the Brazilian territory, only 12.3% of the Brazilian population lives there, and this population is primarily indigenous.

Brazilian Legal Amazonia has 136 VCTCs, 100 of which offer rapid testing for HIV. The VCTCs are fixed facilities run by the MoH and are mainly utilized by vulnerable populations such as sex workers and men who have sex with men. Most of the VCTCs in the Brazilian Legal Amazonia operate from 8:00 to 16:00, uninterrupted. The VCTCs are equipped with laboratory supplies, and approximately 10% can also perform enzyme-linked immunosorbent assay (ELISA) tests. The staff performing counselling and rapid testing comprises nurses, doctors, and psychologists.

Seventy-one VCTCs in the Brazilian Legal Amazonia were eligible for the study because they offered rapid testing for HIV and were trained in 2011 to implement the EQA-RT/DTS-HIV program. These VCTCs were identified through the data available in the Quali-TR online system. The VCTCs were included in the study only if they had HCWs (nurse, psychologist, doctor, social worker, pharmaceutical-biochemical, or biologist) who had high technical skills and were trained by the MoH in 2011 as local trainers to implement the EQA-RT/DTS-HIV program. The content of the training covered recruitment, supervision, and evaluation of the EQA-RT/DTS-HIV program and DTS method performance. Such HCWs were identified through the Quali-TR database. Participation was also offered to VCTC coordinators to obtain program information from service managers. VCTCs were excluded if the dried tube specimens for the external quality assessment (DTS/EQA) had not been delivered by the post office, if they were not performing HIV rapid testing, or if they had changed the HCW team that had been initially trained as local trainers.

The present evaluation utilized both quantitative and qualitative methods. The data were collected in December 2012 and January 2013. The quantitative analysis employed secondary data that were available at the Quali-TR online system and the global report corresponding to the EQA-RT/DTS-HIV first round program (2012). We designed two questionnaires, one for HCWs and one for the VCTC coordinators. Both questionnaires were structured and included closed and open-ended questions. The questionnaires were sent to 71 HCWs and 71 VCTC coordinators via e-mail, along with an informed consent form and an invitation letter containing instructions for self-completion. Receipt of the e-mails was confirmed by phone.

The HCW questionnaire included 26 questions that assessed the following characteristics: knowledge of the EQA-RT/DTS-HIV program and its norms; training, including one question on whether they received training to act as trainers (multiplier agent) and if not, the reasons why, and three questions that rated the contents of the training, the instructions related to the DTS panel technical procedure, and the instructions related to the use of the QUALI-TR online system on a 0 to 10 scale; DTS/EQA panel kit; QUALI-TR online information system; assessment grading from 0 to 10 the delivery of DTS/EQA performed by the post office, including a yes/no question as to whether the results were available after its introduction to the QUALI-TR system; and perceptions of HCWs in the EQA-RT/DTS-HIV program. Perceptions were explored using four open-ended questions that related to the continuity of the EQA-RT/DTS-HIV program, positive and negative features of the program, and suggestions for changes. The questionnaire designed for VCTC coordinators included five questions that assessed their knowledge of the EQA-RT/DTS-HIV program, problems during program initiation, and their perceptions on whether the program eventually improved the performance of HCWs who involved in the EQA, and solicited suggestions for changes.

Quantitative data were described as absolute and percent frequencies for subsequent analysis in an assessment matrix. Following several readings, the qualitative data were categorized based on the question content. That information served as a complementary resource to improve the assessment and, in particular, to investigate the agents' perceptions related to the EQA-RT/DTS-HIV program.

The study design included a logic model, which was developed by the principal investigator as a first approximation to a schematic representation of the EQA-RT/DTS-HIV program (Figure 1). For that purpose, information was collected based on document analysis of didactic materials used for training in the EQA-RT/DTS-HIV program that were available on the MoH website⁽¹⁵⁾. Careful attention was paid to the program steps to achieve a thorough understanding of its components. The resulting information was organized based on the logic model mentioned by Brousselle et al. ⁽¹⁶⁾ on the notions of structure, process, and results formulated by Contandriopoulos et al.⁽¹⁷⁾.

Figure 1:
Logic model of the program for External Quality Assessment of Rapid HIV Testing using the DTS method based on didactic materials provided by the Health Ministry/Secretary of Health Surveillance/Department of STD/AIDS and viral hepatitis (Brasil, 2011)

The logic model served as the basis for our assessment matrix, which included the investigated dimensions, program components, and criteria/indicators. Table 1 details the calculations performed and sources of information used (i.e., analysis of secondary data or questionnaire) for each indicator.

Table 1:
Matrix used for the assessment of the Program for External Quality Assessment of Rapid Testing at Testing and Counseling Centers in the Brazilian Legal Amazonian area, Brazil, 2013.

• Adapted from Consendey et al⁽¹⁴⁾. ANVISA: Agência Nacional de Vigilância Sanitária; EQA-RT/DTS-HIV: external quality assessment of rapid human immunodeficiency virus (HIV) tests using the dried tube specimen (DTS) method; DTS/EQA: dried tube specimens for external quality assessment; HCWs: healthcare workers; SOP: standard operational procedure; VCTCs: voluntary counseling and testing centers.

The scores used to assess the implementation were calculated in two steps. The first step consisted of determining the expected and observed values of each investigated indicator. Because the indicators did not share the same scale, we assigned points to their values as a function of the obtained result so they could be added, and the value of each dimension could be calculated as described by Consendey et al.⁽¹⁸⁾. The points attributed to the expected values systematically corresponded to the highest possible value: 0-25% (1 point), 26-50% (2 points), 51-75% (3 points), and >75% (4 points). Once the expected and observed values of each dimension were estimated, the final calculation was performed, and an aggregate score was presented for each dimension. To classify the quality of implementation of the EQA-RT/DTS-HIV program at the investigated VCTCs, the percent values were replaced by quartile ranges that corresponded to the following scores according to Consendey et al.⁽¹⁸⁾: satisfactory, >75%; acceptable, 50 to 75%; unsatisfactory, 25 to 49%; and critical, <25%.

Ethical considerations

The study was approved by the ethics committee of the Federal University of Amazonas [Comitê de Ética em Pesquisa/Universidade Federal do Amazonas (CEP/UFAM)]. The use of secondary data was authorized by the National Department of STDs, AIDS, and Viral Hepatitis, MoH and the Federal University of Santa Catarina [Universidade Federal de Santa Catarina (UFSC)].

RESULTS

Of the 71 eligible VCTCs that were identified through the Quali-TR online system as initially trained in 2011, the following VCTCs were excluded from the study: six VCTCs in which the DTS/EQA kits were not delivered by mail according to the data included in the global report of the program's first round (2012)⁽⁹⁾, two VCTCs that were not performing HIV rapid testing, and seven VCTCs in which the HCWs who were initially trained were no longer working at the corresponding center. Of the remaining 56 VCTCs (37 located in the northern area, 8 in Mato Grosso, and 11 in Maranhão), four had their HCWs absent (holiday, leave) at the time of the study, 13 did not respond to the questionnaire, and 39 accepted participation and answered the questionnaire. Therefore, the participation rate was 69.6% (39/56). Of these 39 VCTCs with HCWs who were initially trained as local trainers that participated in the study, 32 VCTCs answered the questionnaire. By area, the distribution of the 39 included VCTCs were as follows: 26 (66.7%) were located in the Northern area (zero in Acre, ten in Amazonas, zero in Amapá, thirteen in Pará, one in Rondônia, zero in Roraima, two in Tocantins), five were located in Mato Grosso (12.8%), and eight were located in Maranhão (20.5%).

The results were used for the assessment matrix ( Table 1 ). The data related to the obtained results, expected and observed values of each indicator, and expected and final values of each dimension were organized in tables per dimension and program component ( Table 2 and Table 3 ).

Table 2:
Results corresponding to the structure dimension as average percent values and reference values. Brazilian Legal Amazonia. Brazil, 2013

• aReference values for obtained results of dimensions: >75% (satisfactory); 50-75% (acceptable); 25-49% (unsatisfactory); <25% (critical). ANVISA: Agência Nacional de Vigilância Sanitária; DTS/EQA: dried tube specimens for external quality assessment; DTS: dried tube specimen; SOP: standard operational procedure.

Table 3:
Results corresponding to the dimensions process and results as average percent values and reference values. Brazilian Legal Amazonia. Brazil, 2013

• aReference values for the obtained results of dimensions: >75% (satisfactory); 50-75% (acceptable); 25-49% (unacceptable); <25% (critical). EQA-RT/DTS-HIV: external quality assessment of rapid human immunodeficiency virus (HIV) tests using the dried tube specimen (DTS) method; DTS/EQA: dried tube specimens for external quality assessment; HCWs: healthcare workers; MoH: Ministry of Health; VCTCs: voluntary counseling and testing centers.

Dimension structure

Table 1 shows that the dimension structure included the following components: training of HCWs as local trainers (A), EQA-RT/DTS-HIV panel (B), and information system (C). All the examined indicators scored >75%, except for VCTC that had access to Quali-TR information system (C1), which scored 39%. In this regard, the interviewees reported unavailable internet connection at the center as the main factor that hindered access to the Quali-TR online system ( Table 2 ).

Dimension process

Regarding the dimension of process (Table 3), the indicators corresponding to the component qualification of human resources at the local level (D) (namely, performance of local training by HCWs who were trained as program trainers (D1), and registration of HCWs in the Quali-TR online system at the local level (D2)) scored 53.8% and 38.7%, respectively.

With regard to indicator D1, seven HCWs reported not being trained to that capacity, five did not find any HCWs to train, and six reported other (lack of time or opportunity).

Concerning indicator D2, seven HCWs reported that there were no other HCWs at the center who could be registered in the program, five reported difficulties in running the Quali-TR online system at the time of registration, five reported a lack of internet connection at the center to access the Quali-TR online system, and five reported other reasons (e.g., vacation, sick leave).

The indicators related to the DTS/EQA panel (E) component (i.e., sending DTS panels from MoH to centers (E1) and reception of complete DTS panels at VCTCs (E2)) scored 90.1% and 97.4%, respectively. In this regard, only one person among the 39 interviewed HCWs reported problems with the DTS/EQA panel kit content and reported that the tube with the rehydration buffer solution was empty on arrival.

The indicators related to the component information system scored 70.3% in F1, 66.7% in F2, and 59% in F3. Regarding indicator F1, 45 people from the 65 VCTCs that received the DTS/EQA panels reported the results of EQA to the Quali-TR online system database.

Regarding indicator F2, seven HCWs who responded to the questionnaires corresponded to the 20 VCTCs that did not report the EQA results to the Quali-TR online system. Three of them reported difficulties in using the information system, two stated they had successfully reported the results (although the corresponding data were not registered in the Quali-TR online system database), and two did not report the DTS/EQA panel results to the database because the deadline had passed.

Regarding indicator F3, 32 HCWs reported the results to the Quali-TR online system database, 10 of whom stated that they had no access to the template to check the reported EQA results.

Dimension results

The dimension of results included the component of assessment (G). All the corresponding indicators scored >75% (G2: 100%, G3: 100%, G4: 93.8%, G5: 97.4%), except for the VCTCs who completed the full process in the program's first round (G1) and scored 63.4% (Table 3). With regard to that indicator, the survey of the Quali-TR online system database showed that 40 of the 45 VCTCs that met the deadline performed full reports of the results.

The total score of each dimension showed that following its first round, the implementation of the EQA-RT/DTS-HIV program was satisfactory (>75%) in all three dimensions: 91.7% for the structure, 78.6% for the process, and 95% for the results dimensions (Table 2 and Table 3).

Perception and opinions of healthcare workers s and the voluntary counseling and testing center coordinators

HCW who were trained as local trainers were asked to rate the MoH training program from zero to ten. The average assigned grades were as follows: training content, 9.5; instructions related to DTS panel technical procedures, 9.4; and instructions related to the use of the Quali-TR online system, 8.4. The average grade attributed to the strategy adopted for the delivery of DTS/EQA by the post offices was 8.5, which was also rated on a zero to ten scale.

All HCWs involved in the process of the implementation of the EQA-RT/DTS-HIV program favored its continuity. A summary of the open-ended questions showed that, according to 32% of the interviewees, the EQA enhanced their self-reliance on the delivery of the results and reading of the diagnosis. In total, 30% considered the EQA relevant because it assessed the technical procedures that were performed, and 29% thought it was relevant because it ensured the efficiency and efficacy of the rapid HIV tests stored at the centers. Nine percent of the participants did not provide a response to this item.

Approximately 36% of the HCWs rated enhanced self-reliance on the performance of rapid tests as a positive feature of the program, 28% reported the assessment of the quality of (rapid test) kits available at VCTCs as positive, 26% reported promotion of self-assessment by the HCW due to the test technical procedures as positive, and 8% reported the DST method used in the EQA of rapid HIV testing as a positive feature of the EQA-RT/DTS-HIV program.

Approximately 31% of the interviewees did not report any negative features of the EQA-RT/DTS-HIV program. However, some negative features were noted by HCWs, such as the mail delivery logistics (23%), an insufficient number of HCWs trained as program local trainers (8%), the Quali-TR online system (20%), and other negative features (e.g., lack of periodicity in the delivery of the DTS/EQA panel kits, lack of communication with the program managers, and unavailability of the template to check the results) by 18% of the HCWs.

Some HCWs were assigned to VCTCs with results that were not reported through Quali-TR online system. In these cases, slow mail delivery (it took too long to arrive) was reported as a negative feature. Other HCWs were assigned to VCTCs with results that were reported through the Quali-TR online system, and access to Quali-TR program via the internet and unavailability of template for checking the results were reported as negative features.

With regard to the 32 VCTC coordinators who were asked an open-ended question regarding how the program improved the HCWs' performance, their answers could be summarized in four categories. Half (52%) perceived that the HCW performed the corresponding actions more reliably and with enhanced self-reliance; 35% stated that improvement was reflected in the quality of assistance and performance of the tests, as assessment compels the HCW to review methods and biosafety issues; and 4% stated that the training improved their technical skills related to the performance of rapid testing. Nine percent of the sample did not respond to this item.

When HCWs were asked to suggest changes to improve the EQA-RT/DTS-HIV program, the most frequent responses reported were related to the periodicity of EQA and training. The following quotes reflect the perception of many of the responses raised by them: "(...) it's not like an adjustment, but to ensure the continuity of the process-how?-by sending the three shipments described at the training program" "more frequent assessment and a larger number of assessment kits;" and "as concerns the availability of training in this method, it would be better to include a larger number of the center professionals".

Similar suggestions were also made by the VCTC coordinators, who additionally suggested extending the scope of quality control to other rapid tests (syphilis and hepatitis) and delivering EQA certificates to the centers that perform successful assessments.

As a whole, the HCWs tended to praise the DTS method used in the EQA-RT/DTS-HIV program and expressed particular satisfaction with the rehydration technique: "it's easy to understand" and "I believe the prescribed procedure works well".

DISCUSSION

Considering the structure dimension, the indicator training of HCWs as local trainers (A) exhibited the maximum possible score (>75%, satisfactory), which was aligned with the high grade (9.5 on average) that HCWs gave to the training administered by the MoH. These findings were reinforced by data in the global report of the EQA-RT/DTS-HIV program's first round⁽⁹⁾, which showed 100% agreement with the results reported by the HCWs who worked at VCTCs that succeeded in reporting their results through the Quali-TR online system.

The study conducted by Parekh et al.⁽¹⁰⁾, in which the participants described the DST method as simple to apply, are consistent with the perceptions of the HCWs interviewed in the present study, who also described it as simple to understand and perform.

Nevertheless, the component information (C1) system of the structure dimension scored only 39% (unsatisfactory) due to the difficulties encountered by the interviewed HCWs in accessing the system (unavailability of internet connection at the center). According to a study conducted by the MoH⁽¹⁹⁾, this lack of access is a nationwide problem. Only 60% of VCTCs have access to internet throughout the country, and only one-third (30.5%) have access in the northern areas. This problem might be partially addressed with the installation of 166 internet antennas across the Legal Amazonian states in 2013 within the context of the System for Protection of Amazonia [Sistema de Proteção da Amazônia (SIPAM)]⁽²⁰⁾.

With regard to the process dimension, although performance was rated as satisfactory as a whole, some indicators exhibited low scores.

The fact that the registration of HCWs in the Quali-TR online system at the local level scored only 38.7% is a cause for concern. The interviewees reported that the trained HCWs did not believe that they had sufficient skills to perform that activity. A study by Mie Okamura et al.⁽²¹⁾ conducted during the implementation of rapid HIV testing in the State of Amazonia found similar findings. Although HCWs were duly trained, they did not perform testing for various reasons, including their self-perception of not having enough technical skills.

One additional factor associated with the lack of training of the HCWs by the trainers (multiplier agents) at the local level was the lack of HCWs to be trained at VCTCs. The MoH⁽¹⁹⁾ also previously reported a discrepancy between the actual number of high-skilled employees and the number indicated in the Guidelines for VCTCs Operation.

According to the Stork Network manual, which was developed by the MoH⁽²²⁾, a VCTC is considered an experienced facility for the expansion of the EQA-RT/DTS-HIV program to other basic healthcare networks. In other words, HCWs at VCTCs can provide assistance regarding the application of the DTS method and rapid testing. Mie Okamura et al.⁽²¹⁾ previously emphasized the relevance of selecting eligible HCWs for training to increase the supply of services.

Furthermore, the fact that only 45 of 65 VCTCs that received the DTS/EQA panel kits reported results is a cause for concern. A comparison of that finding to the responses given related to the corresponding indicator (F2) by the HCWs who did not report results points to a lack of precision in the Quali-TR online system because some results were indeed reported but were not registered by the system. One additional problem was the slow delivery of the DTS/EQA panel kits by the post offices, which hindered the reporting of results within the indicated timeframe.

These factors might explain why some VCTCs did not report the results of their assessments before the deadline, as described in the Quali-TR online system manual of the MoH⁽¹²⁾.

Ten HCWs assigned to the VCTCs included in the study did not have access to the template for checking results (indicator F3). According to the ControlLab manual⁽²³⁾, participants who perform proficiency tests should have access to complete information on its functions and to instructions meant to ensure efficient participation, which demands contact with suppliers and access to the reports of the assessments. In support of those facts, a study on the perceptions of pathologists about EQA showed that feedback related to the obtained scores encouraged improving their performance and that internet-based interaction played a relevant role because it facilitated solutions to negative performances⁽²⁴⁾.

Therefore, because the EQA-RT/DTS-HIV program needs to ensure the operation of the Quali-TR online system⁽¹⁵⁾, strategies to improve the system performance should be reviewed so all the centers participating in the EQA have access to inter-laboratory comparisons.

Regarding the dimension of results, although the indicator knowledge of the EQA-RT/DTS-HIV program by VCTC coordinators (G2) and HCWs trained as local trainers (G3) scored 100%, it is worth calling attention to the discrepancy between the calculated results and the HCWs' perceptions. Approximately 29% of the HCWs stated that the program provides a way to ensure the efficiency and efficacy of the rapid tests stored at the center. Such a misconception reappeared when the HCWs were asked about the positive features of the implementation process, as 28% of them answered assessment of the quality of the (rapid test) kits available at the VCTC. These findings point to the need to make the goal of the EQA, which is to assess the technical performance of the HCW⁽⁹⁾, clearer during the training sessions.

Related to the VCTC coordinators' knowledge of the program goals, the results provided by calculation (93.8%) and the reported perception did not exhibit discrepancies. However, in the study by Paham et al.⁽²⁴⁾, 80% of the HCWs reported that the VCTC coordinators did not understand what the goal of EQA was. Regarding the perceived negative features of the program, the reports of the HCWs were in agreement with the assessment matrix results. In both cases, mail delivery logistics and access to the Quali-TR online system appeared as the main negative features.

The suggestions made by both HCWs and VCTC coordinators, as well as by the participants in the study by Parekh et al.⁽¹⁰⁾, point to the need to expand the scope of the EQA-RT/DTS-HIV program to include other tests. In addition, these responses indicated a lack of periodicity in the delivery of the kits, as indicated at the training sessions; the need to train more than one HCW per VCTC; and the certification of centers as further factors to improve the success of the program at VCTCs.

The satisfactory results related to the examined dimensions following the program's first round (2012) do not reflect the degree of implementation of the EQA-RT/DTS-HIV program. Brousselle et al.⁽¹⁶⁾ observed that thorough knowledge of a targeted intervention is required in evaluation studies to avoid inferring mistaken conclusions regarding its effectiveness, while the results of interventions that do not exhibit a satisfactory level of operationalization are not liable to measurement. Thus, as in most continuous programs, long-term follow up is required to obtain a more thorough picture of the performance of the assessment process.

The results corresponding to the program's first round point to the need to improve the Quali-TR information system because inter-laboratory follow-up depends on it. Facing the poor knowledge of the HCWs regarding the investigated subject, the training of human resources personnel should be immediately improved, as should the monitoring of the logistics for mail delivery of the DTS/EQA panel kits to their destinations, including confirmation of arrival.

Nevertheless, the satisfactory results, such as the satisfaction of HCWs with the application of quality control and their enhanced self-reliance in the generation of the rapid HIV test results, lend further support to continue with the implementation of this novel strategy to improve the performance of the rapid HIV testing results at VCTCs.

Of the 71 VCTCs that were initially trained and eligible for the study, 15 (21.1%) could not be evaluated due to exclusion criteria. Although we lost some data, the overall participation rate of the included VCTCs was 69.6%, and the states of the whole Brazilian Legal Amazonian were represented in our study. The data collected in the present study are mostly regional and cannot be assumed to represent the situation in other areas of the country. Nevertheless, the results might be indicative of possibilities, as well as of hindrances to be overcome, related to the operationalization of the EQA-RT/DTS-HIV program in its future rounds in Legal Amazonia and Brazil as a whole.

Point-of-care rapid HIV testing is suitable for regions such as the interior of the Amazon, which are very remote and have a shortage of laboratory infrastructure. Rapid HIV testing is widely used in the VCTCs of the Brazilian Legal Amazonian, where 100 (73,5%) of 136 VCTCs have rapid HIV testing in place. However, similar to the context in Africa⁽⁷⁾, the extension of rapid HIV testing has not maintained the same pace as the extension of the EQA program. Of the 100 VCTCs with rapid HIV testing in place, 71 were initially trained in 2011 to implement EQA program. However, in 15 of those VCTCs, the EQA program was not started (two no longer offering rapid HIV testing, six in which DTS/EQA panel was not delivered by post, and seven that lost trained HCWs). As a result, only approximately half of the VCTC with rapid testing in place (56 of 100) were implementing the EQA program. Moreover, in VCTCs where the EQA program was implemented, we identified some operational barriers regarding the EQA program, such as the delivery of DTS/EQA panel by the post, irregular Internet connection, and not enough trained professionals. These findings raise potential concerns about the accuracy of rapid HIV testing and call for a wider and systematic implementation of the EQA program in the Brazilian Legal Amazonia. Consistent EQA using the DTS method and further training are needed to monitor the performance of rapid HIV testing and to sustain accuracy in the long term.

ACKNOWLEDGMENTS

We acknowledge help from the Department of STD/AIDS and Viral Hepatitis for making data relevant to the VCTCs available for the present study.

• ¹
  Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de DST, Aids e Hepatites Virais. Diretrizes para Organização e Funcionamento dos CTA do Brasil. Brasília: Ministério da Saúde; 2010. [Cited 2014 April 21]. Available at: http://www.aids.gov.br/publicacao/diretrizes-para-organizacao-efuncionamento-dos-cta/.
  » http://www.aids.gov.br/publicacao/diretrizes-para-organizacao-efuncionamento-dos-cta/
• ²
  Ministério da Saúde. Portaria n 151, de 19 de outubro de 2009. Regulamenta o uso de testes rápidos para diagnóstico da infecção pelo HIV em situações especiais. Brasília: Diário Oficial União. [Cited 2014 April 21]. Available at: http://bvsms.saude.gov.br/bvs/saudelegis/svs/2009/prt0151_14_10_2009.html/.
  » http://bvsms.saude.gov.br/bvs/saudelegis/svs/2009/prt0151_14_10_2009.html/
• ³
  Centers for Disease Control and Prevention (CDC). Guidelines for Appropriate Evaluations of HIV Testing Technologies in Africa. CDC; 2001. [Cited 2011 March 1]. Available at: http://stacks.cdc.gov/view/cdc/11100/.
  » http://stacks.cdc.gov/view/cdc/11100/
• ⁴
  Ministério da Saúde. Agência Nacional de Vigilância Sanitária. Resolução da Diretoria Colegiada n 302, de 13 de outubro de 2005. Dispõe sobre Regulamento Técnico para funcionamento de Laboratórios Clínicos. Brasília:. Diário Oficial União [Cited 2014 April 21]. Available at: http://bvsms.saude.gov.br/bvs/saudelegis/anvisa/2005/res0302_13_10_2005.html/.
  » http://bvsms.saude.gov.br/bvs/saudelegis/anvisa/2005/res0302_13_10_2005.html/
• ⁵
  Ministério da Saúde. Departamento de DST, Aids e Hepatites Virais. HIV: Estratégias para utilização de testes rápidos no Brasil. Brasília: Ministério da Saúde; 2008.
• ⁶
  Shott JP, Galiwango RM, Reynolds SJ. A quality management approach to implementing point-of-care Technologies for HIV diagnosis and monitoring in Sub-Saharan Africa. J Trop Med 2012; 2012:651927.
• ⁷
  Parekh BS, Kalou MB, Alemnji G, Ou CY, Gershy-Damet GM, Nkengasong JN. Scaling up HIV testing in developing countries: comprehensive approach for implementing quality assurance. Am J Clin Pathol 2010; 134:573-584.
• ⁸
  Benzaken AS, Bazzo ML, Galban E, Pinto IC, Nogueira CL, Golfetto L, et al. External quality assurance with dried tube specimens (DTS) for point-of-care syphilis and HIV tests: experience in an indigenous populations screening programme in the Brazilian Amazon. Sex Transm Infect 2014; 90:14-18.
• ⁹
  Ministério da Saúde. Departamento de DST, Aids e Hepatites Virais. Avaliação Externa da Qualidade Testes Rápidos para o diagnóstico do HIV. Relatório Global. Florianópolis (SC): Departamento de DST, Aids e Hepatites Virais; 2012.
• ¹⁰
  Parekh BS, Anyanwu J, Patel H, Downer M, Kalou M, Gichimu C, et al. Dried tube specimens: a simple and cost-effective method for preparation of HIV proficiency testing panels and quality control materials for use in resource-limited settings. J Virol Methods 2010; 163:295-300.
• ¹¹
  Smit PW, Mabey D, van der Vlis T, Korporaal H, Mngara J, Changalucha J, et al. The implementation of an external quality assurance method for point-of-care tets for HIV and syphilis in Tanzania. BMC Infect Dis 2013; 13:530.
• ¹²
  Ministério da Saúde. Coordenação Nacional de DST e Aids. Quali-TR: Manual técnico do usuário. Florianópolis (SC): Coordenação Nacional de DST e Aids; 2011.
• ¹³
  Denis JL, Champagne F. Análise de Implantação. In: Hartz ZMA, editor. Avaliação em Saúde: dos modelos conceituais à prática na análise da implantação de programas. Rio de Janeiro: FIOCRUZ; 1997. p. 49-88.
• ¹⁴
  Champagne F, Brousselle A, Contandriopoulos AP. A Análise dos Efeitos. In:, Brousselle A editor. Avaliação: conceitos e métodos. Rio de Janeiro: FIOCRUZ; 2011. p 159-182.
• ¹⁵
  Ministério da Saúde. Departamento de DST/Aids e Hepatites Virais. Material instrucional para Capacitação na metodologia DTS (Dried Tube Specimens) para Avaliação Externa da Qualidade - AEQ dos testes rápidos para HIV e sífilis. 2011. [Cited 2011 December 12]. Available at: http://www.aids.gov.br/.
  » http://www.aids.gov.br/
• ¹⁶
  Brousselle A, Champagne F, Contandriopoulos AP, Hartz ZMA.: Avaliação: conceitos e métodos. Rio de Janeiro FIOCRUZ; 2011.
• ¹⁷
  Contandriopoulos AP, Champagne F, Denis JL, Pinealt R. A Avaliação na Área da Saúde: Conceitos e Métodos. In:, Hartz ZMA editor. Avaliação em Saúde: dos modelos conceituais à prática na análise da implantação de programas. Rio de Janeiro: FIOCRUZ; 1997. p.29-47.
• ¹⁸
  Cosendey MAE. Análise de implantação do programa farmácia básica: um estudo multicêntrico em cinco estados do Brasil. (Doctor´s Thesis) Rio de Janeiro, Escola Nacional de Saúde Pública. Fundação Oswaldo Cruz; 2000 Rio de Janeiro, 2000. 358 p
• ¹⁹
  Ministério da Saúde. Secretaria de Vigilância em Saúde, Programa Nacional de DST e Aids. Contribuição dos Centros de Testagem e Aconselhamento para universalizar o diagnóstico e garantir a equidade no acesso aos serviços. Brasília: Ministério da Saúde; 2008. [Cited 2014 April 21]. Available at: http://bvsms.saude.gov.br/bvs/publicacoes/centros_testagem_aconselhamento_brasil.pdf/
  » http://bvsms.saude.gov.br/bvs/publicacoes/centros_testagem_aconselhamento_brasil.pdf/
• ²⁰
  Ministério da Saúde. Ministério do Desenvolvimento Social e Combate a Fome. Banda larga vai melhorar gestão do Cadastro Único em 166 municípios da Amazônia Legal. 2013[Cited 2013 March]. Available at: http://www.mds.gov.br/saladeimprensa/noticias/2012/.
  » http://www.mds.gov.br/saladeimprensa/noticias/2012/
• ²¹
  Okamura M, Santos EM, Cruz MM, Vasconcelos AL. Avaliação da implantação do teste rápido para diagnóstico do HIV no Estado do Amazonas. Rev Eletrônica Portas 2008; 2(2). [Cited 2014 April 21]. Available at: http://arca.icict.fiocruz.br/handle/icict/4940/.
  » http://arca.icict.fiocruz.br/handle/icict/4940/
• ²²
  Ministério da Saúde. Secretaria de Atenção a Saúde. Orientações para implantação dos testes rápidos de HIV e Sífilis na Atenção Básica: Rede Cegonha. Brasília: Secretaria de Atenção a Saúde; 2012. [Cited 2014 April 21]. Available. at: http://dab.saude.gov.br/portaldab/biblioteca.php?conteudo=publicacoes/ orientacoes_implantacao_testes_rapidos_hiv_sifilis/.
  » http://dab.saude.gov.br/portaldab/biblioteca.php?conteudo=publicacoes/ orientacoes_implantacao_testes_rapidos_hiv_sifilis/
• ²³
  Controle de Qualidade para Laboratórios LTDA (ControlLab). Gestão da fase analítica do laboratório: como assegurar a qualidade na prática. Volume II. 1ª Ed. Rio de Janeiro: ControlLab; 2011.
• ²⁴
  Parham DM. External quality assessment slide schemes: pathologists' experiences and perceptions. J Clin Pathol 2006; 59:530-532.

• The National AIDS/STI and Viral Hepatitis Program from the Brazilian Ministry of Health provided financial support for this study. Andréa Mônica Brandão Beber received a scholarship from the Fundação de Amparo à Pesquisa do Estado do Amazonas (FAPEAM) from 2011-2013. Dr. Sabidó received a fellowship as a visiting researcher at the Fundação de Medicina Tropical Dr. Heitor Vieira Dourado funded by the FAPEAM through the Programa Estratégico de Ciência, Tecnologia, & Inovação nas Fundações de Saúde, Amazonas (PECTI/AM SAÚDE)

Publication Dates

• Publication in this collection
  May 2015

History

• Received
  21 May 2014
• Accepted
  04 Sept 2014