Recommendations for abortion surveys using the ballot-box technique

Recomendações para inquéritos sobre aborto usando a técnica de urna

ARTICLE ARTIGO

Marcelo Medeiros^I; Debora Diniz^II

^IDepartamento de Sociologia, Universidade de Brasilia. Campus Universitário - ICC Centro, Asa Norte. 70.910-900 Brasilia DF. [e-mail]

^IIPrograma de Pós-Graduação em Política Social, Universidade de Brasília

ABSTRACT

The article lists recommendations for dealing with methodological aspects of an abortion survey and makes suggestions for testing and validating the survey questionnaire. The recommendations are based on the experience of the Brazilian Abortion Survey (PNA), a random sample household survey that used the ballot-box technique and covered adult women in all urban areas of the country.

Keywords: Induced abortion, Questionnaire design, Ballot-box technique, Brazilian abortion survey, PNA

RESUMO

O artigo lista recomendações para lidar com aspectos metodológicos de um inquérito sobre aborto e faz sugestões para testar e validar o questionário do levantamento. As recomendações baseiam-se na experiência da Pesquisa Nacional de Aborto (PNA), uma pesquisa domiciliar baseada em amostra aleatória da população urbana do Brasil que utilizou a técnica de urna.

Palavras-chave: Aborto induzido, Desenho de questionários, Técnica de urna, Pesquisa Nacional de Aborto (PNA)

Introduction

Unsafe abortion and its complications have become a public health problem in Latin America, Africa and South-Central Asia. In order to address this problem public health policies require plans based on evidence, such as that provided by abortion surveys. Based on the Brazilian National Abortion Survey (its acronym in Portuguese is PNA) we make recommendations for dealing with some particularities of these surveys. The PNA was a random sample household survey that used the ballot-box technique and covered women aged 18 to 39 years in all urban areas of Brazil. It was the first national household survey on abortion conducted in the country.

We believe the experience gained through the PNA can bring simple solutions for problems that researchers will face in developing countries when conducting similar surveys. To illustrate our recommendations we present the key features in the design of the PNA and some of its methodological aspects that deserve highlighting.

Recommendations

The ballot-box technique

In the above mentioned regions, induced abortion is a sensitive issue, and, in most cases, also illegal. Truthful responses on surveys can result in moral or even legal coercion. As a consequence, conventional face-to-face interviews have found high levels of refusals and of non-responses. Because of the fear of moral and legal coercion among women, more than ensuring confidentiality, an abortion survey has to give respondents a strong feeling of confidentiality. The ballot-box technique is one of the simplest ways to give this feeling. It consists of a self-responded written questionnaire, answered by literate women which is then put in a ballot-box. The ballot-box is promptly associated with the secrecy of political elections, an association that increases response rates.

The technique was originally designed to survey vote intentions¹ and its use for abortion surveys dates to the early 1990s². It achieves one of the lowest false or non-response rates when compared to other approaches^2,3. In Latin America it has been applied nationwide in Colombia⁴ and in smaller studies in Mexico and Brazil^2,5.

Designing questionnaires

The design of the questionnaire of PNA combined qualitative and quantitative methods. It involved three rounds of consultation with independent anonymous (single blinded) specialists, two successive pre-tests with alternative questionnaires applied to random samples of households (n=85 and n=60 women) and a split-halves test of reliability (n=2002). Each pre-test interview consisted of the application of the questionnaire and a meta-interview with questions about the questionnaire itself. The reliability test was an ex post test. The recommendations bellow are mostly based on the results of the meta-interviews.

To summarize the findings of the meta-interviews and consultations, we found that:

(1) the technical wording differentiating induced from spontaneous abortion should be avoided. Particularly in the romance languages of Latin America, asking whether a women "induced an abortion" leads to confusion because "induced abortion" is medical jargon. Women do know whether they "had an abortion" (induced), "had a miscarriage" (spontaneous) [In Portuguese and Spanish it would be literally "made an abortion" (active) for induced abortion and "had an abortion" (passive) for a miscarriage.] or "provided an abortion" (on another woman).

(2) The ballot-box questionnaires should be kept as short as possible, limited to the abortion questions. Non-confidential information such as social and demographic data should preferably be collected using conventional face-to-face interviews. If needed, the matching of the two can be done using a pre-printed discrete encrypted code, with a key to match the two questionnaires not known by interviewers. This is advisable because people feel their confidentiality is threatened when they provide personal information on the same form they provide information about abortion.

(3) In the ballot-box form, the flow of the questions should be graphically determined. Logical changes in the flow - for example, instructions such as "if "yes" skip to question 5" - can be excessively complex to be understood by less educated people (in the pre-test 13 out of 60 women ignored this instruction). A graphical design with arrows that force the changes in the flow is much easier to understand (0 errors in 2002 women).

(4) Interviewers should preferably be women. Women prefer to be interviewed by other women in a survey about abortion, even when they know that interviewers will have no access to their answers. This avoids the risk of refusing to answer a male interviewer in their homes⁶.

(5) In the ballot box self-completion questionnaire, direct questions with closed response alternatives (such as "yes" and "no") have better acceptance because they are simpler to understand and answer⁷. Written instructions should be avoided as they tend to confuse respondents, pushing them to ask interviewers for help.

Testing and validating the questionnaires

If the survey opts for combining the ballot-box questionnaires to collect sensitive data to face-to-face interviews collecting more general information, the use of existing questions from national or regional demographic surveys is advisable. This facilitates comparison, the application of statistical methods for inference, and the focusing of resources, since it spares the effort of re-testing already well established tools.

The abortion questions, however, have to be tested and validated. Criteria, content and construct validation, and reliability verification of a ballot-box abortion questionnaire is not as simple as in other types of surveys. Where abortion is either a crime or a taboo issue, it is hard to imagine a way to criteria validate a questionnaire in a household survey. Administering tests among women with known previous cases of abortion could perhaps allow such validation, but it would not be easy to identify and track women who aborted a long time ago. In the PNA no criteria validation was possible.

In the case of PNA, the validation of the content of the questions by a panel of specialists including epidemiologists, anthropologists, sociologists and demographers was effective to anticipate potential wording problems. Following an extensive review of qualitative studies on abortion⁸ we found that language sensitivity is key for this type of interview. Women are not always familiar with medical terminology and even when they understand it, they may re-describe the meaning of this terminology: for example, associating ´to induce abortion´ with reproachable cold intention and ´to had an abortion´ to a more acceptable act.

The self-completion PNA questionnaire had five questions, one seeking information on whether women had induced abortion and questions about age at the time of the last abortion, hospitalization, use of drugs to initiate the abortion and a check question about age today. The panel of specialists consulted recommend the use of simple language for the question about induction of abortion, and in the qualitative interviews of the pre-tests it became clear that the wording of the question should be non-technical. We opted for "have you ever had an abortion? (yes/no)" - with had being the common expression indicating induction in Brazilian Portuguese (the same would be true for Spanish). The content comprehension of this questions was tested directly, asking women whether they fully understood the question, if they could explain it and if they believe other respondents would fully understand it too; in the 60 meta-interviews of the second pre-test we did not find any negative reaction to such wording. Given these results, our recommendation for other researchers is to give priority to a culturally sensitive framework for wording the questions over technical jargon that could eventually have more precise content.

The conventional way of evaluating the reliability of questions is the test-retest procedure. A second option is the alternative forms approach. None of these can be applied in a ballot-box survey without compromising the research ethically or methodologically. Previously asking women for their consent to apply and re-apply the ballot-box questionnaires in order to compare the results violates the core idea of the technique; asking their consent to review anonymity after the first interview, is ethically controversial and subject to the instrumentation threat to validity. On the other hand, asking the same questions in an alternative form in a five-question questionnaire is pointless because it is likely that respondents will identify these forms and try to make their answers consistent.

An alternative for examining the reliability of the questions is a split-halves technique or one of its equivalents. Of course this technique has two disadvantages: it makes group level comparisons instead of individual comparisons and may require somewhat large samples, increasing the costs of testing. But if testing costs become too high and researchers are willing to take the risks, it can be applied ex post. After the survey is fielded, its sample can be randomly divided in two subsamples and the answers of the partitions compared using chi-square or Pearson correlation tests. This was done in the Brazilian PNA, with very favourable results. The reliability of all questions would be corroborated at a significance level of 0.45, except for a question about the age at the last abortion, which would still be reliable at the significance level of 0.05.

Summary

In short, from our experience with the Brazilian PNA we recommend:

. A core, short ballot-box questionnaire for the abortion questions combined with a longer, face-to-face interview for social and economic data, both matched by an encrypted code.

. A good graphical design, more than written instructions, to guide the respondent across the ballot-box questionnaire.

. Culturally sensitive language with simple answer alternatives in the ballot-box questionnaire.

. Mixed methods (quali and quantitative) pre-tests with meta-interviews and validation by a multidisciplinary panel of specialists.

Statements

Statement of Sources and Selection Criteria

The PNA abortion survey was funded by the Brazilian Ministry of Health. The funder had no participation in the study design or in the collection, analysis, and interpretation of data end the writing of the article and the decision to submit it for publication. We have no competing interests to declare. Participants were randomly chosen by a household sample. They were informed about the survey and voluntarily consented to participate. The research was approved by the Research Ethics Committee of the University of Brasilia.

Artigo apresentado em 13/01/2012

Aprovado em 10/12/2012

Versão final apresentada em 13/03/2012

Authorship

The paper is based on the authors´ experience as designers and coordinators of the PNA and its pre-testing phases. We both have previous experience designing and conducting surveys and qualitative interviews. The authors shared similar responsibilities and work in producing the paper. M Medeiros is responsible for the statistical tests mentioned in the paper. He accepts the responsibilities as its guarantor. D Diniz coordinates a qualitative extension of the PNA in a new sample and this contributed to define what to highlight in this paper.

• 1. Benson L. Studies in secret-ballot technique. Public Opinion Quarterly 1941; 5(1):79-82.
• 2. Olinto M, Moreira Filho DC. Estimativa de aborto induzido: comparação entre duas metodologias [Estimating the frequency of induced abortion: a comparison of two methods]. Rev. Panamericana de Salud Publica 2004; 15(5):331-336.
• 3. Lara D, Garcia SG, Ellertson C, Camlin C, Suarez J. The measure of induced abortion levels in Mexico using random response technique. Sociological Methods and Research 2006; 35(2):79-301.
• 4. Zamudio L, Rubiano N, Wartenberg L. The incidence and social and demographic characteristics of abortion in Colombia. In: Mundigo AI, Indriso C, editors. Abortion in the developing world London, New York: Zen Books; 1999. p. 149-172
• 5. Lara D, Strickler J, Olavarrieta C, Ellertson C. Measuring induced abortion in Mexico: a comparison of four methodologies. Sociological Methods and Research 2004; 32(4):529-558.
• 6. Catania JA, Binson D, Canchola J, Pollack LM, Hauck W, Coates TJ. Effects of interviewer gender, interviewer choice, and item wording on responses to questions concerning sexual behavior. Public Opinion Quarterly 1996; 60(3):345.
• 7. Plummer ML, Wight D, Ross DA, Balira R, Anemona A, Todd J, Salamba Z, Obasi AI, Grosskurth H, Changalunga J, Hayes RJ. Asking semi-literate adolescents about sexual behaviour: the validity of assisted self-completion questionnaire (ASCQ) data in rural Tanzania. Trop Med Int Health 2004; 9(6):737-754.
• 8. Diniz D, Corrêa M, Squinca F, Braga KS. Aborto: 20 anos de pesquisas no Brasil [Abortion: 20 years of Brazilian research]. Cad Saude Publica 2009; 25(4): 939-942.

Recommendations for abortion surveys using the ballot-box technique

Recomendações para inquéritos sobre aborto usando a técnica de urna

Publication Dates

• Publication in this collection
  01 Aug 2012
• Date of issue
  July 2012

History

• Received
  13 Jan 2012
• Accepted
  13 Mar 2012
• Reviewed
  10 Dec 2012