Reliability of data from a very low birth weight population in the Live Birth Information System 2005-2006

Abstracts

O objetivo do presente estudo foi avaliar a completitude e a concordância entre informações do Sistema de Informação sobre Nascidos Vivos (SINASC) e registros hospitalares em uma situação de elevado risco neonatal. Foi realizado o relacionamento probabilístico entre dados de uma Unidade de Cuidados Intensivos Neonatal pública e do SINASC (2005-2006), o que tornou possível analisar os dados de 170 nascidos vivos, com muito baixo peso ao nascer (500g - 1.499g) presentes em ambas as bases. As variáveis analisadas foram: idade materna, número de consultas de pré-natal, tipo de parto, sexo, peso ao nascer, Apgar 1º e 5º minutos e idade gestacional. A completitude no SINASC variou de 91,8% (Apgar 1º minuto) a 100% (sexo, parto e idade materna). Para avaliação da concordância foram utilizados o coeficiente kappa para variáveis dicotômicas, o coeficiente kappa ponderado para variáveis ordinais, e o coeficiente de correlação intraclasse (ICC) e a abordagem gráfica de Bland-Altman para variáveis contínuas. A concordância foi considerada boa a excelente para Apgar 1º minuto (kappa = 0,98), tipo de parto (kappa = 0,96), idade materna (ICC 0,95), Apgar 5º minuto (kappa = 0,93), sexo (kappa = 0,92) e consultas de pré-natal (kappa = 0,76); mas apenas razoável para idade gestacional (kappa = 0,50) e peso ao nascer (ICC = 0,57). Embora o peso ao nascer possa ser mais susceptível a erros de registro em uma população de risco neonatal elevado, a elevada confiabilidade observada na maioria das variáveis analisadas corrobora a importância do SINASC como fonte de informações para estudos de epidemiologia perinatal, mesmo para situações de elevado risco neonatal.

Sistemas de informação; Estatísticas vitais; Controle de qualidade; Confiabilidade; Declaração de nascido; Muito baixo peso ao nascer

ORIGINAL ARTICLE

Reliability of data from a very low birth weight population in the Live Birth Information System 2005-2006

Patrícia Viana Guimarães^I; Claudia Medina Coeli^I; Regina Coeli Azeredo Cardoso^II; Roberto de Andrade Medronho^I; Sandra Costa Fonseca^III; Rejane Sobrino Pinheiro^I

^ICollective Health Studies Institute of the Federal University of Rio de Janeiro - UFRJ

^IIBonsucesso Federal Hospital - Ministry of Health

^IIICommunity Health Institute of the Federal Fluminense University

Correspondence to

ABSTRACT

The objective of the present study was to evaluate the completeness and agreement between data obtained from the Live Birth Information System (SINASC) and hospital records for high neonatal risk situations. Using RecLink III software, a probabilistic data linkage was carried out using databases from a Public Health Neonatal Intensive Care Unit and SINASC (years 2005-2006), which made possible the analysis of data from 170 live births with very low birth weight (between 500g and 1,499g), present at both databases. Variables evaluated were: maternal age, number of antenatal care visits, delivery type, sex, birth weight, Apgar score at 1^st and 5^th minutes and gestational age. Completeness in SINASC varied from 91.8% (1^st minute Apgar Score) to 100% (variables sex, delivery type and maternal age). To evaluate agreement, kappa coefficient was used for dichotomous variables, weighted kappa was used for ordinal variables and intraclass correlation coefficient (ICC) and Bland-Altman graphic approach were used for continuous variables. Agreement was considered good to excellent for 1^ST minute Apgar score (weighted kappa = 0.98), delivery type (kappa = 0.96), maternal age (ICC = 0.95), 5^TH minute Apgar Score (weighted kappa = 0.93), sex (kappa = 0.92) and antenatal care visits (weighted kappa = 0.76), but only fair for gestational age (weighted kappa = 0.50) and birth weight (ICC = 0.57). Although birth weight might be more prone to registry errors in high risk neonatal populations, the elevated reliability observed for most variables analyzed corroborates SINASC importance as a source of information for perinatal epidemiology studies, even when dealing with high risk neonatal situations.

Keywords: Information systems. Vital statistics. Quality control. Reliability. Birth certificates. Very low birth weight.

Introduction

Since its introduction in 1990, the Live Birth Information System (SINASC) has increasingly improved the coverage of events and its information has been widely used to obtain health indicators, epidemiological studies and health surveillance activities^1,2. The potential for using this data source has stimulated the development of studies on the system aimed at reporting the experience linked to its implementation^3,4, assessment of its coverage^5,6 and of the completeness and reliability of data registered^1,2,7-9.

The Live Birth Certificate (DN), the original document that feeds the system, can be filled out by a physician, a member of the delivery room or nursery nursing team, or by previously trained administrative personnel¹⁰, which may generate heterogeneity and inconsistency in the completion.

Although the results of studies on the quality of SINASC indicate good reliability of the data registered on variables such as mother's age, type of delivery, sex of newborn and birth weight^7,8, inconsistencies were appointed in the completion of variables such as gestational age⁷; parturition, schooling and mother's occupation^1,8,9; and congenital malformations ^11,12.

Another condition that could generate inconsistency in the completion of the DN would be the presence of clinical events in the delivery room, whether maternal or neonatal. Our assumption is that in neonatal risk situations, attention is oriented toward the resolution of clinical problems, which could lead to less care in filling out the DN. The quality of SINASC data have been evaluated globally, but we have found very few studies on its quality specifically approaching high neonatal risk situations^1,11.

The present article aims to evaluate the reliability of the information from SINASC, regarding a cohort of very low birth weight (between 500g and 1,499g) newborns and that needed to be admitted into an Neonatal Intensive Care Unit (ICU). Although the percentage of newborns in this weight range is low in Brazil, it is a high risk population for morbidity and mortality. According to the DATASUS¹³, in 2008, babies with weight under 1,500g in the State of Rio de Janeiro were equivalent to 1.5% of total newborns. However, this subgroup accounted for 40.8% of infant deaths in the same state, in 2008. The care in filling out SINASC in this population is important, because the information is vital for the analysis of mother child health status, mainly of infant mortality.

Methods

Study design and data sources

A reliability study was performed by comparing the information registered on a database of 170 newborns with birthweight between 500g and 1,499g, with the exception of twins, admitted to a public Neonatal ICU, in the city of Rio de Janeiro, from January 2005 to December 2006, to information registered on SINASC. Toward that end, the two databases were integrated using a probabilistic technique¹⁴, with the help of the RecLink III computer program¹⁵. The probabilistic technique was used because data linkage was originally done with the SINASC database and a database with 823 records of newborns in the hospital where the Neonatal ICU is, from January 2005 to December 2006, as part of a study aimed at evaluating infant mortality in this population¹⁶. The records of the 170 newborns in intensive care were extracted from the linkage of the original data.

Although twin newborns are a high risk neonatal population, they were excluded during the data linkage process because they share maternal information and date of birth¹⁷ and, if they were the same gender, the only different variables would be birth weight and Apgar score, in addition to time of birth, which is not on the Neonatal ICU record.

For record linkage, a 13 step blockage strategy was used. The key fields used for blockage were: soundex codes of the mother's first and last name, code of the establishment, year of the child's birth, mother's first and last names. The fields used for comparison were mother's complete name, date of the child's birth, code of the city of residence and gender. The field "mother's name" was compared using an algorithm based on the Levenshtein distance¹⁸, while the date fields were compared using the algorithm for the difference of characters. Linkage parameters were estimated on the first blockage step, and parameter estimates were calculated using EM (Expectation Maximization) algorithm based routines¹⁹. The scores of links formed varied from 10.7 to -7.7. At each step, all links formed were manually revised, using rules defined a priori to attribute the status of the pair (true or false). The following fields were used in the manual revision: mother's complete name, child's birth date, birth weight, gender, address, code of the residence neighborhood, code of the establishment. More details on the process used in data linkage are described in Cardoso (2010)¹⁶.

The identified databases from SINASC were obtained at the Rio de Janeiro Health and Civil Defense State Department, after approval of the study by the Ethics in Research Committee of the Federal Bonsucesso Hospital (Protocol 39/09).

Data analysis

The variables analyzed in both databases were: mother's age; number of antenatal visits; type of delivery (vaginal or cesarean); birth weight; gestational age; gender; 1st minute Apgar score; 5th minute Apgar score. The following Neonatal ICU variables were classified to enable comparison with SINASC: gestational age (less than 22 weeks, 22 to 27 weeks, 28 to 31 weeks, 32 to 36 weeks, and 37 to 41 weeks) and number of antenatal visits (none, 1 to 3 visits, 4 to 6 visits and 7 or more visits).

Initially, the completeness of the above mentioned variables was evaluated in both databases, and the variables mother's schooling, number of live born children, number of dead born children and race of newborn only in the SINASC database. Using the score system proposed by Romero and Cunha⁹ as the base, the completeness of study variables was classified as excellent (less than 5% of records not filled out or unknown), good (between 5% and 9%), fair (between 10% and 19%), bad (between 20% and 50%) and very bad (over 50%).

In order to evaluate the level of agreement, all blank records or classified as "unknown" in both databases were excluded. As a measurement of agreement of variables, the kappa coefficient and the percentage of general agreement were used for type of delivery and gender²⁰. The weighted kappa coefficient with quadratic weights²⁰ was used for the ordinal categorical variables antenatal appointments, gestational age and 1º and 5º minute Apgar scores.

Kappa coefficient values were classified according to the revision of the Landis and Koch classification proposed by Shrout²¹: values between 0.81-1.00 (substantial); 0.61 to 0.80 (moderate); 0.41 to 0.60 (reasonable); 0.11 to 0.40 (superficial); and 0.00 to 0.10 (virtually none). The symmetry in the contingency tables was assessed using the McNemar (binary variables) and Bowker (variables with more than two categories) tests²².

For analysis of agreement of the numeric variables mother's age and birth weight, the intraclass correlation coefficient (ICC) was used, using the two-way mixed model, in addition to the Bland-Altman graph approach²³ for birth weight. Statistical analysis was performed with the help of the Stata version 9.0 computer program²⁴ and of the WINPEPI²⁵ statistical pack.

Results

According to the information from the Neonatal ICU database, the median birth weight of the population studied was 930g (Q1 = 772.5; Q3 = 1,145g), and 50.6% of newborns were female. Mothers' age ranged between 13 and 45 years and 53.5% of deliveries were vaginal.

Both databases had a high level of completeness, above 90% for all variables, except for the number of antenatal visits in the hospital database (88.8% completeness). (Table 1)

The kappa coefficient ranged from 0.5 for gestational age to 0.98 for the 1st minute Apgar variable (Table 2). The level of agreement between both databases for mother's age was also high (ICC 0.95, 95% CI 0.93; 0.96) (Table 3). The null hypothesis for symmetry of contingency tables was rejected only for the gestational age variable.

The intraclass correlation coefficient for birth weight was 0.57 (95%CI 0.46; 0.66). There were differences between birth weight on SINASC and on the Neonatal ICU database in 33 records (19.6%), with a difference ranging between 200g and 1,900g, in that the weight on SINASC was higher for 24 records (70.6%). Among the latter, nine weights were ≥ 1,500g in SINASC. Among the 33 records with weights discrepancies on both data sources, there were eight records with differences equal to or over 800g, and the difference in weight in the remaining 25 records was equal to or less than 200g. When we excluded these eight records from the analysis, the level of agreement observed for birth weight went to 0.99 (95% CI 0.990; 0.995).

Figure 1 shows the Bland-Altman graph for weight for all newborns. The limits of agreement were - 577.771 to 716.974, indicating that the range that included 95% of differences was quite wide, in that the mean of differences was 69.601 (95%CI 20.298; 118.905). The null hypothesis of equality of variances was rejected by the Pitman test (r = 0.543; p-valor < 0.001). The Bland-Altman approach, after discarding eight records with discrepancies for birth weight over 800g (Figure 2), showed that the range that includes 95% of weight differences is quite narrow (agreement limits of - 62.333 to 64.833), with a reduction in the mean of differences to 1.250 (95% CI - 3.714; 6.214); so the Pitman test did not reject the hypothesis of equality of variances (r = 0.039, p-valor = 0.621).

Discussion

In the present study we proposed to evaluate the reliability of the data from SINASC in a specific situation, in which data collected referred to a population of a potentially high clinical risk: very low birth weight newborns (< 1,500g) that needed admission to the Intensive Care Unit.

There was a higher frequency of female newborns, unlike the male-female ratio at birth in Brazil, which is 105%²⁶. Such fact may have occurred due to the few number of cases studied and also because it is a population that only includes very low weight newborns, and girls' birth weights are usually lower than that of boys²⁷.

Despite adverse delivery conditions, the completeness of SINASC ranged from good to excellent (above 90%) for all variables analyzed. Romero and Cunha⁹ also found good to excellent completeness for the variables: mother's age, schooling, antenatal visits, newborn's gender and birth weight when SINASC records from all States of the Federation (UF) were analyzed, in 2002. The completeness of the variables race, number of live born children and number of dead born children observed in the SINASC records of patients in the study was classified as excellent, unlike other studies^1,7,9, in which the completeness of these variables was between fair and very bad. The presence of malformations is another potentially high risk situation at delivery. When Guerra et al.¹¹ assessed information of DN with congenital defects registered in the city of Rio de Janeiro in 2004, they found excellent completeness for the variables type of delivery and 1st and 5th minutes Apgar and good for mother's age and gestational age, but only fair for birth weight and bad for number of antenatal visits. In a study to evaluate the quality of DN information involving early neonatal deaths (cases) and survivors (controls), Almeida et al.¹ found a significantly higher completeness in data assessed in cases in comparison to controls. However, they did not find information for birth weight in five of the 146 (3.4%) cases of early neonatal deaths, in comparison to complete records for controls.

In our evaluation, the completeness of SINASC was higher than that of hospital records for variables mother's age and number of antenatal visits. This fact can highlight that when the delivery care team fills out the newborn's record, less importance is given to information on the mother. However, as we know that maternal characteristics are intrinsically related to neonatal outcomes, all professionals should be aware of how essential it is to document appropriately all information available, not only on the DN, but also in the patient record.

According to the parameters that we used to classify reliability²¹, we can consider that the level of agreement of categorical variables was substantial to type of delivery, gender, 1st minute Apgar and 5th minute Apgar; moderate for antenatal visits and only fair for gestational age and birth weight.

In other studies that assessed the quality of information on SINASC and that included the same variables we used^{7, 8}, gestational age was also a variable that presented less agreement. In the newborn cohort evaluated, a possible explanation is the way gestational age is estimated. There was a high frequency of non-existing or absent information on the date of last period (DUM) or on the ultrasound in the first quarter of pregnancy in the newborns' records. This led to the standardization at the study hospital's Neonatal ICU, of the estimate of gestational age using the New Ballard method²⁸ in all hospitalized newborns. Given that in this hospital, who is responsible for completing the DN (professional that cares for newborn in the delivery room) and the physician that examines the newborn in the Neonatal ICU are different individuals, there is considerable likelihood that discordant estimates be found on gestational age, whether due to the method used or due to inaccuracies in the calculation of the New Ballard method itself. Moraes and Reichenheim²⁹ found a kappa coefficient of 0.74 (95%C I 0.49-0.99) when assessing inter-observer reliability of the application of the New Ballard score for the diagnosis of prematurity, even with the participation of examiners submitted to strict previous training. Another fact that may generate inaccuracy in recording gestational age is the difficulty to put information within the strata presented in live born certificates. A newborn whose gestational age is estimated by the Ballard method as 27 weeks + 6 days, could be included in stratum "22 to 27 weeks" by some or have the gestational age approached to 28 weeks by others, which would be included in the "28 to 31 weeks" stratum. The heterogeneity of the professional categories responsible for filling out the DN, thought of as a cause of inconsistency by Mishima et al.³⁰ and Guerra et al.¹¹, is not the case of the study hospital, where all DN are filled out by pediatricians.

The difficulty to register more complex situations also was reported in an American study, with cases (newborns with less than 1,500g and deaths) and controls (healthy newborns)³¹, in which a low sensitivity of data from birth certificates in terms of maternal risk factors, abnormal new born conditions and malformations was observed.

In the present study, despite the disagreement between the gestational age recorded in the hospital database and that of SINASC, all newborns were classified as premature (birth before 37 weeks of pregnancy), an expected fact when working with very low weight newborns. However, for epidemiological studies with SINASC data and that include newborns of all weight ranges, the inaccuracy of the gestational age recorded on the DN may be a potential source of difficulties in the identification of prematurity and of prematurity associated risk factors^7,29.

Regarding birth weight, initially there was not a good correlation (ICC 0.57) between SINASC records and the hospital base, unlike what is reported in the literature^3,7-9. The ICC is a measurement expressed as the ratio between variability of different patients and total variability (variability among different patients added to intra-patient variability)³². As the study population included only patients with birth weight between 500g and 1,499g, the variation among patients became reduced, increasing thus, the impact of variability among records of weight of the same patient. This can be observed comparing the values of ICC before (0.57) and after the exclusion of eight patients, whose difference between weights recorded on both databases was over 800g (ICC 0.99). As intra-patient variability of the remaining records is 200g or less, the total variability falls, increasing ICC. Given weight is an essential information in the care of newborns in the Neonatal ICU and this variable is completed in the database by professionals also involved in care, such errors are not likely to happen (differences above 800g). In fact, when assessing the reason for a discrepancy of over 800g in these eight records, we could confirm, by analyzing patient records, that birth weights registered in the Neonatal ICU database were correct. It was possible to assess maternal records of five of these patients and we observed that the variable birth weight was not filled out in the DN attached to patient records. The patient records of another three mothers were not assessed because we were not able to recover their system entry numbers. Possibly, birth weight was not obtained in the delivery room due to immediate transportation to the Neonatal ICU, inducing to posterior incomplete completion. Based on the information we have, we are not able to explain how the weight values of these patients were filled out in SINASC. Almeida et al.¹, in their case-control study to evaluate the quality of SINASC, observed that the DN under estimates low weight in 1.9% of cases and overestimates it in 8% of controls. These differences related to recording birth weight, although not so numerous, may cause distortions in the assessment of birth weight as a risk factor in perinatal morbidity-mortality studies. If we were to consider SINASC as the only source of information of birth weight, of the 168 patients with weight recorded, nine (5.4%) would be potentially excluded from studies on very low weight.

One limitation of the present study was using only one data source to compare to SINASC, unlike Theme et al.⁸ and Almeida et al.¹, who paired information from hospital records, SINASC and from interviews with mothers. Another limitation was the absence, in the ICU database of other important variables to the mother child health field, like marital status, schooling and mother's parturition, preventing a more comprehensive assessment of SINASC records.

Albeit limitations, the present study approached the completeness and reliability of information from SINASC, regarding a specific population comprised of very low birth weight newborns. We concluded that, even in neonatal high risk situations, results found corroborate the good reliability of SINASC records for most variables analyzed. The exceptions were gestational age, a fact already highlighted by other studies, and birth weight, a variable that may be more susceptible to the absence or completion errors related to severity and to the need for intensive care presented by these newborns. Given the number of newborns with very low weight represents a very small percentage among total live births, maybe the unsatisfactory quality of its completion does not have a sufficient weight to influence the agreement in studies that assess the variable birth weight in SINASC with records of all newborns, regardless of weight or clinical severity.

The trend observed in SINASC of registering a higher birth weight than the actual weight, may lead to an inappropriate classification of some patients and impact negatively on the validity of studies that aim to analyze survival in a very low weight population using SINASC as the data source.

Based on the low level of agreement between birth weight found in the present study and the observation of five DN without the value completed, we suggest that more studies be performed to assess the reliability of SINASC in low birth weight settings. However, despite the discrepancies observed, we consider that birth weight can be a more reliable variable than estimated gestational age for studies to assess neonatal outcomes using SINASC data.

Our results, using very low birth weight as an indicator of high neonatal risk, can be valid for other clinical settings that demand more attention from health professionals, such as gemelar pregnancies, perinatal asphyxia (for example, meconium aspiration and premature placenta detachment) and presence of malformations. However, due to the specificities of each one of these conditions, we suggest performing additional specific studies, to better assess the reliability of SINASC in other high neonatal risk situations.

References

• 1. Almeida MF, Alencar GP, Franca Junior I, Novaes HMD, Siqueira AAF, Schoeps D et al. Validade das informações das declarações de nascidos vivos com base em estudo de caso-controle. Cad Saúde Pública 2006; 22(3): 643-52.
• 2. Costa JMBS, Frias PG. Avaliação da completitude das variáveis da Declaração de Nascido Vivo de residentes em Pernambuco, Brasil, 1996 a 2005. Cad Saúde Pública 2009; 25(3): 613-24.
• 3. Mello Jorge MHP, Gotlieb SLD, Soboll MLM, Almeida MF, Latorre MRDO. Avaliação do Sistema de Informação sobre Nascidos Vivos e o uso de seus dados em epidemiologia e estatísticas de saúde. Rev Saúde Pública 1993; 27(S6): 1-44.
• 4. Mello Jorge MHP, Laurenti R, Gotilieb SLD. Análise da qualidade das estatísticas vitais brasileiras: a experiência de implantação do SIM e do SINASC. Cienc Saúde Coletiva 2007; 12(3): 643-54.
• 5. Frias PG, Pereira PMH, Vidal AS, Lira PIC. Avaliação da cobertura do Sistema de Informação sobre Nascidos Vivos e a contribuição das fontes potenciais de notificação do nascimento em dois municípios de Pernambuco, Brasil. Epidemiol Serv Saúde 2007; 16(2): 93-101.
• 6. Drumond EF, Machado CJ, Elisabeth F. Subnotificação de nascidos vivos: procedimentos de mensuração a partir do Sistema de Informação Hospitalar. Rev Saúde Pública 2008; 42(1): 55-63.
• 7. Silva AAM, Ribeiro VS, Borba Jr AF, Coimbra LC, Silva RA. Avaliação da qualidade dos dados do Sistema de Informações sobre Nascidos Vivos em 1997-1998. Rev Saúde Pública 2001; 35(6): 508-14.
• 8. Theme Filha MM, Gama SGN, Cunha CB, Leal MC. Confiabilidade do Sistema de Informações sobre Nascidos vivos Hospitalares no município do Rio de Janeiro, 1999-2001. Cad Saúde Pública 2004; 20(S1): 83-91.
• 9. Romero DE, Cunha CB. Avaliação da qualidade das variáveis epidemiológicas e demográficas do Sistema de Informações sobre Nascidos Vivos, 2002. Cad Saúde Pública 2007; 23(3): 701-14.
• ¹⁰
  Ministério da Saúde. Manual de instruções para o preenchimento da declaração de nascido vivo. (3ª ed.) Brasília: Ministério da Saúde: Fundação Nacional de Saúde; 2001. 32 p.
• 11. Guerra FA, Llerena Jr JC, Gama SG, Cunha CB, Theme Filha MM. Reliability of birth defect data on birth certificates of Rio de Janeiro, Brazil, 2004. Cad Saude Publica 2008; 24(2): 438-46.
• 12. Luquetti DV, Koifman RJ. Quality of reporting on birth defects in birth certificates: case study from a Brazilian reference hospital. Cad Saúde Pública 2009; 25(8): 1721-31.
• 13. Ministério da Saúde, 2008, Departamento de Informática do SUS - DATASUS. http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvrj.def [Acessado em 23 de janeiro de 2011]
• 14. Camargo Júnior KR, Coeli CM. RECLINK: Aplicativo para o relacionamento de banco de dados implementando o método Probabilistic record linkage Cad Saúde Pública 2000; 16(2): 439-47.
• 15. Camargo Júnior KR, Coeli CM. RecLink 3: nova versão do programa que implementa a técnica de associação probabilística de registros (probabilistic record linkage). Cad Saúde Coletiva 2006; 14: 399-404.
• 16. Cardoso RCA. Mortalidade em um ano de recém-nascidos de muito baixo peso assistidos por uma Unidade Neonatal no Rio de Janeiro [Dissertação de Mestrado]. Rio de Janeiro: Instituto de Estudos em Saúde Coletiva da Universidade Federal do Rio de Janeiro; 2010.
• 17. Méray N, Reistma JB, Ravelli ACJ, Gouke JB. Probabilistic record linkage is a valid and transparent tool to combine databases without a patient identification number. J Clin Epidemiol 2007; 60: 883-91.
• 18. Levenshtein VI. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady 1996; 10: 707-10.
• 19. Junger WL. Estimação de parâmetros de relacionamento probabilístico de bancos de dados: uma aplicação do algoritmo EM para o RECLINK. Cad Saúde Coletiva 2006; 14(2): 225-32.
• 20. Szklo M, Javier Nieto F. Epidemiology: beyond the basics. Sudbury, Massachusetts: Jones and Bartlett Publishers; 2007. p. 313-43.
• 21. Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res 1998; 7(3): 301-17.
• 22. Bowker AH. A test for symmetry in contingency tables. J Am Stat Assoc 1948; 43: 572-4.
• 23. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurements. Lancet 1986; 1(8476): 307-10.
• ²⁴
  STATA Statistical Software: Release 9.0. Stata Corporation. Texas: College Station, USA; 2006.
• 25. Abramson JH. WINPEPI (PEPI-for-Windows): computer programs for epidemiologists. Epidemiol Perspect Innov 2004; 1: 6.
• 26. Ministério da Saúde. Rede Interagencial de Informações para a Saúde. Indicadores de Dados Básicos para a Saúde no Brasil (IDB 2007) Brasília: MS; 2007. Disponível em: www.ripsa.org.br/lildbi/docsonline/get.php?id=328 [Acessado em 07de dezembro de 2011]
• 27. Weinberg CR, Wilcox AJ. Methodological issues in reproductive epidemiology. In: Rothman KJ, Greenland S, Lash TL (eds.). Modern Epidemiology (3^rd ed.). Philadelphia: Lippincott Williams & Wilkins; 2008. p. 620-40.
• 28. Ballard JL, Khoury JC, Wedig K, Wang L, Eilers-Walsman BL, Lipp R. New Ballard Score expanded to include extremely premature infants. J Pediatr 1991; 119(3): 417-23.
• 29. Moraes CL, Reichenheim ME. Validade do exame clínico do recém-nascido para a estimação da idade gestacional: uma comparação do escore New Ballard com a data da última menstruação e ultra-sonografia. Cad Saúde Publica 2000; 16(1): 83-94.
• 30. Mishima FC, Scochi CGS, Ferro MAR, Lima RAG, Costa IAR. Declaração de nascido vivo: análise do seu preenchimento no Município de Ribeirão Preto, São Paulo, Brasil. Cad Saúde Pública 1999: 15(2): 387-95.
• 31. Piper JM, Mitchell Jr EF, Snowden M, Hall C, Adams M, Taylor P. Validation of 1989 Tenessee birth certificates using maternal and newborn hospital records. Am J Epidemiol 1993; 137(7): 758-68.
• 32. Luiz RR. Métodos estatísticos em estudos de concordância. In: Medronho RA, Bloch KV, Luiz RR, Werneck GL (eds.). Epidemiologia (2Ş ed). São Paulo: Editora Atheneu; 2009.

Publication Dates

• Publication in this collection
  20 Feb 2013
• Date of issue
  Dec 2012

History

• Received
  12 June 2011
• Accepted
  09 Apr 2012
• Reviewed
  30 Dec 2011