Acessibilidade / Reportar erro

Double sampling control chart for a first order autoregressive process

Abstracts

In this paper we propose the Double Sampling control chart for monitoring processes in which the observations follow a first order autoregressive model. We consider sampling intervals that are sufficiently long to meet the rational subgroup concept. The Double Sampling chart is substantially more efficient than the Shewhart chart and the Variable Sample Size chart. To study the properties of these charts we derived closed-form expressions for the average run length (ARL) taking into account the within-subgroup correlation. Numerical results show that this correlation has a significant impact on the chart properties.

autocorrelation; double sampling; statistical process control


Neste artigo propomos o gráfico de controle de amostragem dupla para monitoramento de processos nos quais as observações seguem um modelo autoregressivo de primeira ordem. Nós consideramos intervalos de amostragem suficientemente longos em linha com o conceito de subgrupos racionais. O gráfico de controle de amostragem dupla é substancialmente mais eficiente que o Gráfico de Shewhart e do que o Gráfico com Amostra de Tamanho Variável. Para estudar as propriedades destes gráficos nós derivamos expressões de forma-fechada para o Numero Médio de Amostras até o Sinal (NMA) levando em conta a correlação dentro do subgrupo. Os resultados numéricos mostram que esta correlação tem impacto significante sobre as propriedades do gráfico.

autocorrelação; amostragem dupla; controle estatístico do processo


Double sampling control chart for a first order autoregressive process

Fernando A. E. Claro* * Corresponding author / autor para quem as correspondências devem ser encaminhadas ; Antonio F. B. Costa; Marcela A. G. Machado

Production Department, São Paulo State University (UNESP), Guaratinguetá - SP - Brazil, fernandoclaro@uol.com.br; fbranco@feg.unesp.br; marcela@feg.unesp.br

ABSTRACT

In this paper we propose the Double Sampling control chart for monitoring processes in which the observations follow a first order autoregressive model. We consider sampling intervals that are sufficiently long to meet the rational subgroup concept. The Double Sampling chart is substantially more efficient than the Shewhart chart and the Variable Sample Size chart. To study the properties of these charts we derived closed-form expressions for the average run length (ARL) taking into account the within-subgroup correlation. Numerical results show that this correlation has a significant impact on the chart properties.

Keywords: autocorrelation; double sampling; statistical process control.

RESUMO

Neste artigo propomos o gráfico de controle de amostragem dupla para monitoramento de processos nos quais as observações seguem um modelo autoregressivo de primeira ordem. Nós consideramos intervalos de amostragem suficientemente longos em linha com o conceito de subgrupos racionais. O gráfico de controle de amostragem dupla é substancialmente mais eficiente que o Gráfico de Shewhart e do que o Gráfico com Amostra de Tamanho Variável. Para estudar as propriedades destes gráficos nós derivamos expressões de forma-fechada para o Numero Médio de Amostras até o Sinal (NMA) levando em conta a correlação dentro do subgrupo. Os resultados numéricos mostram que esta correlação tem impacto significante sobre as propriedades do gráfico.

Palavras-chave: autocorrelação; amostragem dupla; controle estatístico do processo.

1. Introduction

A Shewhart control chart is a statistical tool applied to data from a process to determine if a quality characteristic has shifted from its target setting (Montgomery, 2001). This form of process monitoring is widely used in industry to distinguish common causes from special causes of variation (often responsible for process shifts) and also to identify the time of a process change in order to understand its main cause and to improve the process by preventing its future occurrence. Despite being a powerful problem-solving and process improvement tool, of inherently simple application, one of the major drawbacks of the Shewhart chart is its slowness to detect small shifts in the process mean. For this reason, the utilization of adaptive or dynamic control charts has progressively grown in the last two decades contributing to remarkably improve the effectiveness of process monitoring.

An adaptive control chart has at least one of its parameters - sampling interval, sample size or control limits coefficient - varying in real time based on the actual values of the sample statistics which provide current information about the status of the process (Tagaras, 1998). Considerable efforts have already been undertaken in the research of the statistical properties of the adaptive Shewhart charts with Variable Sampling Interval (VSI) (Reynolds, 1996; Runger & Montgomery, 1993); with Variable Sample Size (VSS) (Prabhu, Runger, Keats & Croasdale, 1993; Costa, 1994; Costa & Rahim, 2004), and with the Variable Sampling Interval and Variable Sample Size (VSSI) (Costa, 1997, 1998, 1998a, 1999 a/b/c; Carot, Jabaloyes & Carot, 2002). Another alternative used to improve the performance of the traditional Shewhart charts is the Double Sampling (DS) procedure (Croasdale, 1974; Daudin, 1992). He & Grigoryan (2005) and Champ & Aparisi (2006) recently extended the DS procedure for multivariate processes.

The DS chart offers faster process shift detection than the standard Shewhart chart, at the expense of a larger administrative burden. The DS chart also outperforms the VSI chart and performs better than the static Exponentially Weighted Moving-Average (EWMA) and Cumulative-Sum (CUSUM) control charts for large shifts; however it is not as effective in detecting small shifts.

The Average Run Length (ARL) is the usual metric to measure the performance of a control chart with fixed sampling intervals. The ARL is the expected number of samples required by the chart to signal. When there is a change in the process, it is desirable to have a low ARL so that the change will be detected quickly; when the process is in control, it is desirable to have a large ARL so that the rate of false alarms produced by the chart is low (Lu & Reynolds, 1999).

The underlying assumption in Statistical Process Control (SPC) is that the observations from the process are independent and identically distributed (i.i.d.). However, there are many practical situations in which the process data are serially correlated.

Various charting techniques to deal with autocorrelated univariate process have been proposed. In many applications the dynamics of the process induces correlation in observations closely spaced in time. If the interval between observations is short enough to produce correlation, then, one simple approach is to skip some of them, with the consequent disadvantage of disregarding the concept of rational subgroup. A second alternative would be to widen the control limits as a remedial method to deal with the autocorrelated data. Vasilopoulos & Stamboulis (1978) formulated control charts with the control limits coefficients tailored for a first or second order autoregressive model processes. A more typical approach is to fit an appropriate time-series model to the observations and charting the i.i.d. forecast errors or residuals (Alwan & Roberts, 1988; Montgomery & Mastrangelo, 1991). However, additionally to the fact that forecasts "recover" from abrupt changes and thus leave only a limited "window of opportunity" for detection (Vander Wiel, 1996), this approach is not well-suited for detecting small shifts in processes with mild positive autocorrelation. Moreover, users may have some difficulty to interpret the residuals control charts and besides, fitting and maintaining an appropriate time-series model for each required process variable can be cumbersome (Faltin, Mastrangelo, Runger & Ryan, 1997).

Thus there will be many applications for which it will be desirable, from practical considerations, to apply a standard control chart to the original correlated observations. In this case, it is important not only to adjust the chart parameters to obtain the required protection against false alarms, but also to assess the effect of the correlation on the chart's performance (Bisgaard & Kulahci, 2005). It is well-known that when the observations are independent the chart with variable sampling interval detects process shifts faster than the chart with fixed sampling interval. Reynolds, Arnold & Baik (1996) worked with autocorrelated observations and obtained the same conclusion. In this paper, we study the efficiency of the double sampling chart when the observations follow a first order autoregressive model and the samples are formed according to the rational subgroup concept.

2. The process model

The primary purpose of this paper is to address the monitoring of autocorrelated processes with the observations collected in rational subgroups, as it is often the case in the application of SPC, particularly in process industries. The purpose of a sampling strategy based on rational subgroups is to minimize the chance of variability due to assignable causes within a sample and to maximize the chance of variability between samples if assignable causes are present (Montgomery, 2001). As the process autocorrelation attenuates to zero when the time between observations increases; in our model only the within-subgroup autocorrelation needs to be modelled (the correlation between subgroups is practically negligible).

Throughout this paper we assume that the observations of the quality characteristic to be monitored fit to a First Order Autoregressive AR(1) model that has frequently been used in applications (Runger & Willemain, 1996; Montgomery & Mastrangelo, 1991; Wardell, Moskowitz & Plante, 1994; Timmer, Pignatiello Jr & Longnecker, 1998).

The process observations t can be written as:

where µ is the process mean, Φ is the autoregressive coefficient and is the i.i.d. white noise. We additionally consider that the AR(1) model is accurate. The numerical results presented in this paper are for the case of positive autocorrelation, from low to moderately high levels, believed to be prevalent in control charts applications.

3. The Double Sampling (DS) Chart for autocorrelated process

The DS procedure assumes that one master sample (n) is formed by two consecutive subsamples (n=n1+n2) taken without any intervening time, therefore coming from the same probability distribution. The DS procedure follows:

First stage: take a subsample of size n1 and compute the mean 1 . Let where , µ0 is the in-control process mean and is the subsample standard deviation. Alwan & Roberts (1988) give the variance of a sample mean for an AR(1) process and we employ his expression to calculate the standard deviation when the subsample size is n1:

where is the process standard deviation given by:

and . Observe that Ψ=1 when n1=1 or when the process observations are independent.

At this stage there are three possibilities:

• If | Z1 | < L1, conclude that the process mean is in control and the DS procedure finishes at this stage.

• If | Z1 | > L, being L > L1 > 0, conclude that the process mean is off-target and the DS procedure finishes at this stage.

• If L1< | Z1 | < L, the DS procedure goes to the second stage.

Second stage: take a subsample of size n2 and compute the mean 2 . Let where is the master sample mean given by and is its standard deviation.

At this stage there are only two possibilities:

• If | Z2 | < L2, conclude that the process mean is on-target.

• If | Z2 | > L2, conclude that the process is out of control.

Figure 1 is a graphical representation of this procedure.


4. Properties of the DS

control chart

We define the following intervals and unions of intervals in the first stage:

and the interval in the second stage is:

The average sample size per sampling (ASS) is:

where the probability of taking the second subsample is:

where Φ =(•) is the standard Normal distribution function, δ=(µ1 - µ0 X and µ1 is the off-target process mean. When the process is in-control, δ=0.

The efficiency of the control chart is generally measured by the ARL, which is the average number of samples until an out-of-control signal, expressed as:

where P = P1+P2 and Pi (i=1,2) is the probability of deciding that the process is in control at stage i of the sampling procedure. The properties of a DS chart depend on the parameters n1, n2, L, L1 and L2. In a preliminary investigation we found that large values of L leads to the best chart performance, that is, when the false alarm risk during the first sampling stage is practically zero. Daudin (1992) recommends to adopt L=4 or L=5 and following his suggestion we set L=5. In this case, considering expression (4) one has that and from expression (6), when the process is in control, it follows that:

When the process is in control ARL= ARL0.

The expressions (A11) and (A12) with δ = 0 , see the appendixappendix, are used to obtain the values of L1 and L2. According to expression (A11), P1 is an increasing function of L1 so, by grid search, it is always possible to find the value of L1 that matches both sides in the expression (A11) for any given P1. According to expression (A12), P2 is also an increasing function of L2 and again in this case, by grid search, it is always possible to find the value of L2 that matches both sides in the expression (7) for any given values of P1 and ARL0.

The VSS chart and the chart, for the autocorrelated process model under study, are introduced next in order to be compared with the DS procedure.

5. The competing control charts

When the VSS chart is in use, one takes random samples from the process at a fixed unit of time. The process starts in a state of statistical control with mean µ = µ0 and standard deviation σ, and at some random time in the future the mean shifts from µ0 to µ0 ± δσX, where δ 0 . The sample means are plotted on the control chart with warning limits and action limits , where 0 k1

k and is the standard deviation of the sample means. The size of each sample depends on what is observed in the preceding sample. If the value falls in the chart's central region, then the size of the next sample, denoted by n1, should be small. Alternatively, if the point falls in the chart's warning region, then the next sample size, denoted by n2, should be large. The properties of the VSS control chart for the model considered in this study were determined using the expressions given in Costa (1994), however, considering that the standard deviation of the sample means and the standard deviation of the process are given by expressions (2) and (3), respectively.

The design of the chart for autocorrelated processes follows the concepts of the classical methodology applied to independent data, with centreline at µ0 and control limits (CL) set at ; however, taking the autocorrelation into account to determine the sample standard deviation, that is, we employed expression (2) only replacing n1 by . To make a fair comparison between the chart with single sampling and with double sampling, we designed the single sampling scheme with samples of size .

6. The performance of the control charts

The DS chart control limits coefficients L1 and L2 for ={ 3 and 5 } and Φ ={ 0.00, 0.25, 0.50, 0.75 } are in Table 1. The ARL and the probability of taking the second subsample are given in Tables 2 through 4. The ARL for the chart and for the VSS chart are in Tables 3 and 4.

The ARL of all the control charts deteriorates when the correlation is positive, particularly in the range of low mean shifts. For example, for the DS chart, when n1 = 1, n 2 = 8, = 3 and δ=0.50 the ARL increases from 19.8 for Φ = 0 (independent case) to 53.5 for Φ = 0.5 , (see Table 2).

Larger sample sizes contribute to improve the statistical power of all the control charts under study, for all the autocorrelation levels and mean shifts considered. For example, for the variable sample size chart, assuming n1 = 2, n2 = 12 , δ=0.50 and Φ = 0.50 , the ARL is reduced from 83.7 when = 3 to 54.9 when = 5 , (see Tables 3 and 4).

Regardless the magnitude of the process shift, the effect of setting (n1) equal to one or two for the DS chart is practically negligible on the efficiency. As a general rule, larger subsamples in the second stage (n2) contribute to reduce the ARL of the DS chart in the interval 0.25 < δ <1.50 . For example, considering Φ = 0.25 ; n1 = 2, = 3 and δ = 0.50 the ARL varies from 45.6 when n2 = 4 to 23.4 when n2 = 12 , (see Table 3). Regarding the VSS control chart, larger samples in the second stage (n2) improve the efficiency of the control chart depending on the values of Φ, δ and . For example, assuming δ = 0.50 , n1 = 2, = 5 and increasing n2 from 12 to 20 reduces the ARL from 26.9 to 19.8 (26.4% reduction) when Φ = 0.25 ; from 54.9 to 45.6 (16.9% reduction) when Φ = 0.50 and from 100.8 to 95.1 (only 5.7% reduction) when Φ; = 0.75 . The adoption of larger values of n2 for moderately autocorrelated processes (Φ = 0.50 ) improves the control chart efficiency in the interval 0.75 < δ <1.75 for = 3 and in the interval 0.50 < δ <1.25 for = 5.

Assuming a correlation level of Φ = 0.5 , = 5 and 0.50 < δ < 2.00 we compare the ARL of the DS control chart (n1= 2, n2= 20) with the ARL of the Shewhart chart ( = 5 ) and the ARL of the VSS chart (n1= 2, n2= 20), (see Figure 2).


7. A simulation example

The purpose of the following example is to show the effect of the autocorrelation on the ability of the Shewhart and the DS control charts in detecting process location shifts. To this end, we considered the AR(1) model, with Φ = 0.847 and normally distributed i.i.d. residuals with mean zero and σε = 3.867 , presented in the example 1 of Montgomery & Mastrangelo (1991), resulting from the monitoring of a critical parameter in a chemical process.

Assuming initially a process in-control with µ0 = 84.60 and samples of size n=3, taken according to the rational subgroup concept, we plot the mean values on a chart, with the process standard deviation estimated by , where is the sample range mean of 50 samples. The control limits of CL= 84.60±4.95 were determined by (Montgomery, 2001) with =4.84 and d2= 1.693 , ignoring the serial correlation, (see Figure 3). The occurrence of many sample points beyond the control limits (samples 1, 20, 23 to 27, 29, 36, 37, 40, 41, 43, 47 and 48) indicates very tight control limits and high probability of false alarms due to the design of the control chart without recognizing the serial dependency within the subgroups.


Afterwards, using the same data set and with determined employing the expression (2) with n1 replaced by n, new control limits calculated by were set to account for correlation. Additionally, the last 20 samples were simulated again, now with µ1= 1.5 σX . In this case, the control chart signalled for the first time at sample 43 (run length = 13), (see Figure 4).


Finally, we investigated the process monitoring strategy using the DS chart. Subsamples with size n1=2 and n2=4 were adopted and we set = 3, aiming a fair comparison between the DS and the Shewhart control charts. With coefficients L1 = 1.1503 and L2 = 3.0000, the control limits were set at ±8.041 and ±18.824 , for the first and second stages, respectively CL1 and CL2. The action limits of the first stage, with coefficient L=5.00, are not shown on the chart. The values of 1 and are plotted on the chart according to the left hand vertical scale and the right hand vertical scale, respectively. As before, the last 20 master samples were simulated with µ1= 1.5σX . According to the left hand vertical scale, an 1 value beyond the control limit CL1 triggers the inspection of the whole sample. In this case, the value is computed. According to the right hand vertical scale, a signal is given by an value beyond the control limit CL2. The control chart now signals the disturbance at sample 39 (run length = 9), (see Figure 5), faster than in the previous simulation with the chart.


8. Conclusion

Most standard control charting schemes for Statistical Process Control are based on the assumption that measurements of the product quality variable are independent within and between the subgroups. However, positive autocorrelation at low lags is commonplace in chemical processes (Montgomery, 2001), where, given the advances in sensor technologies, observations are closely spaced in time. In the present paper, we assume that successive observations collected from the process fit to a First Order Autoregressive model, and furthermore, that such observations are grouped following the rational subgroup concept (each sample consists of units that were consecutively produced and the subgroups are far spaced in time to maximize the chance of detecting difference between samples, if an assignable cause is present). This sampling strategy results in within-subgroup correlation and significant effects on the performance of control charts if the traditional control chart methodology is applied. Aiming to partially recover the chart efficiency lost due to the autocorrelation, we propose a Double Sampling control chart for autocorrelated processes, as an extension of a previously introduced methodology (Daudin, 1992). We set up all the charts for the same false alarm rate and the same Average Sample Size per sampling when the process mean is on target, we derived suitable control limits and measured the process monitoring efficiency in fair comparison with competing schemes. The major findings from the present research are the following:

• In all cases, the positive correlation has a detrimental effect on the control chart's ability to detect process mean shifts;

• When the correlation within the subgroup ranges from low to moderate, the Average Run Length of the Double Sampling chart is substantially better than the Average Run Length of both, the Shewhart and the Variable Sample Size control charts, regardless the magnitude of the process shift;

• For processes with moderate correlation level, larger values of n2 improve the mean shift detection ability of the Double Sampling scheme.

Acknowledgement

The authors gratefully acknowledge the financial support of this joint work provided by National Council for Scientific and Technological Development (CNPq) and the assistance of three anonymous referees, whose suggestions led to a considerable improvement in the contents and the structure of this paper.

Recebido em 01/2008; aceito em 07/2008 após 1 revisão

Received January 2008; accepted July 2008 after one revision

P1 and P2 expressions

The observations Xt for a first order autoregressive process AR(1), expression (1), are given by + εt, t=1, 2, 3,...,T ; where Φ is the autoregressive coefficient, µ is the process mean and εt is the white noise, i.i.d., εt ~ 0,σε). Without loss of generality, we take µ = µ0 = 0 and σε= 1 when the process is in-control. Based on the DS procedure, a master sample of size n, comprising sequential process observations, can be partitioned in two subsamples of sizes n1 and n2 units with n = n1+n2. Furthermore, we assume that the first master sample is taken at any given random point of time, and the subsequent master samples are taken in fixed time intervals sufficiently long, so that the effect of the autocorrelation between them is practically negligible. The mean of the first subsample n1 is written as:

where:

Given the fixed correlation structure within the subgroup, the expression of the second subsample mean (

2) comprises two terms, Xi,k and Xi,u , to distinguish respectively the known and unknown information ( εt values) after the measurement of the first subsample.

where:

and

where:

The combined (or master) sample mean () is:

thus:

and finally:

where:

with and . The variable introduced at (A7) is given by:

where:

j=1,2,3,........n2

and the variance is:

Based on the DS procedure, one has:

that can be rewritten as:

where:

The probability P2 is:

that can be rewritten as:

where is given by expression (2) replacing n1 by n.

  • (1) Alwan, L.C. & Roberts, H.V. (1988). Time-series modelling for statistical process control. Journal of Business & Economic Statistics, 6(1),87-95.
  • (2) Bisgaard, S. & Kulahci, M. (2005). Quality Quandaries: The effect of autocorrelation on statistical process control procedures. Quality Engineering, 17,481-489.
  • (3) Carot, V.; Jabaloyes, J.M. & Carot, T. (2002). Combined double sampling and variabl sampling interval chart. International Journal of Production Research, 40(9),2175-2186.
  • (4) Champ, C.H. & Aparisi, F. (2008). Double sampling Hotelling's T2 charts. Quality and Reliability Engineering International, 24(2),153-166.
  • (5) Costa, A.F.B. (1994). charts with variable sample size. Journal of Quality Technology, 26(3),155-163.
  • (6) Costa, A.F.B. (1997). charts with variable sample size and sampling intervals. Journal of Quality Technology, 29(2),197-204.
  • (7) Costa, A.F.B. (1998). VSSI charts with sampling at fixed times. Communications in Statistics-Theory and Methods, 27(11),2853-2869.
  • (8) Costa, A.F.B. (1998 a). Joint and R charts with variable parameters. IIE Transactions, 30(6),505-514.
  • (9) Costa, A.F.B. (1999 a). Joint and R charts with variable sample sizes and sampling intervals. Journal of Quality Technology, 31(4),387-397.
  • (10) Costa, A.F.B. (1999 b). charts with variable parameters. Journal of Quality Technology, 31(4),408-416.
  • (11) Costa, A.F.B. (1999 c). AATS for the chart with variable parameters. Journal of Quality Technology, 31(4),455-458.
  • (12) Costa, A.F.B. & Rahim, M.A. (2004). Joint and R charts with two-stage samplings. Quality and Reliability Engineering International, 20,699-708.
  • (13) Croasdale, P. (1974). Control charts for a double sampling scheme based on average production run length. International Journal of Production Research, 12(5),585-592.
  • (14) Daudin, J.J. (1992). Double sampling charts. Journal of Quality Technology, 24(2),78-87.
  • (15) Faltin, F.; Mastrangelo, C.M.; Runger, G.C. & Ryan, T.P. (1997). Considerations in the monitoring of autocorrelated and independent data. Journal of Quality Technology, 29(2),131-133.
  • (16) He, D. & Grigoryan, A. (2005). Multivariate multiple sampling charts. IIE Transactions, 37,509-521.
  • (17) Lu, C.-W. & Reynolds, M.R. Jr. (1999). Control charts for monitoring the mean and variance of autocorrelated processes. Journal of Quality Technology, 31(3),259-274.
  • (18) Montgomery, D.C. (2001). Introduction to Statistical Quality Control (4th ed.), New York, NY, Wiley.
  • (19) Montgomery, D.C. & Mastrangelo, C.M. (1991). Some statistical process control methods for autocorrelated data. Journal of Quality Technology, 23(3),179-193.
  • (20) Prabhu, S.S.; Runger, G.C.; Keats, J.B. & Croasdale, P. (1993). chart with adaptive sample sizes. International Journal of Production Research, 31(12),2895-2909.
  • (21) Reynolds, M.R. Jr. (1996). Shewhart and EWMA variable sampling interval control charts with sampling at fixed times. Journal of Quality Technology, 28(2),199-212.
  • (22) Reynolds, M.R. Jr.; Arnold, J.C. & Baik, J.W. (1996). Variable sampling interval charts in the presence of correlation. Journal of Quality Technology, 28(1),12-30.
  • (23) Runger, C.G. & Montgomery, D.C. (1993). Adaptive sampling enhancements for Shewhart control charts. IIE Transactions, 25(3),41-51.
  • (24) Runger, C.G. & Willemain, T.R. (1996). Batch-means control charts for autocorrelated data. IIE Transactions, 28,483-487.
  • (25) Tagaras, G. (1998). A survey of recent developments in the design of adaptive control charts. Journal of Quality Technology, 30(3),212-231.
  • (26) Timmer, D.H.; Pignatiello Jr., J.J. & Longnecker, M.T. (1998). The development and evaluation of CUSUM-based control charts for an AR(1) process. IIE Transactions; 30,525-534.
  • (27) Vander Wiel, S.A. (1996). Monitoring processes that wander using integrated moving average models. Technometrics, 38(2),139-151.
  • (28) Vasilopoulos, A.V. & Stamboulis, A.P. (1978). Modification of control charts limits in the presence of data correlation. Journal of Quality Technology, 10(1),20-30.
  • (29) Wardell, D.G.; Moscowitz, H. & Plante, R.D. (1994). Run-length distributions of special-cause control charts for correlated processes. Technometrics, 36(1),3-17.

appendix

  • *
    Corresponding author / autor para quem as correspondências devem ser encaminhadas
  • Publication Dates

    • Publication in this collection
      26 Jan 2009
    • Date of issue
      Dec 2008

    History

    • Accepted
      July 2008
    • Received
      Jan 2008
    Sociedade Brasileira de Pesquisa Operacional Rua Mayrink Veiga, 32 - sala 601 - Centro, 20090-050 Rio de Janeiro RJ - Brasil, Tel.: +55 21 2263-0499, Fax: +55 21 2263-0501 - Rio de Janeiro - RJ - Brazil
    E-mail: sobrapo@sobrapo.org.br