The performance surface in nonlinear mean square estimation: application to active noise control problems with correlated signals

THE PERFORMANCE SURFACE IN NONLINEAR MEAN SQUARE ESTIMATION: APPLICATION TO ACTIVE NOISE CONTROL PROBLEMS WITH CORRELATED SIGNALS

Márcio H. Costa *

[e-mail]

José C. M. Bermudez^† [e-mail]

Neil J. Bershad ^‡ [e-mail]

*Lab. de Engenharia Biomédica, Escola de Engenharia, Universidade Católica de Pelotas, Pelotas, RS, Brazil

^†Depto de Engenharia Elétrica, Universidade Federal de Santa Catarina, Florianópolis, SC, Brazil

^‡Department of Electrical and Computer Engineering, University of California, Irvine, CA, USA

ABSTRACT

This paper investigates the properties of the performance surface for the problem of nonlinear mean-square estimation of a random sequence. The problem studied has direct application to the study of active noise control (ANC) systems when the transducers are driven into a nonlinear behavior. A deterministic expression is derived for the mean-square error (MSE) surface as a function of the system's degree of nonlinearity for Gaussian correlated input signals. It is shown how the presence of the nonlinearity deforms the MSE surface. It is demonstrated that the surface is unimodal, and the expression for the optimum weight vector is determined. The new results are then used to quantify the behavior of ANC systems employing the LMS adaptive algorithm. Important algorithm properties are derived from this study. Examples are presented which verify the analytical models derived.

KEYWORDS: active noise control, adaptive filters, adaptive algorithms, nonlinear systems, estimation theory.

RESUMO

Este artigo investiga as propriedades da superfície de desempenho para o problema de estimação média quadrática não-linear de uma seqüência aleatória. Os resultados obtidos possuem aplicação direta no estudo de sistemas de controle ativo de ruído (CAR) quando os transdutores possuem um comportamento não-linear. É desenvolvida uma expressão determinística para a superfície do erro médio quadrático em função do grau de não-linearidade, supondo-se sinais de entrada gaussianos correlacionados. Através deste resultado é determinado o vetor de coeficientes ótimo, demonstrada a unimodalidade da superfície de erro e a maneira pela qual a presença da não-linearidade a deforma. A partir disto, os resultados obtidos são utilizados para quantificar o comportamento de sistemas CAR que empregam o algoritmo adaptativo LMS. Como resultado, importantes propriedades do algoritmo são verificadas. Finalizando, simulações comprovam a validade dos modelos analíticos desenvolvidos.

PALAVRAS-CHAVE: controle ativo de ruído, filtros adaptativos, sistemas não-lineares, teoria de estimação.

1 INTRODUCTION

Mean square estimation plays a crucial role in many problems of adaptive control (Ren and Kumar, 1992) and adaptive signal processing (Haykin, 1996). Therefore, the question of how to design adaptive estimation systems for optimized behavior has met considerable interest. Optimal system designs require detailed knowledge about the theoretical problem and about the adaptive algorithm performance in solving that problem. Such knowledge is obtained through analysis of the system behavior and derivation of analytical models that can accurately predict this behavior.

Analyses of practical systems' behavior always rely on restrictive assumptions to make the mathematical task feasible. The resulting analytical model is applicable to situations in which the effects of the neglected nonidealities are not significant.

Active noise and vibration control (ANC) is a technique extensively used by the control (Angevine, 1995; Füller and Flotow, 1995) and signal processing (Kuo and Morgan, 1996; Hansen, 1997) communities. It consists of cancelling sound or vibrational waves through destructive interference. A practical example is the noise reduction in ventilation ducts (Massarani et alii, 1990; Osório and Nóbrega, 1995).

Adaptive linear control techniques have been largely applied in ANC (Kuo and Morgan, 1996). However, considerable nonlinear effects from overdriven loudspeakers, piezoelectric transducers or power amplifiers in the secondary path (the path leading from the adaptive filter output to the cancellation point) have been reported in actual ANC systems. In addition, correlated input signals are very common in ANC applications (Kuo and Morgan, 1996; Hansen, 1997).

Fig. 1 shows the block diagram of an ANC system influenced by a nonlinearity in the secondary path. The block g(·) is a saturation nonlinearity. It represents the composed nonlinear effects in that path (Costa et alii, 1999). The design problem usually consists of determining the optimum control weight vector W that minimizes the mean square error (MSE) at the system output (Osório and Nóbrega, 1995). In this case, minimization of the MSE defines a nonlinear mean square estimation problem. The random signal d(n) is estimated by a nonlinear function of the reference signal x(n) (Papoulis, 1991 – section 7-5).

Although ANC system nonlinearities are quite common, very little has been reported in the literature on their effects on the MSE surface. Therefore, very little is known about the behavior of adaptive algorithms used to cancel undesirable noise under this constraint. A recent paper (Costa et alii, 1999) has studied the statistical behavior of the system in Fig. 1 when the filter coefficient vector W is adjusted using the Least Mean Square (LMS) adaptive algorithm. The analysis presented in Costa et alii (1999) determined analytical models for the mean weight and the MSE behaviors for slow adaptation and white input signals. Very accurate estimates of the transient and steady-state algorithm behavior were obtained. However, this analysis does not provide all the necessary design information if the MSE performance surface properties are unknown. The knowledge of such properties allows the designer to determine the algorithm behavior for a given degree of nonlinearity, as compared to the optimum. In addition, the MSE surface properties are necessary for a meaningful performance comparison among different adaptive algorithms.

An initial investigation by Costa et alii (2000) has determined the MSE surface properties for white Gaussian inputs. Though these results lead to important insights on the characteristics of the nonlinear estimation problem, they do not provide accurate information for the important case of correlated input signals (Kuo and Morgan, 1996; Hansen, 1997).

This paper extends the analysis of (Costa et alii, 2000) to determine the MSE surface property for systems with correlated input signals. A deterministic expression is derived for the MSE surface as a function of the system's degree of nonlinearity. It is shown how the presence of the nonlinearity deforms the MSE surface. The surface is shown to remain unimodal for any degree of nonlinearity. The optimum weight vector and the minimum MSE are determined.

These results can be directly applied to several ANC systems (designed through different control techniques). They permit the evaluation of the canceller performance in the presence of nonlinearities. Thus, minimizing the cost of the transducers and associated hardware with a predictable loss in performance becomes feasible.

Next, the particular but important case of the LMS algorithm is studied. The MSE surface properties are used to provide new insights on the behavior of the algorithm when applied to ANC systems. New results are presented for the LMS algorithm behavior with correlated input signals. The MSE misadjustment and the converged weight misalignment are determined as functions of the system's degree of nonlinearity. It is verified that the converged mean weight vector for the LMS is a scaled version of the optimum weight vector.

2 ANALYSIS OF THE MSE SURFACE

The block diagram in Fig. 1 shows the nonlinear mean square estimation problem studied. This diagram is representative of an ANC system with loudspeakers or piezoelectric transducers driven into nonlinear operation (Bernhard et alii, 1997).

W^o = [w₀^o w₁^o ¼ w_{N – 1}^o ]^T is the vector of the impulse response samples of a linear system (plant). W = [w₀ w₁ ¼ w_{N – 1} ]^T is the transversal FIR linear controller weight vector. d(n) is the signal to be estimated (primary signal). x(n) is the reference signal which is assumed stationary, zero-mean and Gaussian. X(n) = [x(n) x(n – 1 ) ¼ x(n – N + 1 ) ]^T is the observed data vector and R_XX = E{ X(n)X^T (n) } is the reference input correlation matrix. z(n) is the measurement noise, assumed stationary, white, gaussian, zero-mean, with variance and uncorrelated with any other signal. y(n) is the control signal, and e(n) is the error signal to be minimized in the MSE sense. The nonlinearity is modeled by the scaled error function:

Note that

g(y) = y and g(y) = s sgn(y). Hence, g(y) properly scaled can range between a linear device and a hard limiter by varying s.

2.1 MSE Performance Surface

The error signal in Fig. 1 is given by:

Squaring (2) and taking the expected value yields:

The first four expectations are easily evaluated using the statistical properties of x(n) and z(n): E{ X(n)X^T (n) } = R_XX ; E{ z²(n) } = ; E{ z(n)X(n)} = 0 and E{ g[ W^TX(n)]z(n)} = 0.

Since x(n) is zero-mean Gaussian and W is constant, the last two terms in (3) include expectations of nonlinear functions of zero-mean Gaussian variables. The fifth expectation can be obtained from Shynk and Bershad (1991) for b₁ = 0, s_q = s, c = 1\s and = W^TR_XXW. Thus,

The last expectation can be obtained from Bershad et alii, (1993)(Appendix) for a_f = 1, b₁ (n) = 1 and b = W^TR_xxW as

Combining the above results into (3) yields an analytical expression for the MSE surface:

The system's degree of nonlinearity is defined as

This parameter relates the power of d(n) to the maximum power at the output of the nonlinearity. It can be easily shown by taking the limit of (1) as y ® ¥ that max{} = s². Thus,

For the linear case, s ® ¥ and h² ® 0.

Fig. 5 presents examples of the MSE surface for different degrees of nonlinearity h². Notice that the surface deforms as h² increases. Regions of slower convergence (small gradient) appear as h² increases. However, it remains unimodal for any degree of nonlinearity. This important result will be demonstrated in the next subsection. Note that the MSE (6) is not minimized by W = W^o unless h² = 0 (linear case).

2.2 Stationary Points

Differentiating (6) with respect to the weight vector and equating it to zero yields an expression for the minima of the MSE surface. Thus,

Evaluating the derivatives in (9) and setting W = yields:

Equation (10) can be written as:

Note that all terms in the fraction multiplying W^o in (11) are real nonegative scalars. Thus, the optimum weight vector is a scaled version cW^o of W^o, c Î + .

Substituting cW^o for and using (8) in (11) yields:

Note that c Î + . For given W^o and h², the constant c must be a solution of:

Equation (13) yields four solutions:

Because c must be real and positive, only the positive sign is acceptable outside the square roots. In addition, it is well known that c = 1 is the only optimal solution (the Wiener solution) for h² ® 0 (g(y) = y, linear case). This eliminates the possibility of the minus sign within the square root (the minus sign would lead to a complex value for c as h² ® 0). Thus, the only allowable solution for (13) is:

The minimum of the MSE surface is then:

Fig. 2 shows the effect of the nonlinearity on the positioning of the optimum weight vector in the direction of W^o.

Using (16) in (6) yields an expression for the minimum of the MSE performance surface:

Fig. 3 shows the excess MSE (x_ex = x_MIN – ) caused by the nonlinearity, relative to the linear case, for the normalized case W^oTR_xxW^o = 1. Eq. (17) determines the best performance that can possibly be expected from any adaptive algorithm used to solve the nonlinear estimation problem depicted in Fig. 1.

3 APPLICATION TO ANC SYSTEMS

Several control techniques can be used to optimize the weight vector in the system of Fig. 1. One of them is the adaptive system depicted in Fig. 4. Thus, the results of Section 2 can be used to quantify the performance of the LMS algorithm in a nonlinear ANC system. A similar approach can be used with other solutions to the ANC problem (Massarani et alii,1990; Osório and Nóbrega, 1995).

Analytical expressions have been derived in Costa et alii (1999) for the converged mean weight and MSE for the LMS algorithm with white Gaussian inputs and slow adaptation. These expressions can be expanded to the correlated input signal case as:

Note that the converged mean LMS weight vector W_¥ is also a scaled version of the optimum solution W^o for the linear case. Using (16) and (18) it is easy to show that:

Equation (20) shows that the LMS algorithm produces a biased estimate of the optimum weight vector . The multiplicative bias b in (20) is a function of the system's degree of nonlinearity.

Using (17) and (19), the LMS excess MSE can be determined:

The misadjustment can be obtained normalizing (21):

Note that (20), (21) and (22) hold only for h² < 1. The mean weight of the LMS algorithm does not converge for h²³ 1 (Costa et alii, 1999).

3.1 Numerical Example

This section presents simple examples to illustrate the application of the analytical results. Consider the system in Fig. 1 with W^o = [0.707 0.707]^T, W^oTW^o = 1, = 10 ^{– 6} and a unit-variance correlated input signal. The eigenvalue spread (_max / _min) of R_XX is equal to 24.

Fig. 5 shows the MSE surface for three different degrees of nonlinearity. Note the increasing deformation (with increasing asymmetry) of the MSE surface with increasing h². The LMS algorithm does not converge for case (c). Notice also that the nonlinearity increases the region of small gradient in the surface.

Assuming Fig. 4 with the same parameters described as above, Fig. 6 shows the MSE surface contours and the LMS weight trajectories for random initialization, m = 0.01 and four different degrees of nonlinearity. Vectors W^o and are also shown. Notice that in all cases W^o, and the converged LMS weight vector W_¥ are aligned with the point (0,0) (the LMS weights converge asymptotically to this line in plot (d)). This behavior is in accordance with equations (16) and (20). Figs. 6a-6c show the weights convergence for h² < 1. For a small h² (Fig. 6a) the converged weights closely approach the minimum of the MSE surface (which also tends to W^o). Fig. 6d show the divergence of the LMS algorithm for h² > 1.

Figs. 7 and 8 show the weight bias b (Eq. (20)) and the MSE misadjustment as a function of h². As h² ® 0 (linear case), b ® 1. When h² ® 1, b increases very fast and finally grows without bound.

Figs. 9 and 10 show the simulated MSE and the behavior of the first coefficient for a system with 30 coefficients, m = 0.01, = 10 ^{– 6}, 500 runs and R_XX eigenvalue spread equal to 32. Three different degrees of nonlinearities are considered for each figure, chosen in order to permit a clear separation among curves. Fig. 9 shows the MSE behavior for the LMS algorithm (ragged curves). The minima of the MSE surface (Eq. (17)) for each degree of nonlinearity are shown as lines of circles. The distance between each curve in steady-state and the corresponding minimum confirm the difference between (17) and (19). Fig. 10 shows the behavior of the first adaptive weight (similar behavior was verified for all coefficients). Again, the optimum solutions (Eq. (16)) are shown as lines of circles. Mismatches between steady-state weight behavior and optimum weight confirm the weight mismatch derived in Eq. (20).

Two important results can be inferred from Fig. 9 and 10: (1) the LMS algorithm cannot achieve the minimum of the performance surface, generating a biased solution (multiplicative bias); (2) if the real system is incorrectly modeled as a linear system, the effect of the nonlinearity can cause a significant overestimation of the noise cancellation capabilities of any adaptive algorithm based on mean square estimation.

Fig. 11 presents the function g(·) and the histograms for the amplitude of the nonlinearity input (output of the adaptive filter). The histograms were determined from the signal amplitudes for all the 7000 iterations, averaged over 10 runs (10 realizations). Fig. 11 clearly shows that in cases (b), (c) and (d) the system is driven into a nonlinear region of operation. This emphasizes the importance of analytical models that take into consideration the effect of the secondary path nonlinearity whenever it is physically unavoidable in practical applications such as active noise and vibration control.

Table 1 compares the MSE of the converged LMS with the minimum MSE for several degrees of nonlinearity. These values were obtained from (17) and (19) and confirmed through simulation. The last column shows the corresponding MSE misadjustment.

4 SUMMARY

This paper has investigated the properties of the performance surface for a nonlinear mean-square estimation of a Gaussian random sequence. It expands a previous study, generalizing the statistical characteristics of the input signal. The results of this study have direct application to active noise and vibration control systems when the transducers are driven into a nonlinear behavior and the input signal is correlated. They can be used to evaluate the performance of several methods for determining the optimum controller. A deterministic expression was derived for the MSE surface as a function of the system's degree of nonlinearity. It was shown how the nonlinearity deforms the MSE surface. This surface was shown to be unimodal, and the optimum weight vector determined. Finally, as an example, the new results were used to quantify the behavior of ANC systems employing the LMS adaptive algorithm. The MSE misadjustment was evaluated as a function of the degree of nonlinearity. The converged mean weight vector for the LMS was shown to be a scaled version of the optimum weight vector.

ACKNOWLEDGMENTS

This work was supported in part by CAPES (Brazilian Ministry of Education) under Grant No. PICDT 0129/97-9, and by CNPq (Brazilian Ministry of Science and Technology) under Grant No. 352084/92-8.

• Angevine, O.L. (1995). Active Systems for attenuation of Noise. International Journal of Active Control, 1:1, pp. 65-78.
• Bernhard, R., P. Davies, and S. Kurth, (1997). Effects of Nonlinearities on System Identification in Active Noise Control Systems. Proceedings of Noise-Con 97, Pennsylvania, USA, pp. 231-236.
• Bershad N.J., J.J. Shynk and P.L. Feintuch (1993). Statistical Analysis of the Single Layer Backpropagation Algorithm: Part II - MSE and Classification Performance. IEEE Transactions on Signal Processing, 41:2, pp. 583-591.
• Costa, M.H., J.C.M. Bermudez, and N.J. Bershad (1999). Statistical Analysis of the LMS Algorithm with a Zero-Memory Nonlinearity after the Adaptive Filter. Proceedings of ICASSP 99, Phoenix, USA.
• Costa, M.H., J.C.M. Bermudez, and N.J. Bershad (2000). The Performance Surface in Nonlinear Mean Square Estimation: Application to the Active Noise Control Problem. Proceedings of ICASSP 2000, Istambul, Turkey.
• Füller, C.R. and Flotow, A.H. (1995). Active Control of Sound and Vibration. IEEE Control Systems, Dec., pp. 9-19.
• Hansen, C. (1997). Active Noise Control - From Laboratory to Industrial Implementation. Proceedings of Noise-Con 97, Pennsylvania, USA, pp. 3-38.
• Haykin, S. (1996). Adaptive Filter Theory Prentice-Hall, third edition.
• Kuo, S. and D. Morgan (1996). Active Noise Control Systems: Algorithms and DSP Implementations John Wiley and Sons.
• Massarani, P.M., Zindeluk, M. and Tenenbaum, R. (1990). Realizaçăo Experimental de Controle Ativo em Dutos. Proceedings of XI Encontro da Sociedade Brasileira de Acústica, Săo Paulo.
• Osório, P.O. and Nóbrega, M.V. (1995). Controle Ativo de Ruído de Banda Larga em Dutos. Sba Controle & Automação, 6:2, pp. 70-78.
• Papoulis, A. (1991). Probability, Random Variables and Stochastic Processes McGraw-Hill, third edition.
• Ren, W. and Kumar, P.R. (1992). Stochastic Parallel Model Adaptation: Theory and Applications to Active Noise Cancelling, Feedforward Control, IIR Filtering and Identification. IEEE Transactions on Automatic Control, 37:5, pp. 566-578.
• Shink, J.J. and N.J. Bershad (1991). Steady-State Analysis of a Single-Layer Perceptron Based on a System Identification Model with Bias Terms. IEEE Transactions on Circuits and Systems, 38:9, pp. 1030-1042.

Publication Dates

• Publication in this collection
  15 Jan 2003
• Date of issue
  Apr 2002