Auto Regulated Recursive Non-Quadratic Algorithm

Abstract

The problem of non-stationary signals constitutes one of the most serious challenges in the context of adaptive filters, since it simultaneously demands high convergence speed, good tracking capability, and stability. However, several algorithms are developed based mainly on second-order statistics in strongly non-stationary scenarios or those subject to non-Gaussian perturbations. The temporal variability of the signal’s statistical properties imposes the need for statistical detection and monitoring mechanisms for abrupt variations in the input signal, increasing the computational complexity of the algorithms. In this context, unlike basic adaptive filtering methods and approaches such as the Kalman filter algorithm, which presuppose second-order statistics, we present the Self-Regulated Non-Quadratic Recursive (RNQA) algorithm, using as a performance surface a sum of weighted even-power error functions and the dynamic adjustment of the correlation matrix normalized by the square of the error, which acts as a mechanism for continuous adaptation of the matrix, capable of responding effectively in non-stationary environments. In a system identification configuration, RNQA showed superior convergence to classical adaptive algorithms such as RNQ and the recursive Kalman filter.

Key words
adaptive filter; convergence rate; non-stationary signal; regularization

Introduction

Adaptive filters are important in various application areas such as biomedical engineering, control systems, channel equalization, and telecommunications. Several well-known filters, such as Recursive Least Squares (RLS) and Least Mean Squares (LMS), are established as the main algorithms for tracking adaptive linear filtering.

The problem of non-stationary signals constitutes one of the most serious challenges in the context of adaptive filters. RLS-type algorithms designed for low steady-state mismatch and numerical stability typically rely on strong stabilization mechanisms or a slow forgetting factor, which significantly degrades their ability to quickly track sudden changes. On the other hand, aggressive adaptation schemes improve transient response at the cost of noisy estimates, loss of robustness, and potential numerical instability. In Kalman Filter (KF) algorithms, small process covariances limit the convergence rate and degrade tracking performance in the face of rapidly varying non-stationary signals, (Hayking 1991).

To mitigate the effects of non-stationarity, the literature proposes several strategies. Among the desirable requirements in an adaptive filter is that it guarantees stable filter behavior.

The commitment to ensuring fast convergence and good tracking motivated extensions such as Variable Step Least Squares Means (VSS-LMS), Momoh-2021, and the Variable Forgetting Factor (VFF) mechanism, cai2013, in the automatic adjustment of the RLS forgetting factor. Previous works include instantaneous error in the RLS algorithm, but the methods were heuristic. An automatic gain control (AGC) scheme implemented by estimating the cross-correlation between the error and the input signal was included in Shan-6. A technique to improve the convergence rate of the RLS algorithm in (Ali et al. 2010) is presented, where the gain vector takes into account the value of $|1/\epsilon$|, where $\epsilon$ is a priori estimate of the error.

A variant of the Least Mean Squares (LMS) algorithm is the Normalized Least Mean Squares (NLMS) algorithm, which improves convergence speed and stability by normalizing the step size with the input signal strength (Widrow & Stearns 1985, Sayed 2003).).

The Kalman Filter (KF) is a benchmark, often referred to as the optimal recursive filter due to its ability to optimally estimate the state parameters of a linear dynamic system by minimizing the mean squared error in the presence of noise. Optimal performance is known for its ability to track time-varying parameters with an appropriate asymptotic response, but it has a high computational cost and requires careful tuning of covariance matrices to ensure robust real-world performance, especially in high-dimensional or nonlinear systems (Sayed 2003, Hayking 1991).

A brief overview of the classical Kalman filter and its variants highlights that the traditional Kalman filter faces two main issues: parameter identification of the system model and the noise model (Bai et al. 2023).

A fundamental limitation of the classical Kalman filter approach lies in the assumption that process and measurement noise are Gaussian. However, in real-world applications involving non-stationary signals with abrupt variations, such assumptions are rarely met. Sudden parameter changes, intermittent excitation, impulsive disturbances, and outliers introduce error distributions. Under these conditions, estimators based exclusively on the quadratic criterion become excessively sensitive to large instantaneous errors, resulting in significant performance degradation and transient instability (Huber & Ronchetti 2009, Zoubir et al. 2018).

Nonetheless, there are new computational algorithms that can improve and outperform the KF in terms of proposals to accelerate the convergence of the mean squared error (MSE).

In various works, it is common to evaluate the performance of algorithms based on criteria such as improving parameter estimation accuracy and enhancing the convergence properties of the algorithms.

Recently, in , the Recursive Non-Quadratic (RNQ) algorithm, which incorporates the algorithm’s own error into the calculation of its gain, demonstrated better convergence than the RLS in a system identification framework. The RNQ adjusts the weights of an adaptive filter using cost functions based on the sum of even powers of the error.

Another desired criterion for algorithms in real-time applications is computational complexity, requiring fewer operations per input data. The RNQ algorithm has the same computational cost as the RLS algorithm, $O(L^2)$ for a filter of size L, (da Silva et al. 2015). For general state-space models, the Kalman filter involves more matrices depending on the linear model and has a higher computational complexity of $O(L^3)$.

Auto-Regulated Recursive Non-Quadratic Algorithm

The proposed Auto-Regulated Recursive Non-Quadratic (ARNQ) algorithm is derived from the RNQ (da Silva et al. 2015), where the update structure of the algorithm is similar to the RLS algorithm but is based on a weighted sum of even powers of the error as the criterion to be minimized. The ARNQ incorporates the instantaneous error into the update of the Kalman gain, effectively controlling the adaptation speed, while both algorithms still yield the same solution (da Silva et al. 2015).

The basic structure of the finite impulse response (FIR) adaptive filter, Fig. 1, consists of the desired signal $d_i$, an input vector ${\mathit{𝒖}}_i=[u_i,u_{i-1},\dots ,u_{i-L+1}]$, and an error vector, which is used to update the filter’s weight vector ${\mathit{𝒘}}_n=[w_{0,n},w_{1,n},\dots ,w_{L-1,n}{]}^T$ with the goal of approximating $d_i$ by estimating the output signal $y_i={\mathit{𝒘}}_{n-1}^T{\mathit{𝒖}}_i$. After calculating the error, ${\epsilon }_i=d_i-y_i$, where $0\le i\le n$ represents the current sample and $L$ is the filter length.

Figure 1
Basic structure of an adaptive filter: ${\mathit{𝒖}}_i$ is the input vector, ${\mathit{𝒅}}_{\mathit{𝒊}}$ is the desired signal, ${\mathit{𝒆}}_{\mathit{𝒊}}$ is the error signal and ${\mathit{𝒚}}_{\mathit{𝒊}}$ is the filter output.

Let $J_n$ be an even, continuous, and symmetric function based on the sum of the even powers of the errors. The regularizing term as $\gamma$ (Hayking 1991) in the cost function defined by

$$J_n=\sum _{j=1}^m{k}^{m-j}\sum _{j=1}^m\{{\lambda }^{n-1}[e_i{]}^{2j}\}+\gamma ||w_n|{|}^2$$(1)

Let $m$, $j$, and $k$ be positive integers and $\gamma$ regularization a small positive value constant parameter. For a fixed value of $m$, within the interval $[-\delta ,\delta ]$, these functions exhibit a steeper slope than the quadratic function within this range (da Silva et al. 2015). The exponential weighting factor $\lambda$ is the conventional forgetting factor, chosen to be less than zero and close to 1, $0\ll \lambda <1$.

The weighting $k^{m-j}$ in Eq.(1) is a multiplicative term that accelerates the convergence of the algorithm. For $j>1$, it defines the even power of the error, and $m$ adds different powers of the error, determining the cost function as a sum of even powers of the error. The larger the value of $j$, the steeper the cost function and the greater the gradient, except near the minimum, where the slope becomes almost flat. This can be understood that as the error power increases, causing the eigenvalue spread of the input autocorrelation matrix to increase proportionally. If $j=1$, $k=1$, and $m=1$, expression in Eq.(1) is equivalent to the MSE cost function. For $j>1$, effectively selects an even power of the error, and $m$ adds different powers of the error.

To obtain the optimal weight vector ${\widehat{\mathit{𝒘}}}_n$, the instantaneous gradient of $J_n$ is calculated by

$$\nabla J_n=\sum _{i=1}^n\{{\lambda }^{n-i}[e_i{]}^{2j-1}{u}_i\}$$(2)

where $a_j=j{k}^{m-j}$ and ${\alpha }_j=2j-2$. By setting the equation $\nabla J_n$ to zero, we define the optimal value of the weight vector ${\widehat{\mathit{𝒘}}}_n$ through the matrix equation: ${\widehat{ww}}_n=\$\tilde{\mathrm{\Phi }}\${\$\tilde{\mathrm{\Phi }}\$}_n^{-1}z{z}_n$, where the $L\times L$ autocorrelation matrix of the input vector

$$\tilde{\mathrm{\Phi }}\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}\tilde{\mathrm{\Phi }}}_n=\sum _{j=1}^m\{a_j\sum _{i=1}^n\{{\lambda }^{n-i}{e}_i^{{\alpha }_i}u{u}_i^{T}u{u}_i\}+\gamma I$$(3)

and the $L\times 1$ cross-correlation vector ${\mathit{𝒛}}_n$, between the inputs and the desired response with dimensions is defined by:

$${\mathit{𝒛}}_n=\sum _{j=1}^m\{a_j\sum _{i=1}^n[{\lambda }^{n-i}{e}_i^{{\alpha }_i}{d}_i{\mathit{𝒖}}_i^T]\}$$(4)

Both the autocorrelation function, Eq.(3), and the cross-correlation vectors, Eq.(4), are now an explicit function of the instantaneous error (da Silva et al. 2015), unlike the conventional quantities in adaptive filtering. This means that the instantaneous error is now influencing the shape of the performance surface, although the optimal solution remains independent of the error and continues to be the Wiener solution (da Silva et al. 2015). This implies that the algorithm gains more control over the learning dynamics without affecting the final solution.

The recursive equations for $$\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n$$ and ${\mathit{𝒛}}_n$ are obtained by isolating the term $i=n$:

$$\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n=\lambda \tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_{n-1}+\sum _{j=1}^m[a_j{e}_n^{{\alpha }_i}]u{u}_{n}u{u}_n^T$$(5)

$$z{z}_n=\lambda z{z}_{n-1}+\sum _{j=1}^m[a_j{e}_n^{{\alpha }_i}]d_{n}u{u}_n$$(6)

where $$\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n$$ is the a priori value of the autocorrelation matrix and ${\mathit{𝒛}}_n$ is the a priori value of the cross-correlation vector.

To calculate the optimal estimate ${\widehat{\mathit{𝒘}}}_n$, the inverse of the correlation matrix $$\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n$$ must be determined, where a fundamental result in matrix algebra known as the matrix inversion lemma is also employed. Applying this lemma, as shown in (Hayking 1991), yields Eq.(3):

$$\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n^{-1}={\lambda }^{-1}\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_{n-1}^{-1}-\frac{{\lambda }^{-1}\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_{n-1}^{-1}u{u}_{n}u{u}_n^T\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_{n-1}^{-1}}{\lambda \sum _{j=1}^m{a}_j{e}_n^{{\alpha }_i}]+u{u}_n^T\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_{n-1}^{-1}u{u}_n}$$(7)

By convention, setting $$P{P}_n=\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n^{-1}$$ and the vector

$${\mathit{𝒈}}_n=\frac{{\mathit{𝑷}}_{n-1}{\mathit{𝒖}}_n}{\lambda [\sum _{j=1}^m{a}_j{e}_n^{{\alpha }_i}]+{\mathit{𝒖}}_n^T{\mathit{𝑷}}_{n-1}{\mathit{𝒖}}_n}$$(8)

where ${\mathit{𝒈}}_n$ is the gain vector with dimensions $L\times 1$. In comparison to the conventional gain vector of the RLS algorithm, the error appears in the summation term, directly affecting the forgetting factor $\lambda$, thereby modulating this factor with instantaneous error information.

Thus,

$${\mathit{𝑷}}_n=\lambda {\mathit{𝑷}}_{n-1}+{\mathit{𝒈}}_n{\mathit{𝒖}}_n^T{\mathit{𝑷}}_{n-1}$$(9)

Rearranging the gain equation, Eq.(8), and substituting it into the autocorrelation matrix, Eq.(9), follows

$${\mathit{𝒈}}_n={\mathit{𝑷}}_{n-1}{\mathit{𝒖}}_n[\sum _{j=1}^m{a}_j{e}_n^{{\alpha }_i}]$$(10)

After simple manipulations in $${\widehat{ww}}_n=\tilde{\mathrm{\Phi }}{\tilde{\mathrm{\Phi }}}_n^{-1}z{z}_n$$ using the expanded ${\mathit{𝒛}}_n$ and Equation (10), the final update structure is given by:

$${\widehat{\mathit{𝒘}}}_n={\widehat{\mathit{𝒘}}}_{n-1}+{\mathit{𝒈}}_n{e}_n,$$(11)

where $e_n$ is a priori estimate of the error, given by

$$e_n=d_n-{\widehat{ww}}_{n-1}^{T}u{u}_n$$(12)

Among the desirable characteristics of a filtering algorithm, in addition to lower computational complexity and good convergence rate, an important concern is to ensure numerical stability. In adaptive filtering, a common approach to this problem is to add a small positive constant term to the covariance matrix, known as regularization or adjustment, in order to prevent its singularity. The regularization in the standard RNQ algorithm uses a small positive value constant parameter $\beta$ added to Eq.(9), resulting in the matrix ${\mathit{𝑷}}_n=\lambda {\mathit{𝑷}}_{n-1}+{\mathit{𝒈}}_n{u}_n^T{\mathit{𝑷}}_{n-1}+\beta I$, where $I$ is an identity matrix (da Silva et al. 2015).

Normalized correlation matrix

Unlike the RNQ method, RNQA acts dynamically using its instantaneous error as an adjustment parameter, adaptively updating the autocorrelation matrix without statistical monitoring and without any mechanism for detecting abrupt changes in the input signal.

The update equation is defined in ${\mathit{𝑷}}_n=\lambda {\mathit{𝑷}}_{n-1}+{\mathit{𝒈}}_n{\mathit{𝒖}}_n^T{\mathit{𝑷}}_{n-1}+\beta \mathit{𝑰}$ , (da Silva et al. 2012). By multiplying both sides by $e_n^2/\beta$, we introduce the a priori error $e_n$ in the update phase of the matrix $P_n$, normalizing the matrix resulting in Eq.(13), where we define ${\widehat{\mathit{𝑷}}}_n=(e_n^2/\beta ){\mathit{𝑷}}_n$. Since $e_n^2>0$ it follows that $e_n^2\mathit{𝑰}\succ 0$ $\Rightarrow {\widehat{\mathit{𝑷}}}_n\succ 0$.

$${\widehat{\mathit{𝑷}}}_n=\lambda {\widehat{\mathit{𝑷}}}_{n-1}+{\mathit{𝒈}}_n{\mathit{𝒖}}_n^{\top }{\widehat{\mathit{𝑷}}}_{n-1}+e_n^2\mathit{𝑰}$$(13)

Rearranging the gain equation, Eq.(8) can be written as follows:

$${\mathit{𝒈}}_n={\widehat{\mathit{𝑷}}}_{n-1}{\mathit{𝒖}}_n[\sum _{j=1}^m{a}_j{e}_n^{{\alpha }_i}]$$(14)

The final update structure is given by:

$${\widehat{\mathit{𝒘}}}_n={\widehat{\mathit{𝒘}}}_{n-1}+{\mathit{𝒈}}_n{e}_n,$$(15)

The adaptive adjustment dynamics of the matrix in eq.(13), allows stationary signals from the system input, ensuring a compromise between alertness and tracking capability.

The proposed ARNQ algorithm involves combining the parameter $m$, which defines the number of even powers in the cost function (1), with the square of the instantaneous error $e_n^2$ as a regularization factor for its autocorrelation matrix $\mathit{𝑷}$, Eq.(9). This has a significant impact on the filter gain, given by ${\mathit{𝒈}}_n={\mathit{𝑷}}_{n-1}{\mathit{𝒖}}_n[{\sum }_{j=1}^m{a}_j{e}_n^{{\alpha }_i}]$, Eq. (10).

The continuous auto-regularization of the ARNQ in relation to the square of the error $e_n^2$ makes the update of the autocorrelation matrix adaptive:

$${\mathit{𝑷}}_n={\lambda }^{-1}({\mathit{𝑷}}_{n-1}{\mathit{𝒈}}_n{\mathit{𝒖}}_n^T{\mathit{𝑷}}_{n-1})+e_n^2\mathit{𝑰}$$(16)

The equations (12),(8),(15),(16), in this order, constitute the ARNQ algorithm summarized in Algorithm 1.

Inicialization:$P_0={\sigma }^{-1}I$, $w_0=0$For:$n=1,2,3...$do$e_n=d_n-{\hat{w}}_{n-1}^T{u}_n$$g_n=\frac{P_{n-1}{u}_n}{\lambda [{\sum }_{j=1}^m{a}_j{e}_n^{{\alpha }_i}]+u_n^T{P}_{n-1}{u}_n}$${\hat{w}}_n={\hat{w}}_{n-1}+g_n{e}_n$$P_n={\lambda }^{-1}(P_{n-1}{g}_n{u}_n^T{P}_{n-1})+e_n^{2}I$

When compared to the KF, Appendix I, the performance of the ARNQ algorithm in parameters tracking is evaluated in terms of convergence speeds by the respective learning curves, based on mean squared errors (MSE).

Results

In the simulations, as shown in Fig. 2, we used a low-pass FIR filter with an impulse response given by $\mathit{𝒉}$, length $L=30$, and cutoff frequencies $f_c=0.5\pi$ and $f_c=0.2\pi$. The input signal ${\mathit{𝒖}}_i$ follows a Gaussian distribution and is to be filtered by $\mathit{𝒉}$. The desired signal $d_i$ is obtained by filtering the input signal ${\mathit{𝒖}}_i$ through $\mathit{𝒉}$. The output of the adaptive filter is denoted by $y_i$. We executed 100 Monte Carlo simulations.

Figure 2
Representation of system identification structure using adaptive filter, where ui is the input vector, $\mathit{𝒉}$ is the FIR filter, ${\mathit{𝒅}}_{\mathit{𝒊}}$ is the desired signal, ${\mathit{𝜼}}_{\mathit{𝒊}}$ is the noise signal, ${\mathit{𝒆}}_{\mathit{𝒊}}$ is the error signal, and ${\mathit{𝒚}}_{\mathit{𝒊}}$ is the filter output.

The proposed algorithm, based on function (1), includes three free parameters: $m$, which represents the highest order of the cost function and controls the number of terms in the sum; $k$, which serves as an additional gain term and a scaling factor for the error; and the forgetting factor $\lambda$. It also features continuous regularization of its covariance matrix $\mathit{𝑷}$, updated using the instantaneous error $e_n^2$. The goal is to analyze the learning dynamics (adaptation time and steady-state MSE) as a function of the parameter $m$ and the regularization term $e_n^2$, to systematically study the behavior of the ARNQ.

Initial parameters and regularization terms:

• ARNQ: ${\mathit{𝑷}}_0={10}^{-5}I$,$\lambda =0.96$, (m=5, m=10 and m=15).

• KF: ${\mathit{𝑷}}_0={10}^{-5}I$, ${\mathit{𝑹}}_1={10}^4$

In order to show the effect of the self-regularization of the ARNQ on the regularization with a constant term, we include the standard RNQ that uses a regularization term $\beta ={10}^{-4}$ with equal initialization ${\mathit{𝑷}}_0={10}^{-5}\mathit{𝑰}$, and parameter $\lambda =0.96$, $m=15$. The learning curves of the algorithms involved in the simulation process for system identification are illustrated in Fig. 3.

Figure 3
Comparison of the learning curves of the ARNQ with $\mathit{𝒎}\mathbf{=}\mathbf{𝟓}$ based on ${\mathit{𝒆}}_{\mathit{𝒏}}^{\mathbf{𝟐}}$, the Kalman Filter for system identification, and the RNQ with $\mathbf{(}\mathit{𝒎}\mathbf{=}\mathbf{𝟏𝟓}\mathbf{)}$, using a fixed regularization factor of $\mathit{𝜷}\mathbf{=}{\mathbf{𝟏𝟎}}^{\mathit{-}\mathbf{𝟒}}$. a) The simulations were conducted with a noise-free system. b) The simulations were conducted with additive Gaussian noise with SNR = 60 dB.

Discussion

In this system identification structure, the results are illustrated in Fig. 3, where the learning curves are analyzed in terms of convergence rates, defined as the moment of convergence when the number of interactions reaches $5\%$ of the steady-state error.

The ARNQ, with different quantities of even powers $m=5,m=10$ and $m=15$, self-adjusting by $e_n^2$, results in varying convergence rates. In Fig. 3a, the learning curves of the algorithms are shown for a noise-free system, ${\eta }_i=0$, where at interaction $n=1000$, the parameters were altered. While the Kalman algorithm (KF) (Söderström & Stoica 1989, Appendix I), took 700 iterations to reach its steady state, the ARNQ achieved convergence between 100 and 500 iterations, demonstrating that the ARNQ was significantly faster, exhibiting a higher convergence rate than the Kalman filter algorithm, particularly for $m=15$.

In Fig. 3b, the simulations were conducted with additive Gaussian noise with SNR = 60 dB. The same behavior was observed in the simulations with noise. It is notable that the Kalman filter exhibits linear convergence, while the others demonstrate nonlinear characteristics.

In the simulation with noise, shown in Fig. 3b, it is interesting to note a certain oscillation in the behavior of the ARNQ during the initial interactions, where it lags in convergence compared to the standard RNQ with $m=15$ and a constant regularization factor of $\beta ={10}^{-4}$. This can be attributed to the fact that, at the beginning, the instantaneous error $e_n^2$, which serves as a regularization factor for the covariance matrix ${\mathit{𝑷}}_n={\lambda }^{-1}({\mathit{𝑷}}_{n-1}{\mathit{𝒈}}_n{u}_n^T{\mathit{𝑷}}_{n-1})+e_n^2\mathit{𝑰}$, is quite large, as expected.

Consequently, the initial parameter characteristics of the RNQ, with its maximum parameter $m$, yield better initial conditions for performance. The advantage of the ARNQ emerges when there are abrupt changes in the parameters. The variations introduced by the adapted error continuously reinitialize the regularization factor in matrix $\mathit{𝑷}$, and, combined with the higher power of $m=15$, allow the ARNQ to be more responsive, converging faster than the others. This is evidenced in Fig. 3b, which highlights the superior performance of the ARNQ during the parameter change cycles at iterations 500, 1000, and 1500.

Conclusions

In this work, we present the performance of the ARNQ algorithm, which results from the inclusion of the square of the instantaneous error of the algorithm as a regularization factor for the covariance matrix $\mathit{𝑷}$. This modification has led to a significant increase in adaptation speed compared to the KF algorithm, which employs Kalman filter methodology, and has also outperformed the standard RNQ.

In real-time applications within non-stationary environments, the ARNQ, with its higher convergence rate, lower computational complexity, and characteristic adaptive self-adjustment of the covariance matrix, demonstrates promise compared to the Kalman filter, which entails greater computational complexity due to the involvement of a larger number of matrices and consideration of the hypothesis of Gaussian process and measurement noise.

• Data availability
  All data generated or analyzed during this study are included in this published article.

References

• ALI A, REHMAN AU & ALI R. 2010. An Improved Gain Vector to Enhance Convergence Characteristics of Recursive Least Squares Algorithm. Int J Hybrid Inf Technol 4: 99-107.
• BAI Y, YAN B, ZHOU C, SU T & JIN X. 2023. State of art on state estimation: Kalman filter driven by machine learning. Annu Rev Control 56: 100909. doi:10.1016/j.arcontrol.2023.100909.
  » https://doi.org/10.1016/j.arcontrol.2023.100909
• DA SILVA C, BARROS AK, SANTANA E, DE ARAÚJO MA, LOPES MVS, FONSECA JV & PRINCIPE JC. 2015. An algorithm based on non-squared sum of the errors. Signal Process 117: 188-191. doi:10.1016/j.sigpro.2015.03.012.
  » https://doi.org/10.1016/j.sigpro.2015.03.012
• DA SILVA C, SANTANA E, AGUIAR E, ARAÚJO MA & BARROS AK. 2012. An adaptive recursive algorithm based on non-quadratic function of the error. Signal Process 92(4): 853-856. doi:10.1016/j.sigpro.2011.09.019.
  » https://doi.org/10.1016/j.sigpro.2011.09.019
• HAYKING S. 1991. Adaptive Filter theory. Prentice-Hall.
• HUBER PJ & RONCHETTI EM. 2009. Robust Statistics. Wiley.
• SAYED AH. 2003. Fundamentals of adaptive filtering. John Wiley & Sons.
• SÖDERSTRÖM T & STOICA P. 1989. System Identification. Prentice-Hall international series in systems and control engineering. Prentice Hall.
• WIDROW B & STEARNS SD. 1985. Adaptive Signal Processing. Prentice-Hall signal processing series. Prentice-Hall.
• ZOUBIR AM, KOIVUNEN V, OLLILA E & MUMA M. 2018. Robust Statistics for Signal Processing. John Wiley & Sons, Ltd. doi:10.1002/9780470434697.fmatter.
  » https://doi.org/10.1002/9780470434697.fmatter

Appendix I Kalman filter

A Kalman filter as a parameter estimator is presented in (Söderström & Stoica 1989).

Considering the constant parameter model

$$y_n={\phi }_n^T\theta +e_n$$(17)

it can be described in state space

$$x_{n+1}=x_n$$ (18)

$$y_n={\phi }^T{x}_n+e_n$$(19)

where $x_n$ is the state vector $x_n=[a_1\dots a_{na},b_1\dots b_{nb}{]}^T=\theta$ and $e_n$ is the perturbation with variance ${\sigma }_e^2$. The optimal state estimate ${\hat{x}}_{n+1}$ can be calculated based on the measurements $y_1\dots y_n$ using the Kalman filter.

Applying the Kalman filter to the state model, Eq. (A.2), results precisely in the basic least squares (LS) algorithm. A modification in Eq. (A.2) allows for better tracking of time-varying parameters:

$$x_{n+1}=x_n+v_n$$(20)

where $E\{v_n{v}_n^T\}=R_1\delta$. This means that the parameter vector is modeled as a random walk variable. Applying the Kalman filter to the model (A.4),(19)

$x_{n+1}=x_n+v_n$

$y_{n+1}={\phi }^T{x}_n+e_n$

the following recursive equations are obtained:

${\hat{\theta }}_n={\hat{\theta }}_{n-1}+K_n{\epsilon }_n$ ${\epsilon }_n=y_n-{\phi }_n^T{\hat{\theta }}_{n-1}$ $K_n=P_{n-1}{\phi }_n/[1+{\phi }_n^T{P}_{n-1}{\phi }_n]$ $P_n=P_{n-1}-P_{n-1}{\phi }_n{\phi }_n^T{P}_{n-1}/[1+{\phi }_n^T{P}_{n-1}{\phi }_n^T]+R_1$

In the algorithm, the matrix $R_1$ is added to its matrix $P_n=P_{n-1}-P_{n-1}{\varphi }_n{\varphi }_n^T{P}_{n-1}/[1+{\varphi }_n^T{P}_{n-1}{\varphi }_n^T\theta ]+R_1$, where the matrix $R_1$ plays a role similar to the forgetting factor $\lambda$ in the RLS algorithm (Söderström & Stoica 1989) and is used to describe how quickly the different state components vary.

Edited by

• Handling editor
  Andrea Balbo

Data availability

All data generated or analyzed during this study are included in this published article.

Publication Dates

• Publication in this collection
  14 Sept 2026
• Date of issue
  2026

History

• Received
  29 Apr 2025
• Accepted
  07 Apr 2026