Physics-Informed Neural Network for monitoring the sulfate ion adsorption process using particle filter

Abstract

Fixed-bed columns are a well-established water purification technology. Several models have been constructed over the decades to scale up and predict the breakthrough curve of an adsorption column varying the flow rate, length, and initial concentration of solute. In this work, we proposed using an emerging computational approach of a physic-informed neural network (PINN) that uses artificial intelligence to solve the partial differential equation model of adsorption. The effectiveness of this approach is compared with finite-volume methods and experimental data. We also couple the PINN with a sampling importance resampling particle filter, a Bayesian technique that allows the filter and estimate states of the process, quantifying uncertainties of experimental measurements. The results shows physic-informed neural network capability in solving the proposed model and its uses as an evolution model for sequential estimation.

Key words
adsorption; PINN; machine learning; particle filter; fixed-bed column

INTRODUCTION

Fixed-bed columns are widely used in several industrial processes, mainly chemical and environmental engineering. These columns are packed with solid materials, such as catalysts, adsorbents, or ion exchange resins, and play a crucial role in substance separation, purification, and conversion. The process involves passing a fluid, such as a gas or a liquid, through the fixed bed, allowing the fluid to interact with the packed materials. The fixed-bed column facilitates chemical reactions in catalytic applications, transforming reactants into desired products. In adsorption processes, the column captures specific components from the fluid. Additionally, fixed-bed columns are employed in water treatment facilities for removing contaminants and softening water. Their versatility and efficiency make them indispensable tools in numerous industries, contributing to technological advancements and sustainable practices. (Muzic et al. 2010, Thongsamer et al. 2023)

In industrial problems, modeling involves identifying a complex and unknown relationship between inputs and outputs. Subsequently, engineers face the challenge of finding solutions for the proposed models. Since the late 1940s, the widespread availability of digital computers has driven remarkable growth in the use and development of numerical methods (Chapra & Canale 2021). In 2017, the Physical Informed Neural Networks (PINNs) were introduced as a novel class of data-driven solvers in a two-part article Raissi et al. (2017a, b). Subsequently, in 2019, the same work was consolidated into a single merged version Raissi et al. (2019). Raissi et al. (2019) elaborate on the PINN approach and illustrate its effectiveness in solving nonlinear PDEs, such as the Schrödinger, Burgers, and Allen-Cahn equations. Thus, the PINN can be defined as a specific type of deep neural network architecture designed to tackle supervised learning tasks while respecting any physical laws described by differential equations. This approach combines the machine learning capability of neural networks with physical information during training. Since the publication of Raissi et al. (2019), numerous new papers have emerged addressing PINN in a multidisciplinary context. Lu et al. (2021) proposed a Python library for PINNs, named DeepXDE, which was designed to serve both as an academic and research tool for solving problems in computational science and engineering. Specifically, DeepXDE can handle both forward and inverse problems, given some additional measurements. Below, it is possible to observe some applications in various areas of knowledge.

To overcome the limitations of the computational resources involved in computational fluid dynamics simulations, Queiroz et al. (2021) proposed PINN as a reduced-order data-driven model that respects the flow field behavior and mass and momentum conservation of the Navier-Stokes equations. The results demonstrated that PINN could capture the complex flow behavior for both velocity and pressure fields, with the advantage of being 200 times faster than CFD simulations. Sel et al. (2023) established PINN models for physiological time series data, using minimal accurate information to extract complex cardiovascular insights. The Taylor approximation was employed to gradually modify the known cardiovascular relationships between input and output and incorporated into the training of the proposed neural network. As a result, they demonstrated that when using PINNs in state-of-the-art tested time series models, high correlations and low errors were acquired.

Despite several applications of PINNs found in literature, there needs to be more application in the adsorption process. A recent work proposed by Santana et al. (2022) aimed to evaluate the use of PINN to address the numerical solutions of a fixed-bed column where a monoclonal antibody is purified. The PINN solutions were compared with a traditional numerical method of lines showing accuracy. Ngo & Lim (2021, 2022) used a physics-informed neural network approach to solve an isothermal fixed-bed model for CO₂ catalytic methanation. The PINN includes a feed-forward artificial neural network (FF-ANN) and physics-informed constraints, such as governing equations, boundary conditions, and reaction kinetics. The direct hybrid model solves the plug-flow reactor, while the inverse problem used PINN model reveals an unknown effectiveness factor in the reaction kinetics. The direct PINN showed excellent extrapolation performance with high accuracy in their results. On the other hand, the inverse identified an unknown effectiveness factor with low error, suggesting that direct and inverse PINNs can be used in determining solutions and model systems of fixed-bed reactors with chemical reaction kinetics. Wu et al. (2023) investigated a binary gas CO₂/N₂ adsorption process of fixed-bed filled with MOFs, using the PINN to calculate the breakthrough curve of CO₂. Serebrennikova et al. (2022) applied PINNs to solve inverse and forward transport problems of organic volatile compounds (VOCs) through porous media with active surfaces.

Guo et al. (2022) proposed a stochastic deep placement method (DCM) based on neural architecture search (NAS) and transfer learning, applied to the stochastic analysis of heterogeneous porous materials. In the research, the DCM-based NAS presented saves the weights and biases of the most favorable architectures, which are then used in the fine-tuning process. Transfer learning techniques were also employed to drastically reduce computational cost. Regarding performance, the DCM-based NAS was verified in different dimensions using the method of manufactured solutions. The research shows that it significantly outperforms finite difference methods in both accuracy and computational cost.

Nguyen-Thanh et al. (2021) proposed a Parametric Deep Energy Method (P-DEM) for elasticity problems that consider strain gradient effects. The approach is based on physics-informed neural networks (PINNs) for solving the underlying potential energy. Thus, a cost function related to the potential energy was minimized. Among the advantages, P-DEM does not require classical discretization and only needs a definition of the potential energy, facilitating implementation. The forward-backward mapping was established using NURBS basis functions. The performance of the method was demonstrated through various numerical benchmark problems in elasticity and compared to analytical solutions. Strain gradient elasticity was also considered, which poses challenges for conventional finite elements due to the requirement of continuity.

Samaniego et al. (2020) explored Deep Neural Networks (DNNs) as an option for approximating Partial Differential Equations (PDEs). In their paper, these networks were applied in the field of Computational Mechanics, exploring various problems and the method’s capabilities for engineering applications. Most contributions adopted a collocation strategy and analyzed the energetic form of the PDEs. One of the advantages of the proposed approach is its ease of implementation, as it allows for the use of an almost mathematical notation to define the loss function, which represents the energy. Conceptually, this opens the door to exploring different mathematical models, defining corresponding energies, and implementing them in a very direct and transparent manner.

The studies conducted by Guimarães & Leão (2014a, b) aimed to investigate the application of ion exchange resins for sulfate removal in fixed-bed columns. Two macroporous ion exchange resins were investigated, Purolite A500 and Amberlyst A21. Adsorption experiments in a fixed-bed, using synthetic sulfate solutions, were evaluated, and a set of experimental data covering different configurations of fixed-bed columns was presented. Further, Carvalho et al. (2019) used the method of lines to model the fixed-bed column and used a Bayesian framework to estimate operational parameters using the Sampling Importance Resampling (SIR) particle filter as a fast and robust tool for monitoring a sulfate ion removal problem. In this article, experimental measurements were used to validate the methodology, and the particle filter (PF) performance was evaluated through metrics such as error and computational time. The results showed that the PF could filter the uncertainties in a way that approximated the actual behavior of the process from the experimental measurements.

Extending the investigations based on Guimarães & Leão (2014a), the focus of this article will be to model and numerically solve the problem using the PINNs. In our analyses, the parameter to be varied is the bed length, represented by L. After obtaining this model, the surrogate model will be used as an evolutionary model for the particle filter algorithm with importance sampling importance resampling (SIR). The strategy of using PINN as an evolutionary model has not been addressed in the literature, as the standard practice is to use classical methods as seen in Carvalho et al. (2019), which can make the application of the filter unfeasible in some cases due to computational costs. Thus, the approach in this article of using PINN can reduce the computational cost and contribute to the development of this monitoring technique. The paper is structured as follows: "Methodology" describes the methodology showing governing equations of physical problem and presents the architecture of the Physics-Informed Neural Network, two numerical solutions were used for comparison, and the construction of the particle filter algorithm. In "Results and Discussion", numerical results and corresponding discussions are presented. Finally, in "Conclusions", the conclusions obtained from this study are presented.

METHODOLOGY

Isothermal Fixed-Bed Column for Sulphate Adsorption Model

The equation employed in this study is consistent with references Chu (2010) and Carvalho et al. (2019), where the model is characterized by the independent variables time ($t$) and column length ($z$). In the partial differential Eq. (1), the interstitial velocity of the fluid phase in the column is denoted by $u_0$, the concentration of the adsorbate in the liquid phase is represented by $c$, the bed porosity is indicated by $\epsilon$, the density of the adsorbent is expressed by ${\rho }_L$, the average concentration of the adsorbate in the adsorbent is denoted by $q$, and the axial dispersion coefficient is represented by $D_L$ (Carvalho et al. 2019, Chu 2010),

$$\frac{\partial c}{\partial t}-D_L\frac{{\partial }^{2}c}{\partial z^2}+u_0\frac{\partial c}{\partial z}+\left(\frac{1-\epsilon }{\epsilon }\right){\rho }_L\frac{\partial q}{\partial t}=0$$(1)

The variation in adsorption capacity over time is governed by the mass transfer of the solute from the liquid phase to the surface of the adsorbent, as well as the deposition or weak binding of solute molecules on the adsorbent surface. The overall adsorption rate, $r_a$, is described by a linear driving force model, in which the overall mass transfer coefficient of the liquid phase plays a major role. The model considered is analogous to those used in the works of Muzic et al. (2010), Carvalho et al. (2019), Chern & Chien (2002),

$$\frac{\partial q}{\partial t}=r_a=\frac{3k_e}{r_p}(c-c_{eq})$$(2)

In Eq. (2), the parameter $k_e$ is the mass transfer coefficient in the liquid film surrounding the particle, $a_v=3/r_p$ is the surface area of the adsorbent particle, and $c_{eq}$ is the equilibrium concentration in the liquid phase. For this particular problem, after several experiments, Carvalho et al. (2019) estimated the value of $c_{eq}$ to be 427.8 mg/L.

While the value remains deterministic, as indicated by Guimarães & Leão (2014a), Carvalho et al. (2019), where the adsorption kinetics of sulfate ions in the resin are notably rapid. In this work, given the emphasis on neural network utilization, this simplification has been embraced. Although studies such as those conducted by Oliveira et al. (2023), Jurado-Davila et al. (2023), Nunes et al. (2021) have employed the Bayesian Markov Chain Monte Carlo method to estimate parameters and explore various adsorption isotherms from existing literature, this stands as a prospective area for future consideration in the integration of neural networks and online estimations employing sequential Monte Carlo methods.

The initial and boundary conditions for the column, which initially has no solute adsorbed and is subjected to an abrupt change in adsorbed concentration at the column inlet at time zero, are described by Eq. (3), Eq. (5) and Eq. (6), as presented by Carvalho et al. (2019), Chu (2010),

$$\begin{array}{ccc}\hfill \mathrm{\text{Initial condition}}& t=0\hfill & \hfill c(z,0)=0\end{array}$$(3)

$$\begin{array}{ccc}\hfill & t=0\hfill & \hfill q(z,0)=0\end{array}$$(4)

$$\begin{array}{ccc}\hfill \mathrm{\text{Dankwerts boundary condition}}& z=0\hfill & \hfill \frac{\partial c}{\partial z}=\frac{u_0}{D_L}(c-c_0)\end{array}$$(5)

$$\begin{array}{ccc}\hfill \mathrm{\text{Neumann boundary condition}}& z=L\hfill & \hfill \frac{\partial c}{\partial z}=0\end{array}$$(6)

where $c_0$ represents the inlet concentration of sulfate. Aiming to ensure dimensional homogeneity and simplify the physical problem, Carvalho et al. (2019) proposed a scaled model by introducing the following variables,

$$x=\frac{z}{\mathrm{\text{L}}}\tau =\frac{u_{0}t}{\mathrm{\text{L}}}\bar{q}=\frac{q{\rho }_L}{c_{in}}$$(7)

where $\tau$ represents the dimensionless time, $x$ represents the dimensionless length. Substituting the new variable in Eq. (1), also in initial and boundary conditions, the dimensionless governing equation to be solved by PINN is given by

$$\frac{\partial c}{\partial \tau }-\frac{1}{\mathrm{\text{Pe}}}\frac{{\partial }^{2}c}{\partial x^2}+\frac{\partial c}{\partial x}+\left(\frac{1-\epsilon }{\epsilon }\right){\rho }_L\frac{\partial \bar{q}}{\partial \tau }=0$$(8)

The following dimensionless constitutive equations were considered to calculate the mass transfer coefficient in the liquid film and axial dispersion, (see Table I)

Table I
Dimensionless numbers used in scaled governing equation.

Dimensionless	Expression
Reynolds Number	Re $=\frac{{\rho }_L{V}_s{d}_p}{{\mu }_m}$
Schimidt Number	Sc $=\frac{{\mu }_m}{{\rho }_L{D}_m}$
Peclet Number	Pe $=\frac{u_{0}L}{D_L}$
Sherwood Number	Sh $=2+1.1{\mathrm{\text{Re}}}^{0.6}{\mathrm{\text{Sc}}}^{1/3}$

The mean superficial velocity in columns is calculated as a ratio between flow rate (Q) and cross-sectional area (S), and the interstitial velocity ($u_0$) is expressed in Eq. (9)

$$V_s=\frac{Q}{S},u_o=\frac{V_s}{\epsilon }$$(9)

The transport diffusion coefficient was obtained in terms of molecular diffusivity Eq. (10), where $Z$ represents the positive and negative values of the charges of the involved ions, and $D_{N{a}^{+}}^0$ and $D_{S{O}_4^{2-}}^0$ represent the diffusivity of $N{a}^{+}$ and $S{O}_4^{-}$ ions in aqueous solution.

$$D_m=\frac{(Z_{+}+\left|Z_{-}\right|)D_{N{a}^{+}}^0+D_{S{O}_4^{-}}^0}{Z_{+}{D}_{N{a}^{+}}^0+\left|Z_{-}\right|D_{S{O}_4^{-}}^0}$$(10)

and further by relations established by Ruthven (1984)

$$D_L=\left(u_0{d}_p\right)\frac{20}{\epsilon }\frac{D_m}{u_0{d}_p}+\frac{1}{2}$$(11)

Finally the mass transfer coefficient in liquid film is stated as,

$$k_e=\frac{\mathrm{\text{Sh}}{D}_m}{d_p}$$(12)

Figure 1 illustrates the process represented by an upward flow fixed-bed column considered in this study, which represents the experimental setup conducted by Guimarães & Leão (2014a). The experimental apparatus consists of a reservoir with sulphate concentration, a peristaltic pump and a fixed-bed column. The column is filled with the resin Purolite A500. The operational parameters of column and technical information about Purolite A500 are showed in Table II.

Figure 1
A schematic setup representation of sulfate adsorption system in a fixed-bed column.

Table II
Operating parameters of the column and the process.

Parameters	Symbol	Value
Cross-sectional area, [m]	S	0.000531
Length of bed, [m]	L	0.06/0.09/0.12
Porosity, [-]	$\epsilon$	0.35
Particle diameter, [m]	$d_p$	0.0007
Density of liquid, [kg m-3]	${\rho }_L$	1000
Density of adsorbent, [kg m-3]	${\rho }_s$	1134
Mixture viscosity, [kg m-1 s-1]	${\mu }_m$	89.15 10-6
Inlet concentration, [kg m-3]	$c_0$	0.17
Flow rate, [m3 s-1]	Q	8.50 10-9
Equilibrium constant, [kg m-3]	$c_{eq}$	0.4278
Diffusion coeff. [m2 s-1]	$D_{N{a}^{+}}^0$	1.34 10-9
Diffusion coeff. [m2 s-1]	$D_{S{O}_4^{2-}}^0$	1.06 10-9

Physics-Informed Neural Network (PINN) Model

The unknown solution of the partial differential equation (PDE) $y(x,t)$ can be approximated through a deep neural network $\hat{y}(x,t,\theta )$, where $\theta$ represents the trainable parameters of the network. This type of approach allows for the construction of physics-informed neural networks. Consider a general partial differential equation, with a spatio-temporal domain $(x,t)\in \Omega \subset {ℝ}^D$.

$${\partial }_{t}y(x,t)+𝒩\left[y\right](x,y)=0,x\in \Omega ,t\in [0,T]$$(13)

$$B\left[y\right](x,y)=0,x\in \partial \Omega$$(14)

where ${\partial }_{t}y(x,t)$ denotes the time derivative of $y$, $𝒩\left[y\right]$ is a nonlinear differential operator, and $B\left[y\right]$ represents the boundary conditions. The approximation by deep neural network is performed in the form of a residual and is denoted as $\hat{y}(x,t,\theta )$.

In general, for the application of deep learning, a sampling of points is performed in the domain $\left(N_f\right)$, on the boundaries $\left(N_{bc}\right)$, and at the initial condition $\left(N_i\right)$. These points are used to calculate the global residual function $ℒ$ as follows:

$$ℒ\left(\theta \right)=w_f{\mathrm{\text{MSE}}}_f+w_{bc}{\mathrm{\text{MSE}}}_{bc}+w_{i}MS{E}_i$$(15)

where $w$ represents the weights for each set. In a general form, the loss function can be written as:

$${\mathrm{\text{MSE}}}_f=\frac{1}{N_f}\sum _{(x,t)\in N_f}{\left|f(\hat{y},x,t)\right|}^2$$(16)

$${\mathrm{\text{MSE}}}_{bc}=\frac{1}{N_{bc}}\sum _{(x,t)\in N_{bc}}{\left|g_{bc}(\hat{y},x,t)\right|}^2$$(17)

$${\mathrm{\text{MSE}}}_i=\frac{1}{N_i}\sum _{(x,t)\in N_i}{|\hat{y}(x,0)-u(x,0)|}^2$$(18)

By minimizing the sum of the equations mentioned above, represented by the global loss function $ℒ$, using an appropriate optimizer or a combination of optimizers, the parameters $\theta$ of the neural network $\hat{y}(x,t,\theta )$ can be found to minimize the loss function. This process leads to obtaining the solution of the partial differential equation (PDE) (Raissi et al. 2019, Santana et al. 2022).

PINN Architecture

For this case, the neural network receives three distinct inputs: the spatial variable $x$, the temporal variable $t$, and the previous state $y_0$. These inputs are essential for calculating the desired output, representing the concentration output. Figure 2 below clearly illustrates the relationship between the inputs and the output of the neural network.

Figure 2
Illustration of the inputs and output of the neural network.

The output of the network is given by:

$$y\left(t_k\right)=\hat{f}(x_{\mathrm{\text{obs}}},t_k,y_{k-1})$$(19)

where $x_{\mathrm{\text{obs}}}$ represents the spatial point at which the measurement is taken. In column adsorption, it is commonly adopted to measure at the outlet point, i.e., at x=1. The following loss functions were considered:

$${\mathrm{\text{MSE}}}_f=\frac{1}{{\mathrm{\text{N}}}_f}\sum _{(x,\tau \in {\mathrm{\text{N}}}_f)}{|\frac{\partial \hat{y}}{\partial \tau }-\frac{1}{Pe}\frac{{\partial }^2\hat{y}}{\partial x^2}+\frac{\partial \hat{y}}{\partial x}+\left(\frac{1-\epsilon }{\epsilon }\right){\rho }_L\frac{\partial \bar{q}}{\partial \tau }|}^2$$(20)

$${\mathrm{\text{MSE}}}_{bcl}=\frac{1}{{\mathrm{\text{N}}}_{bcl}}\sum _{(x,\tau \in {\mathrm{\text{N}}}_{bcl})}{|\frac{\partial \hat{y}(0,t)}{\partial x}-Pe(\hat{y}(0,t)-C_0)|}^2$$(21)

$${\mathrm{\text{MSE}}}_{bcr}=\frac{1}{{\mathrm{\text{N}}}_{bcr}}\sum _{(x,\tau \in {\mathrm{\text{N}}}_{bcr})}{\left|\frac{\partial \hat{y}(1,t)}{\partial x}\right|}^2$$(22)

It was proposed to include this value as an input to the network to allow the neural network to consider the concentration value at the previous time step. To achieve this, a hard boundary was adopted, which modifies the network’s output according to the following equation:

$$t_{\alpha }=\frac{e^{a{t}_n}-e^{-a{t}_n}}{e^{a{t}_n}+e^{-a{t}_n}},\mathrm{\text{where}}\{\begin{array}{cc}{t}_n=0,\hfill & t_{\alpha }=0\hfill \\ t_n\ne 0,\hfill & t_{\alpha }=1\hfill \end{array}$$(23)

$${\hat{y}}_f=t_{\alpha }[(1-b)\hat{y}+b{y}_0]$$(24)

In Eq.(24), the neural network output forced by the hard condition is represented by ${\hat{y}}_f$. It is important to note that $t_{\alpha }$ is an auxiliary time introduced to ensure the satisfaction of the zero initial condition. In the steady state, the network’s output equals the previous condition. The parameters a and b have specific values of $a=10$ and $b=0.5$, respectively.

Alternative Numerical Solution

The finite volume method (FVM) was proposed to solve the numerical model. The methodOn the other hand, the second method is based on the conservation of mass, energy, or quantity for each finite volume. This approach involves discretizing a set of control volumes and applying the conservation laws to all elemental volumes. The final result consists of a set of coupled algebraic equations that can be solved. These equations are obtained by integrating the differential equations over each finite volume and applying the boundary conditions (Mazumder 2016).

Particle Filter for State-Space Estimation

The particle filter is an estimation approach that tackles problems that are both nonlinear and non-Gaussian in nature. By employing the Monte Carlo method and a large number of samples, it becomes feasible to obtain an equivalent representation of the posterior probability function, thereby approximating the optimal estimation according to the Bayesian approach (Tulsyan et al. 2016).

Given that the state at a particular time instant $x_k$, where $x\in {ℝ}^N$, is a function of the previous state $x_{k-1}$ and a noise component $v_k$, where $v\in {ℝ}^N$, we also consider that the vector of measurements observed at a time instant $z_k$, where $z\in {ℝ}^M$, depends on both the state vector and the noise associated with the measurement ($n_k$). Therefore, we have the evolution and observation models, mathematically represented by the Eq. (25) and (26).

$$\begin{array}{c}\hfill x_k=f(x_{k-1},v_{k-1})\end{array}$$(25)

$$\begin{array}{c}\hfill z_k=h(x_k,n_k)\end{array}$$(26)

In this article, the measurements will be initially simulated synthetically, calculated using the finite volume method with the Eq. (27), and later experimental data from Guimarães & Leão (2014a) will be used.

$$z_k^{obs}=x_k+\sigma ϵ$$(27)

Here, $x$ is obtained by solving the direct problem using the finite difference method, $\sigma$ is the standard deviation of the measurements, and $\epsilon$ is a random variation with a normal distribution $\sim 𝒩(0,\sigma )$.

To implement the particle filter algorithm sampling importance resampling (SIR), the model by Tulsyan et al. (2016) was used, which outlines the fundamental steps for implementation. The summary of the stages in this article is shown below:

1. Start.

2. Construct $N$ particles denoted as $x_k^i$ such that $[i=1,...,N]$ distributed according to the initial state density.

3. for $t=1$ to $t_f$:

4. Predict the new $N$ particles using the evolution model fed by the prior particles.

5. Use the likelihood to calculate the corresponding weights $w_k^i$.

6. Calculate the total weight $T_w={\sum }_{i=1}^N{w}_k^i$, then normalize the particle weights by $w_k^i=T_w^{-1}{w}_k^i$. Also, calculate the sample degeneracy as $N_{eff}={\left[{\sum }_{i=1}^N{w_k^i}^2\right]}^{-1}$.

7. Perform resampling.

8. Calculate the sample mean after resampling.

9. End.

After implementing the filter, an analysis was proposed to evaluate the performance of the PINN as an evolution model. This analysis involved considering the number of particles ($N_p$), sample degeneracy ($N_{\mathrm{\text{eff}}}$), the Root Mean Square (RMS) metric described by Eq. (28), and the Total Error Reduction (TER) criterion, which quantifies the filter’s performance in reducing measurement uncertainty. This analysis aims to find the optimal configuration for the filter’s operation. Next, a comparison was made between the approach proposed in this article and the widely used finite volume method.

$$\mathrm{\text{RMS}}=\sqrt{\frac{1}{\mathrm{\text{N}}}\sum _{i=1}^{\mathrm{\text{N}}}(C_{\mathrm{\text{meas}}}^i-C_{\mathrm{\text{est}}}^i)}$$(28)

$$\mathrm{\text{TER}}=\left(\frac{\sqrt{\sum _{i=1}^{\mathrm{\text{N}}}\left(\frac{{(C_{\mathrm{\text{meas}}}^i-C_{\mathrm{\text{exa}}}^i)}^2}{{\sigma }^2}\right)}-\sqrt{\sum _{i=1}^{\mathrm{\text{N}}}\left(\frac{{(C_{\mathrm{\text{est}}}^i-C_{\mathrm{\text{exa}}}^i)}^2}{{\sigma }^2}\right)}}{\sqrt{\sum _{i=1}^{\mathrm{\text{N}}}\left(\frac{{(C_{\mathrm{\text{meas}}}^i-C_{\mathrm{\text{exa}}}^i)}^2}{{\sigma }^2}\right)}}\right)$$(29)

To succinctly outline the utilization of the methodologies delineated in preceding sections for addressing the challenge of filter uncertainties within the sulfate adsorption column’s breakthrough curve, a two-block flowchart (Figure 3) has been constructed. Block 1 elucidates the process of producing synthetic measurements through the finite volume method. Meanwhile, Block 2 delineates the implementation of data assimilation, employing PINN as the transitional model for the particle filter algorithm.

Figure 3
Framework of the proposed PF-PINN. Blue box depicts the generation of synthetic measurements by adding a Gaussian noise to FVM solution. Gray box shows the sequential estimation algorithm using PINN as transition model for data assimilation of measurements $\tilde{z}$.

RESULTS AND DISCUSSION

Training Progress of Neural Network with Informed Physics

The physics-informed neural network (PINN) was implemented using the DeepXDE framework available from Lu et al. (2021). The architecture used consists of 5 layers, each composed of 50 neurons. The architecture was chosen based on a series of tests to verify the architecture’s convergence, in which the network’s performance varied according to the number of layers and the number of neurons per layer, evaluating the Root Mean Squared Error (RMS) metric. In Figure 4, the color gradient indicates a region of good network performance when solving the model to approximate the data from Guimarães & Leão (2014a). This region occurs with 5 to 8 layers and 45 to 60 neurons per layer. The architecture used falls within this region. Other configurations with a higher number of layers (9+) and neurons (65+) were not considered, as the network already demonstrated low RMS in these tests, meeting the requirements to be incorporated into the particle filter.

Figure 4
Convergence archtecture.

In all cases, hyperbolic tangent (tanh) activations were used in each layer. All terms of the loss function were weighted equally with a weight of 1.0. The dataset used in the experiment consisted of 1500 points for the partial differential equation residual ($X_f$), 550 points for the boundary conditions ($X_b$), and 700 points in the domain for testing. The Latin hypercube sampling (LHS) strategy was employed to obtain these points (McKay et al. 1979).

The loss function was minimized by running 60000 iterations of the Adam algorithm Kingma & Ba (2014) with a learning rate of ${10}^{-3}$, followed by the L-BFGS-B algorithm Byrd et al. (1995) until convergence was achieved.

Figure 5 illustrates the training progress of the neural network in terms of the number of epochs for the architecture with 5 layers and each composed of 50 neurons. Initially, a rapid decrease in loss is observed, accompanied by oscillations throughout 60000 epochs. Subsequently, the loss stabilizes around ${10}^{-3}$. When using the L-BFGS-B algorithm, an additional reduction in loss is noticeable, reaching a final value on the order of ${10}^{-4}$.

Figure 5
Training progress of the neural network.

It is important to note that the evaluated points along the domain differ from those used in the training process. Furthermore, it is noteworthy that, for all considered points, the loss function reaches the order of ${10}^{-4}$, indicating that the physical model is respected within the observed domain.

Validation and Comparison of Proposed Solutions

The spatial mesh was discretized into 100 equally spaced points to perform the numerical solutions. The temporal mesh was defined based on the experimental data collection time suggested by Guimarães & Leão (2014a), resulting in a fixed time spacing of 10 minutes.

The experimental measurements have associated errors. To quantify this uncertainty, a 5% deviation in the measurements of sulfate concentrations at the bed outlet was considered. Considering that the sulfate concentrations range from 0 to 0.17, the equipment used, ICP-OES (Spectro, CCD Cirus), has a 5% margin of error relative to the maximum concentration value. This information was obtained from the specifications provided by the measuring equipment Guimarães & Leão (2014a). Therefore, the trust region is represented by yellow shading.

Table III
Analysis of results proximity based on root mean squared metrics for three bed length conditions.

2-3	Root Mean Squared
Length [cm]	FVM	PINN
6	0.0488	0.0299
9	0.0749	0.0360
12	0.0888	0.0455

From Figure 6 and Table III, it can be inferred that the solutions are consistent with the laboratory data collected for the three column lengths. It is noted that the proposed methods showed low RMS despite the coarse temporal mesh. The PINN approach was the one that best approximated the experimental data. Its main advantage lies in not following mesh stability criteria. Therefore, using a larger time interval did not compromise the solution. The opposite was observed in the finite volume method, where the distance between mesh points is of paramount importance for converging the results.

Figure 6
Comparison between FVM, PINN and experimental data Guimarães & Leão (2014a) for three bed lengths.

Application of Particle Filter

As a case study for filter application, two scenarios were investigated. The first scenario involved controlled experiments using synthetic measurements, while the second scenario utilized experimental data of sulfate adsorption at the column outlet collected by Guimarães & Leão (2014a) as the observation model.

The primary objective of the following cases was to determine the optimal filter configuration to be applied in the experimental bench-scale case. In the synthetic scenario, the measurements were subject to uncertainty with a standard deviation ranging from 5% to 30% of the maximum concentration. The model discrepancy was set at 4% for all cases.

Figure 7 presents the results, showing the influence of the number of particles ($N_p$) on state estimation, root mean square error (RMS), and total error reduction (TER) computational time using the PINN as the evolution model, and comparing it with the FVM. Figure 8 demonstrates the ability to reduce computational costs of physics-informed neural network compared with finite volume method.Table III presented the results, showing the influence of the number of particles ($N_p$) on state estimation, sample degeneracy (Neff), total error reduction (TER), and computational time using the PINN as the evolution model, and compared with the FVM.

Figure 7
Convergence analysis of filtered states using measurements uncertainty ranging from 5 - 30%, for the PINN and FVM methods.

Figure 8
Comparative of CPU time to solve using particle filter in adsorption problem.

Considering both low and high measurement uncertainties, the cases explored, as illustrated in Figure 8, provided a more in-depth analysis of the particle filter’s effectiveness. It is worth noting that in situations of low measurement uncertainties, the filter is limited since the high reliability of the measurements results in less flexibility and robustness of the filter in determining the filtered measurement compared to scenarios of high uncertainties. This practical limitation is related to calculating the likelihood function, as the reduction in measurement uncertainty implies a lower tolerance, which in turn restricts the number of viable solutions. Consequently, the filter performed slightly better in environments with high uncertainties when compared to situations with low uncertainties. It can be seen that the root mean square (RMS) between the filtered measurement and the exact one is low, in the order of ${10}^{-3}$. Therefore, implementing the filter provided satisfactory results at all magnitudes of measurement noise. This characteristic is also associated with the effectiveness of the evolution model, as indicated by the low uncertainty incorporated into the model. A robust evolution model is essential in the filtering process, as increasing the model’s uncertainty makes the filtered state closer to the measurement, thus compromising the filtering potential.

It can be inferred that the optimal filter configuration is 500 particles and increase the number of particles brings no further improvement in analyzed metrics. Additionally, the computational cost becomes significant when using a larger number of particles, while the benefit remains negligible, as indicated by the constant RMS and TER values in both cases. When comparing the computational time between the two strategies, using the PINN resulted in a 65% reduction in computational time for the selected configuration. This time difference arises from the fact that the finite volume method requires solving a system of equations of size $N\times N$ for each particle $N_p$ at each time step $M$. On the other hand, when using the PINN, predictions are made for each particle $N_p$ at each time step $M$, and the network has already been trained, eliminating the cost associated with solving the system of equations and resulting in a relatively lower computational cost.

Figure 9a illustrates the performance of the SIR filter using 500 particles in both scenarios. For the first case, where measurements were synthetically generated with controlled errors, it was possible to obtain the exact error-free curve as the solution from the finite volume method (FVM). Notably, the filter successfully estimated the concentration and filtered out a significant portion of the measurements, as some fall outside the confidence limit. Therefore, it can be inferred that the measures improved after being processed by the filter. The real-world application is depicted in Figure 9b, where the filter estimated the concentrations but did not show statistically significant improvement in the measurements, as they all fall within the confidence region. However, it is observed that a range of measurements between 150 to 190 minutes lies close to the limit, suggesting that the proposed filtered state would serve as a valid correction for those measurements.

Figure 9
a) Application of the filter using the PINN as an evolutionary model in the controlled scenario to estimate and filter the concentration of $𝐒{𝐎}_{\mathrm{𝟒}}^{\mathrm{𝟐}\mathbf{-}}$ at the outlet of a fixed-bed column with an SIR filter, using 500 particles and a model uncertainty of $\mathrm{𝟒}\mathbf{\%}$. b) Application of the filter in the real-life scenario.

Another crucial aspect to consider is the effective particles. In Figure 10, it can be observed that the majority of particles provided useful information about the posterior probability distribution. This indicates that the degeneration effect was minimal, revealing the regions where the filter faced the most difficulty representing this distribution, particularly in areas where the curve slope changes sharply and the measurements deviate from the ideal. The filter encounters more significant challenges in describing the posterior information in such cases.

Figure 10
Effective sample size during the estimation procedure.

CONCLUSIONS

In general, the findings of this study highlight the potential of physics-informed neural networks (PINN) implemented recursively and the SIR filter in modeling and estimating the behavior of sulfate adsorption in fixed-bed columns. PINN demonstrated superior accuracy compared to traditional methods, while the SIR filter effectively improved estimates by incorporating measurement uncertainties. These findings contribute to advancing computational techniques in solving complex engineering problems and have practical implications for optimizing the design and operation of sulfate removal processes.

Numerical experiments demonstrated the effectiveness of PINNs in capturing the underlying physics of the problem and accurately predicting sulfate concentrations at the column outlet. The results indicated that PINN achieved the best agreement with experimental data, exhibiting low root mean square error (RMS) values despite the coarse temporal mesh.

Applying the SIR filter, using PINN as the evolution model, was investigated in a controlled scenario with synthetic measurements. The filter successfully estimated the sulfate concentration at the column outlet, filtering out measures that fell outside the confidence limits. Analysis of the effective sample size demonstrated that most particles provided useful information, indicating low degeneracy. Regarding computational time, employing the PINN as the evolution model in the particle filter led to a 65% reduction in the computational time of the tool in the analyzed scenarios.

Furthermore, a real-world application of the SIR filter using experimental data validated the approach’s effectiveness in estimating the breakthrough curve for sulfate adsorption in a fixed-bed column. The estimated concentration values were satisfactory, and all measurements were within the 95% confidence region.

ACKNOWLEDGMENTS

This paper was funded by the following Brazilian agency: Fundação de Amparo à Pesquisa e Inovação do Espírito Santo (FAPES).

References

• BYRD RH, LU P, NOCEDAL J & ZHU C. 1995. A Limited Memory Algorithm for Bound Constrained Optimization. SIAM J on Sci Comput 16(5): 1190-1208. doi:10.1137/0916069.
• CARVALHO RNS, GUIMARÃES D, LEÃO VA, DUTRA JCS & SILVA WB. 2019. Estimation of parameters and states using a bayesian particle filter for the sulfate ion adsorption process in a fixed bed column. Braz J Chem Eng 36(3): 1223-1236. doi:10.1590/0104-6632.20190363s20180059.
• CHAPRA SC & CANALE RP. 2021. Numerical methods for engineers. McGraw-Hill, 8th edition.
• CHERN JM & CHIEN YW. 2002. Adsorption of nitrophenol onto activated carbon: isotherms and breakthrough curves. Water Res 36(3): 647-655. doi:10.1016/s0043-1354(01)00258-5.
• CHU KH. 2010. Fixed bed sorption: Setting the record straight on the Bohart-Adams and Thomas models. J Hazard Mater 177(1-3): 1006-1012. doi:10.1016/j.jhazmat.2010.01.019.
• GUIMARÃES D & LEÃO VA. 2014a. Fundamental aspects related to batch and fixed-bed sulfate sorption by the macroporous type 1 strong base ion exchange resin Purolite A500. J Environ Manag 145: 106-112. doi:10.1016/j.jenvman.2014.06.006.
• GUIMARÃES D & LEÃO VA. 2014b. Batch and fixed-bed assessment of sulphate removal by the weak base ion exchange resin Amberlyst A21. J Hazard Mater 280: 209-215. doi:10.1016/j.jhazmat.2014.07.071.
• GUO H, ZHUANG X, CHEN P, ALAJLAN N & RABCZUK T. 2022. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Eng Comput 38(6): 5173-5198. doi:10.1007/s00366-021-01586-2.
• JURADO-DAVILA IV, SCHNEIDER IAH, ESTUMANO D & FÉRIS LA. 2023. Phosphate removal using dolomite modified with ultrasound: mathematical and experimental analysis. J Environ Sci Heal A 58(5): 469-482. doi:10.1080/10934529.2023.2196928.
• KINGMA DP & BA J. 2014. Adam: A method for stochastic optimization. Available at: https://arxiv.org/abs/1412.6980
  » https://arxiv.org/abs/1412.6980
• LU L, MENG X, MAO Z KARNIADAKIS GE. 2021. DeepXDE: A deep learning library for solving differential equations. SIAM Rev 63(1): 208-228. doi:10.1137/19M1274067.
• MAZUMDER S. 2016. Introduction to numerical methods for solving differential equations. In: Numerical methods for partial differential equations. Elsevier, 1st edition, p. 1-49.
• MCKAY MD, BECKMAN RJ CONOVER WJ. 1979. A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code. Technometrics 21(2): 239. doi:10.2307/1268522.
• MUZIC M, GOMZI Z SERTIC-BIONDA K. 2010. Modeling of the Adsorptive Desulfurization of Diesel Fuel in a Fixed-Bed Column. Chem Eng Technol 33(7): 1137-1145. doi:10.1002/ceat.201000013.
• NGO SI LIM Y. 2021. Solution and Parameter Identification of a Fixed-Bed Reactor Model for Catalytic CO2 Methanation Using Physics-Informed Neural Networks. Catalysts 11(11): 1304. doi:10.3390/catal11111304.
• NGO SI LIM Y. 2022. Forward physics-informed neural networks for catalytic CO2 methanation via isothermal fixed-bed reactor. Comput Aided Chem Eng 49: 1675-1680. doi:10.1016/B978-0-323-85159-6.50279-7.
• NGUYEN-THANH VM, ANITESCU C, ALAJLAN N, RABCZUK T ZHUANG X. 2021. Parametric deep energy approach for elasticity accounting for strain gradient effects. Comput Methods Appl Mech Eng 386: 114096. doi:10.1016/j.cma.2021.114096.
• NUNES KGP, DÁVILA IVJ, AMADOR ICB, ESTUMANO DC FÉRIS LA. 2021. Evaluation of zinc adsorption through batch and continuous scale applying Bayesian technique for estimate parameters and select model. J Environ Sci Heal A 56(11): 1228-1242. doi:10.1080/10934529.2021.1977059.
• OLIVEIRA JT, ARSUFI ABL, ESTUMANO DC FÉRIS LA. 2023. Bayesian computational technique for modeling caffeine adsorption in a fixed-bed column: use of the maximum adsorption capacity deterministically and experimental design. Ind Eng Chem Res 62(18): 7127-7137. doi:10.1021/acs.iecr.3c00303.
• QUEIROZ LH, SANTOS FP, OLIVEIRA JP SOUZA MB. 2021. Physics-Informed deep learning to predict flow fields in cyclone separators. Digit Chem Eng 1: 100002. doi:10.1016/j.dche.2021.100002.
• RAISSI M, PERDIKARIS P KARNIADAKIS GE. 2017a. Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations. doi:10.48550/ARXIV.1711.10561.
• RAISSI M, PERDIKARIS P KARNIADAKIS GE. 2017b. Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations. doi:10.48550/arXiv.1711.10566.
• RAISSI M, PERDIKARIS P KARNIADAKIS GE. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378: 686-707. doi:10.1016/j.jcp.2018.10.045.
• RUTHVEN DM. 1984. Principles of adsorption and adsorption processes. John Wiley Sons, 464 p.
• SAMANIEGO E, ANITESCU C, GOSWAMI S, NGUYEN-THANH VM, GUO H, HAMDIA K, ZHUANG X RABCZUK T. 2020. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Comput Methods Appl Mech Eng 362: 112790. doi:10.1016/j.cma.2019.112790.
• SANTANA VV, GAMA MS, LOUREIRO JM, RODRIGUES AE, RIBEIRO AM, TAVARES FW, BARRETO AG NOGUEIRA IBR. 2022. A First Approach towards Adsorption-Oriented Physics-Informed Neural Networks: Monoclonal Antibody Adsorption Performance on an Ion-Exchange Column as a Case Study. Chem Eng 6(2): 21. doi:10.3390/chemengineering6020021.
• SEL K, MOHAMMADI A, PETTIGREW RI JAFARI R. 2023. Physics-informed neural networks for modeling physiological time series for cuffless blood pressure estimation. NPJ Digit Med 6(1): 6. doi:10.1038/s41746-023-00853-4.
• SEREBRENNIKOVA A, TEUBLER R, HOFFELLNER L, LEITNER E, HIRN U ZOJER K. 2022. Transport of Organic Volatiles through Paper: Physics-Informed Neural Networks for Solving Inverse and Forward Problems. Transp Porous Media 145(3): 589-612. doi:10.1007/s11242-022-01864-7.
• THONGSAMER T, VINITNANTHARAT S, PINISAKUL A WERNER D. 2023. Fixed-bed biofilter for polluted surface water treatment using chitosan impregnated-coconut husk biochar. Environ Pollut 334: 122137. doi:10.1016/j.envpol.2023.122137.
• TULSYAN A, GOPALUNI RB KHARE SR. 2016. Particle filtering without tears: a primer for beginners. Comput Chem Eng 95: 130-145. doi:10.1016/j.compchemeng.2016.08.015.
• WU X, WANG C, CAO Z CAI W. 2023. Deep learning model of fixed bed adsorption breakthrough curve hybrid-driven by data and physical information. Huagong Xuebao/CIESC J 74(3): 1145-1160. doi:10.11949/0438-1157.20221609.

Publication Dates

• Publication in this collection
  25 Nov 2024
• Date of issue
  2024

History

• Received
  19 Mar 2024
• Accepted
  7 July 2024