Contact Failure Identification in Multilayered Media via Artificial Neural Networks and Autoencoders

Abstract

The estimation of defects positioning occurring in the interface between different materials is performed by using an artificial neural network modeled as an inverse heat conduction problem. Identifying contact failures in the bonding process of different materials is crucial in many engineering applications, ranging from manufacturing, preventive inspection and even failure diagnosis. This can be modeled as an inverse heat conduction problem in multilayered media, where thermography temperature measurements from an exposed surface of the media are available. This work solves this inverse problem with an artificial neural network that receives these experimental data as input and outputs the thermalphysical properties of the adhesive layer, where defects can occur. An autoencoder is used to reduce the dimension of the transient 1D thermography data, where its latent space represents the experimental data in a lower dimension, then these reduced data are used as input to a fully connected multilayer perceptron network. Results indicate that this is a promising approach due to the good accuracy and low computational cost observed. In addition, by including different noise levels within a defined range in the training process, the network can generalize the experimental data input and estimate the positioning of defects with similar quality.

Key words
Inverse heat conduction problem; artificial neural networks; contact failure identification; applied artificial intelligence; denoising autoencoder

Introduction

Heat transfer in multilayered media is a phenomenon that can be fundamentally important in different applications of science and engineering, such as manufacturing of composite materials (Danes et al. 2003), oil and gas industry (Su et al. 2012), thermal based health treatments (Sarkar et al. 2015), aerospace engineering (Jayaseelan et al. 2015) and many others. More specifically, defect identification in the process of bonding among different materials can be formulated as a multilayered media and solved as an Inverse Heat Conduction Problem (IHCP), where estimating the positioning of air gaps, oil bubbles, and cracks, usually is performed by estimating thermophysical properties of the adhesive layer. Therefore, estimating heterogeneity in this layer may be crucial to evaluate the bonding quality (Abreu et al. 2014, 2016, Mascouto et al. 2020).

In classical approaches, IHCPs can be treated as optimization problems, where a least squares functional must be minimized (Beck et al. 1985, Moura Neto & Silva Neto 2013), or as bayesian inference problems, such as the Maximum a Posteriori and the Markov Chain Monte Carlo methods (Kaipio & Fox 2011, Orlande 2012). Since these types of IHCPs formulations require intensive iterative procedures, many efforts of the scientific community are focused on reducing the computational costs related to techniques, algorithms and mathematical formulations (Orlande et al. 2014, Vakili & Gadala 2009). Parallel to these efforts, the use of computational intelligence emerged as a possible approach for solving IHCPs, such as Genetic Algorithms (Garcia et al. 1998, Imani et al. 2006) and Artificial Neural Networks (ANN) (Balaji & Padhi 2010, Czél et al. 2014, Shiguemori et al. 2004, Soeiro et al. 2004), which are considered soft computing approaches, as opposed to classical ones.

The use of ANNs has been permeating the most eclectic types of applications, such as engineering, financial analysis, image and speech recognition, medicine, among many others. This success can be credited to several factors, one of them being the notable improvements and developments of different ANN architectures, such as Autoencoders (AE), Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), Extreme Learning Machines (ELM), and so on. The use of any of these architectures must take into consideration its specific features along with characteristics of the problem being solved.

Recently, several of these architectures emerged as viable techniques for defect characterization using IHCPs with infrared thermography. Hu et al. ( 2019) employed a long short term memory RNN for automatically classifying different types of common defects occurring in honeycomb materials. Later, Fang & Maldague ( 2020) used a Gated Recurrent Units (GRU) for depth prediction in composite material samples. Kaur et al. ( 2021) introduced a novel Constrained Autoencoder in order to reduce the dimensionality of temporal thermographic images for defect depth estimation in steel plates. Marani et al. ( 2021) used step-heating thermography combined with CNN in order to describe defects in composite laminates. Xu & Hu ( 2021) performed defect depth identification by using a method based also on GRUs.

Besides the cited innovation in networks architectures, the most widely used in the field of heat transfer engineering is the simple feed-forward back propagation Multilayer Perceptron (MLP). Indeed, this network has been proved to be a great general purpose approximator in countless applications, but it lacks some advantages that a specific deep neural network, such as CNNs, AEs, RNN and others may give.

In this context, AEs (or Replicator networks) are great tools designed for reducing the dimension of any given data (Haykin 2009, Hinton & Salakhutdinov 2006). Such network have the input equal to the output, but it main aspect is a bottleneck-like hidden layer (latent space) which is smaller in dimension than the input-output. When properly trained, the original data can be encoded into this smaller dimension latent space, then the information in this space can be decoded back into the original information size. Furthermore, AEs can be specially designed as a noise reduction network, also known as Denoising Autoencoders (DAE). This can be performed by giving an noisy data as input, passing it through the latent space, then outputting a noiseless version of that same data. If well trained, the information stored in the latent space is an almost-noiseless representation of the original data.

In this work the IHCP of estimating the spatially varying thermal properties of multilayered media is solved via an ANN, where the input data are temperature measurements of a exposed surface and the output is a set of discrete points representing the thermal properties of different materials. More specifically, an encode-decode architecture is used, where the normalized experimental temperature and the thermal properties are encoded separately with respectives AE, then the latent space of each one of them are used as input and output, respectively, for the multi layer perceptron (MLP) neural network. Although this approach imposes the training of three ANN (two AEs and one MLP), which yields an extra computational cost when compared to a simple MLP, the use of the AE shows advantages such as the robustness to different architectures, as presented in the obtained results.

In order to test this approach, the problem of contact failures identification was chosen, where two materials are joined with an adhesive layer. Gaps of air can form in this layer, meaning that the adhesion process is not perfect (Abreu et al. 2014, 2016, Mascouto et al. 2020). Identifying the thermal properties of this layer containing gaps is crucial to access the adhesion quality in manufacturing and preventive maintenance.

In this work, the training data set is generated by solving the direct problem, i.e., the temperature profiles are simulated and obtained for different configurations of failures. Furthermore, the temperature data containing simulated noise is used to train a DAE, where its output is the noise free version of the same data. Due to its dimensionality and noise reduction, this approach can greatly benefit the use of increasingly larger data, as presented in the results. Also, some effort goes into fine tuning the parameters of the ANN itself, such as activation functions, layer sizes, number of neurons, and so on. As mentioned, this is where the encoded-decoded approach shows its promising characteristics, by yielding robust results for different types of configurations.

Direct Problem Formulation and Solution

Consider the $L_x\times L_y$ multilayered rectangular medium presented in Fig. 1 composed by three different layers. Such representation can model the bonding of two materials, where Layer 1 and 3 are joined by an adhesive layer shown as Layer 2. In the configuration of Fig. 1, two defects are illustrated in the adhesive layer. Both left and right boundaries are considered adiabatic, the constant heat flux $q$ is prescribed at the bottom boundary, and the top one is considered exposed, with ambient temperature of $T_{\infty }$ and natural convection heat transfer coefficient $h$. Temperature measurements ${\mathbf{T}}^{\mathbf{\text{exp}}}$ are acquired at the exposed boundary, i.e., $y=L_y$.

Figure 1
Schematic representation of the multilayered medium.

Using a single-domain formulation (Mascouto et al. 2020) and taking in consideration the mentioned assumptions, the heat conduction problem can be written as

$$\frac{\partial }{\partial x}\left(k(x,y)\frac{\partial T(x,y,t)}{\partial x}\right)+\frac{\partial }{\partial y}\left(k(x,y)\frac{\partial T(x,y,t)}{\partial x}\right)=\rho (x,y)c_p(x,y)\frac{\partial T(x,y,t)}{\partial t}$$ (1a)

$${\frac{\partial T(x,y,t)}{\partial x}|}_{x=0}=0,{\frac{\partial T(x,y,t)}{\partial x}|}_{x=L_x}=0$$ (1b,c)

$$-k(x,y){\frac{\partial T(x,y,t)}{\partial y}|}_{y=0}=q,k(x,y){\frac{\partial T(x,y,t)}{\partial y}|}_{y=L_y}+hT(x,L_y,t)=h{T}_{\infty }$$ (1d,e)

$$T(x,y,0)=T_{\infty }$$ (1f)

where $h$ is the convection heat transfer coefficient at the top boundary, and $q$ is the prescribed heat flux at the bottom boundary. The materials properties $k(x,y),\rho (x,y)$, and $c_p(x,y)$ are the thermal conductivity, specific mass, and specific heat, respectively. These spatially dependent properties are modeled in such way to represent the different materials of the multilayered medium. Furthermore, the initial temperature is considered equal to the ambient temperature $T_{\infty }$.

If all the thermophysical properties and all the conditions and contact failures geometries are known, the problem described by Eq. (1) can be solved yielding the temperature profile $T(x,y,t)$. In this work this solution is obtained by using the built-in “NDSolve" routine of the Wolfram Mathematica 11.0 software. This routine is executed with the Finite Element Method option and the mesh is generated with rectangular user defined uniform cell size. This routine returns an interpolated temperature profile $T(x,y,t)$ for the whole $L_x\times L_y$ medium and for times ranging from $t=0$ to $t=t_{\mathrm{\text{max}}}$, which is a prescribed maximum time instant.

Inverse Problem Formulation

Consider that the materials of the multilayered medium presented in Fig. 1 are known, but the existence and position of possible defects in Layer 2 are unknown. In this case, the inverse problem is formulated based on the assumption that Layer 2 is composed entirely by adhesive or by adhesive with air gaps (defects). This leads to the definition of an auxiliary function $\gamma \left(x\right)$ which represents the thermal properties $k(x,y)$ and $\rho (x,y)c_p(x,y)$ of Layer 2, defined as

$$\gamma \left(x\right)=\frac{\varphi \left(x\right)-{\varphi }_{\mathrm{\text{gap}}}}{{\varphi }_{\mathrm{\text{layer}}}-{\varphi }_{\mathrm{\text{gap}}}}$$(2)

where $\varphi \left(x\right)$ is the related thermophysical property, that is, the thermal conductivity $k(x,y)$ or the volumetric heat capacity $\rho (x,y)c_p(x,y)$, of Layer 2 materials. Assuming that there is no variation in the $y$ direction within the materials of Layer 2, $\gamma \left(x\right)$ is considered only as an $x$ variable function. Also, the subscripts “gap" and “layer" represent the related material thermal properties. The function $\gamma$ is used algorithmically as a way to translate the thermal properties to the spatial dependent boolean function “gap, no-gap", “0" or “1", respectively. Furthermore, $\gamma$ can be reversely translated to the thermal properties $k(x,y)$ and $\rho (x,y)c_p(x,y)$ of Layer 2, since the values ${\varphi }_{\mathrm{\text{gap}}}$ and ${\varphi }_{\mathrm{\text{layer}}}$ are known.

In order to obtain estimates of the gaps positioning in Layer 2, an ANN is used to represent the inverse model, i.e. the network receives temperature experimental data as input and outputs a vector representing discrete points of the auxiliary function $\gamma \left(x\right)$. Experimental Data. [paragraph:simulated_data] In order to simulate real measurements of temperature, the experimental data is computationally generated, where random numbers $r$ drawn from a normal distribution centered at zero with known standard deviation ${\sigma }_{\mathrm{\text{exp}}}$ are added to solution of the direct problem, i.e.,

$$T_{i,j}=T(x_i,L_y,t_j)\mathrm{\text{with}}i=1,2,...,N,j=1,2,...,M$$ (3a)

$$T_{i,j}^{\mathrm{\text{exp}}}=T_{i,j}+r,r\sim \mathrm{\text{N}}(0,{\sigma }_{\mathrm{\text{exp}}}^2),\mathrm{\text{with}}i=1,2,...,N,j=1,2,...,M$$ (3b)

where $x_i$ represents the positioning of temperature acquisition along the $x$ coordinate with the total number of $N$ points, and $t_j$ the time instants with total number of $M$ levels. Therefore, the total number of experimental data considered is $N_{\mathrm{\text{exp}}}=N\times M$.

Inverse Problem Solution with Artificial Neural Networks

As previously mentioned, the IHTP of this work has the objective of estimating the positioning of gaps that can occur in Layer 2 of the medium (see Fig. 1). This task is performed by estimating discrete points of the auxiliary function $\gamma \left(x\right)$: this is the ANN output. The input consists of transient temperature measurements that are encoded into a latent space of an AE, which is used with the purpose of reducing the dimension and noise on the experimental data. This latent space serves as the input to a fully connected MLP network that yields another latent space of the same size of the first one. This last latent space is then decoded to a full representation of the auxiliary function $\gamma$. With this approach, both latent spaces are used as input and output, respectively, to the fully connected MLP neural network.

In the following subsections, the two AEs and MLP construction are explained and detailed. Firstly, the temperature and $\gamma$ AE are presented. Next, the complete network architecture is showcased. Lastly, general aspects necessary to build the network and the simulated data set are shown.

Autoencoders

Temperature Autoencoder (T-AE). Consider the AE structure presented in Fig. (2). The input $\mathbf{A}$ is fully connected to one hidden layer, represented by ${\mathbf{H}}_{\mathbf{A}}$, which is the latent space. Then ${\mathbf{H}}_{\mathbf{A}}$ is fully connected to the output $\hat{\mathbf{A}}$. These connections are all modified by weights, biases, and activation functions.

Figure 2
Schematic representation of the Temperature Denoising Autoencoder.

As presented in Fig. (2), the AE input $\mathbf{A}$ and output $\hat{\mathbf{A}}$ are normalized versions of ${\mathbf{T}}^{\mathbf{\text{exp}}}$ and $\mathbf{T}$, respectively (see Eq. 3). The components of $\mathbf{A}$ are the normalized version of ${\mathbf{T}}^{\mathbf{\text{exp}}}$, and the ones for $\hat{\mathbf{A}}$ are the normalized version of $\mathbf{T}$, both ranging between $-1$ and $1$. These linear normalizations are performed with limits of $T_{\mathrm{\text{min}}}$ and $T_{\mathrm{\text{max}}}$, which are the minimum and maximum exact temperature values of the entire training set (see subsection entitled General Aspects of the Artificial Neural Network for details).

The T-AE implemented in this work has only one hidden layer, which is the latent space itself, but in some cases, AE’s can have many hidden layers as the problem requires. Using the architecture with the single hidden layer, tests with different simulated experimental data led to good approximations, making the increased network with more layers unnecessary.

Choosing the latent space ${\mathbf{H}}_{\mathbf{A}}$ dimension is crucial to a good performance of the AE. If too small, some information will be lost, leading to a poor representation of the temperature data. On the other hand, if its dimension is too large, training requires more examples and the problem can become less efficient overall. Therefore a comparison of different ${\mathbf{H}}_{\mathbf{A}}$ dimensions is performed in this work. Adhesive Layer Properties Autoencoder ($\gamma$-AE). The adhesive layer thermophysical properties is represented by the auxiliary function $\gamma \left(x\right)$, which ranges from $0$ to $1$. The profile of this function represents different material properties and its behavior indicates where there are gaps in the layer.

As presented in Fig. 3, this second AE is used to represent $\gamma$ and has input $\mathbf{G}$, output $\hat{\mathbf{G}}$ and latent space ${\mathbf{H}}_{\mathbf{G}}$. The input $\mathbf{G}$ is composed by discrete points used to represent $\gamma$ across the $x$ direction. $\hat{\mathbf{G}}$ is the exact representation of $\mathbf{G}$, therefore, they are trained to be equal. The latent space ${\mathbf{H}}_{\mathbf{G}}$ dimension is chosen to be equivalent as ${\mathbf{H}}_{\mathbf{A}}$, i.e., equal to the T-AE latent space dimension.

Figure 3
Schematic representation of the Adhesive Layer Properties Autoencoder using the Auxiliary Function $\gamma \left(x\right)$.

It is important to note that this Adhesive Layer Properties AE has to encode less information than the T-AE presented, since it has only to output 0 or 1, so tests with dimensions smaller than the T-AE size yielded good results, which is expected. Nevertheless, in this work the choice of maintaining both latent space sizes equal is kept.

Final Artificial Neural Network Model

In order to transform ${\mathbf{H}}_{\mathbf{A}}$ into ${\mathbf{H}}_{\mathbf{G}}$, a fully connected MLP is used, i.e., this network model has to translate latent spaces information from temperature measurements to the adhesive layer properties in terms of $\gamma \left(x\right)$. Therefore, ${\mathbf{H}}_{\mathbf{A}}$ and ${\mathbf{H}}_{\mathbf{G}}$ are the input and output to the MLP, respectively, where hidden layers are connected and the information passing is modified by weight, biases and activation functions.

The final ANN architecture is displayed in Fig. (4), where it is possible to see the normalized experimental temperature input $\mathbf{A}$. This input is encoded into ${\mathbf{H}}_{\mathbf{A}}$, which is then passed and transformed into ${\mathbf{H}}_{\mathbf{G}}$ to finally be decoded as $\hat{\mathbf{G}}$. With this approach the actual $\gamma \left(x\right)$ estimation process utilizes only machine information, which is an advantage to obtain more robust models. Also, it is important to note that such network model allows to an increased dimension of the temperature input data, which is desirable with a transient problem. To illustrate this aspect of the ANN model, different time instants are considered and compared in this work.

The training of the Fully Connected MLP in Fig. (4) is performed by using the same training set that trained all the other models. To perform this task, firstly, the two AE’s must have been properly trained. Then, all the experimental temperature of the training set is encoded into ${\mathbf{H}}_{\mathbf{A}}$ and all the respective discretized $\gamma \left(x\right)$ profiles are encoded into ${\mathbf{H}}_{\mathbf{G}}$. With this, the pairs ${\mathbf{H}}_{\mathbf{A}}\to {\mathbf{H}}_{\mathbf{G}}$ are assembled and the model can be trained.

Figure 4
Schematic representation of the complete network model.

General Aspects of the Artificial Neural Network

To build the training data set for the three ANN, a list of input-output pairs is necessary. One training pair for the T-AE is composed by the rule $\mathbf{A}$ $\to$ $\hat{\mathbf{A}}$, which are the normalized versions of the experimental data ${\mathbf{T}}^{\mathbf{\text{exp}}}$ and the exact temperature data $\mathbf{T}$, respectively. Similarly, the material property AE uses the pair $\mathbf{G}$ $\to$ $\mathbf{G}$, which is the discrete representation of the exact $\gamma$ used to generate that particular $\mathbf{T}$ and ${\mathbf{T}}^{\mathbf{\text{exp}}}$. After completing the training of both AE’s, the set for the MLP can be build with input ${\mathbf{H}}_{\mathbf{A}}$ and output ${\mathbf{H}}_{\mathbf{G}}$ by using the two AE’s to translate the original training set into the related latent space.

To build this training data set, the direct problem is solved with a given random thermophysical properties profile for Layer 2 containing gaps, which is represented as the auxiliary function $\gamma$. Furthermore, it is important to follow some criteria that represents the physics of the problem in order to avoid unnecessary computations (Czél et al. 2014). In this work, the training data set is built by assuming that:

• There is a maximum number of gaps in Layer 2, i.e. $N_{\mathrm{\text{gap}}}=1,2,...,M_{\mathrm{\text{gap}}}$;

• There is a minimum gap size, namely $L_{\mathrm{\text{min}}}$;

• The gaps only occur in the region of $x_{\mathrm{\text{min}}}\le x\le x_{\mathrm{\text{max}}}$, where $x_{\mathrm{\text{min}}}$ and $x_{\mathrm{\text{max}}}$ are chosen to avoid gaps being too close to the boundaries.

• The T-AE and final MLP dataset are composed of multiple copies of the original set, with each one containing different level of noise for ${\mathbf{T}}^{\mathbf{\text{exp}}}$.

These rules are a way to generate meaningful information to the ANN model. For example, if the gap is too narrow, it will not influence the temperature on the exposed surface of the medium, specially considering experimental noise. Furthermore, the last rule is included in order to generalize the noise reduction aspect of the T-AE, by giving different noise levels in the data set.

Results and discussion

Direct Problem Solution

Considering a steel plate for Layer 1, adhesive for Layer 2, composite protection for Layer 3 and air as the inclusions in the possible defects (gaps), the respective thermophysical properties used are displayed in Table I.

Table I
Thermal Properties for the Multilayered Medium.

Layer	$k$ [W/mK]	$\rho c_p$ [MJ/m${}^3$K]
1	13.400	3.860
2	0.700	1.750
3	1.171	2.650
Gaps	0.0263	1.170

Furthermore, to obtain the results of the direct problem, the prescribed heat flux, the natural convection coefficient, and the ambient temperature are defined as $q=10000$ W/m${}^2$, $h=15.0$ W/m${}^2$K, and $T_{\infty }=20.0$ °C, respectively. The medium dimensions are defined as $L_x=0.2$ m and $L_y=0.071$ m, and Layer 2 is vertically positioned in the range $0.05\le y\le 0.051$ m.

Training, Validation and Testing Sets Generation

All the training and validation data are generated with 1, 2 or 3 gaps in Layer 2, i.e. $M_{\mathrm{\text{gap}}}=3$. For each particular number of gaps, 2000 randomly positioned gaps configurations are used to obtain the respective transient temperature profile, yielding a total of 6000 direct problem evaluations. The training and validation data are then split into a 90% to 10% relation, respectively. Therefore, from this total set, 5400 are used for training and 600 for validation.

To build the simulated experimental data used for the training process of the T-AE and the MLP, four levels of random noise are considered for ${\sigma }_{\mathrm{\text{exp}}}$, they are 0.05 °C, 0.1 °C, 0.15 °C and 0.20 °C (see Eq. 3b). This type of measurement is typically obtained via infrared thermography techniques (Knupp et al. 2013). Hence, four sets of 6000 input-output pairs compose the final training data, yielding 24000 examples (21600 for training and 2400 for validation). It is important to notice that the $\gamma$-AE training is performed with the original 6000 auxiliary function profiles.

The testing data is generated similarly to the training and validation set. It is composed of 150 random positioned defects (50 for 1, 2, and 3 simultaneous failures). The same noise levels are considered, therefore the original 150 data is copied yielding 600 examples for the T-AE and MLP testing - the $\gamma$ AE keeps the original 150.

In order to illustrate how efficiently the AE can store information in its latent space, the results of this work compares three problems with different time instants for the measured experimental data. For $M=3$ time instants, 600 s, 800 s, and 1000 s are considered, for $M=5$, 600 s, 800 s, 1000 s, 1200 s and 1400 s, and lastly, for $M=7$, the instants range from 600 s to 1800 s in steps of 200 s. For every instant, $N=321$ points in the $x$ coordinate are considered. So, for problems with 3, 5 and 7 time levels, the ANN must deal with a input of 963, 1605 and 2247 data, respectively. These time instants and total amount of input data are displayed in Table II.

Table II
Time Instants and number of temperature data used.

M	Time Instants	Data Size
$M=3$	600 s, 800 s, 1000 s	963
$M=5$	600 s, 800 s, 1000 s, 1200 s, 1400 s	1605
$M=7$	600 s, 800 s, 1000 s, 1200 s, 1400 s, 1600, 1800 s	2247

It is important to notice that the model proposed in this work considers increasingly time levels of physical experiment. In other words, for $M=3$, $M=5$ and $M=7$, the final time of temperature acquisition is 1000 s, 1400 s and 1800 s, respectively. Therefore there is a trade off between amount of data and temporal information available. At the lower spectrum, for $M=3$, less temperature readings are used, making the model training and configuration more efficient. On the other hand, for $M=7$, the larger data set can contain useful transient information of the problem, in exchange of training and model configuration efficiency. Hence, $M=5$ can be considered as the middle point of the two mentioned.

The exact maximum temperature values necessary to normalize the training data set between $-1$ and $1$ are different for the sets with 3, 5 and 7 time instants, they are: 35.5718 °C, 47.8281 °C and 60.5670 °C, respectively. The minimum temperature obtained is equal for the three sets, with value of 21.1653 °C, which occurs for $t=600$ s.

Temperature Autoencoder Results

The T-AE implementation is performed using the Keras library, from Python 3.9 programming language. The Logistic Sigmoid activation function is selected for the hidden layer and the output layer is kept as linear. The Adam optimizer is selected as implemented within the Keras library and the Logarithmic Hyperbolic Cosine (log-cosh) is chosen as the loss function. Such function has the advantage of being totally differentiable, with a $L_2$ norm behavior for small values of its argument, and as $L_1$ for larger values (Queiroz et al. 2021).

Early Stopping is selected as the stopping criterion. When the minimum loss function value found during training for the validation set does not change for 200 epochs, the iterative procedure is stopped. Then, the weights and biases obtained for the best epoch are used. This approach avoids over fitting of the training data (Chollet 2017). Furthermore, the maximum number of epochs is set to 6400, but none of the configurations tested reached that value. Finally, the batch size selected is 256, which is a relatively large size, but results have shown to be a good compromise between quality and speed of training.

In Table III the results obtained for the T-AE with $M=3$ time instants are displayed for different latent spaces dimensions, namely “AE dim" column. The table displays the Validation and Testing Mean Squared Errors obtained (MSE), along with the number of Training Epochs needed for that particular result. The result displayed as “Max. error" represents the maximum relative temperature error. This error is obtained with the following expression

$$\mathrm{\text{Max. error}}=\mathrm{\text{max}}\left(\left|\frac{{\hat{T}}_{i,j,k}-T_{i,j,k}}{T_{i,j,k}}\right|\right)\times 100$$(4)

where $\hat{T}$ and $T$ are the denormalized T-AE output and exact temperatures, respectively, both obtained for each testing data $k$, time level $j$ and point in space $i$.

Table III
Temperature Autoencoder (T-AE) MSE results for 3 time instants and different Autoencoder (AE) dimensions.

AE dim.	Val. MSE $\left({10}^{-6}\right)$	Test MSE $\left({10}^{-6}\right)$	Max. error %	Epochs
20	25.170385	24.700969	1.021519	1314
40	14.667914	14.616520	0.881631	1126
60	10.545812	10.572952	0.673743	1021
80	7.853343	7.673714	0.543680	1875
100	8.831671	8.996550	0.544467	1217
120	8.089550	8.317249	0.594848	1295
140	7.684247	7.809486	0.542980	1583
160	8.395999	8.263888	0.516542	1027

Table IV
T-AE MSE results for 5 time instants and different AE dimensions.

AE dim.	Val. MSE $\left({10}^{-6}\right)$	Test MSE $\left({10}^{-6}\right)$	Max error %	Epochs
20	31.454491	30.781830	2.285817	1182
40	9.214962	9.460896	0.984900	2242
60	5.495272	5.311201	0.602837	1664
80	5.777369	5.447388	0.769883	1185
100	5.811261	5.820386	0.725761	774
120	4.109372	4.142637	0.532461	1481
140	4.518737	4.532548	0.600322	1157
160	3.567894	3.335813	0.564507	1095

As it is possible to see in Table III, the AE dimensions such as 20, 40 and 60 generated the worst MSE results for the validation and testing set. Nevertheless, it is possible to notice that, with the exception of the AE with 20 dimensions, every configuration yielded a maximum error lower than 1%. Here is also possible to see that, in general, the epochs necessary to reach the stopping criterion ranges from 1021 to 1875, which can be considered a relatively low discrepancy. From the Test MSE criterion, the best configuration was the 80-dim AE. Here it is important to note that the MSE metric tends to increase larger errors, due to its squared argument. This is a desirable feature for comparing different ANN configurations, the best networks tend to have small and low discrepancy among them, whereas the worst configurations will generate larger values with higher discrepancy among themselves.

The T-AE results for $M=5$ and $M=7$ time instants are presented in Table IV and V, respectively. For both results it is possible to see that the validation and testing MSE tends to decrease as the AE dimension increases. For $M=5$, the maximum error values are also lower than 1% with exception of the 20-dim AE, which is not true for $M=7$, since only the dimensions 120, 140 and 160 generated lower than 1% maximum relative errors. This can be explained in the light of the amount of data considered for 7 time instants and their range, i.e., since all the data is normalized between -1 and 1, the more information one fits inside this range, the more difficult becomes to the network train with efficiency. Nevertheless, the lowest validation and testing MSE generated the lowest maximum relative error for $M=7$ (120-dim: 4.08, 3.97 and 0.63%).

Table V
T-AE MSE results for 7 time instants and different AE dimensions.

AE dim.	Val. MSE $\left({10}^{-6}\right)$	Test MSE $\left({10}^{-6}\right)$	Max error %	Epochs
20	32.861363	32.541317	2.374726	2217
40	9.426475	9.157836	1.144972	2459
60	10.696455	9.888308	1.453967	921
80	8.383932	8.169459	1.184161	892
100	6.024927	6.137550	1.347719	659
120	4.084239	3.969270	0.632970	938
140	4.491201	4.456416	0.728079	932
160	4.480562	4.385102	0.735544	732

MLP Results

To construct the complete ANN architecture, it is necessary to specify the elements for the MLP and $\gamma$-AE. Firstly, the activation functions for $\gamma$-AE latent space and output layer are the ReLU and Logistic Sigmoid functions, respectively. Since $\gamma$-AE outputs the auxiliary function, which ranges between $0$ and $1$, the Logistic Sigmoid is a reasonable choice that acts similar to a filter, limiting the output values inside that range and, furthermore, results with a linear output yielded undesired oscillations around the exact value. Regarding the loss function for training, the Binary Cross Entropy was selected, which have a good behavior for optimizing values that can be either $0$ or $1$. Secondly, the MLP is composed of four hidden layers, with the Hyperbolic Tangent (tanh) activation function for each and linear output layer. It is important to note that the MLP input and output sizes are equal as the latent spaces of both AE’s, as previously mentioned. The output layer activation is linear and the loss is calculated using the log-cosh functional. Other activation functions such as ReLu, Leaky ReLU and Logistic Sigmoid were tested for the hidden layers, but the best result (including faster training) were obtained with the tanh function. The stopping criterion for training for both $\gamma$-AE and MLP is similar as the T-AE, after 200 epochs with no improvements on the validation set loss function ADAM optimizer iterative process stops.

Tables VI, VII and VIII present the MSE between the exact discretized $\gamma$ profile and the respective ANN final output for different MLP sizes (number of neurons per hidden layer). Therefore these “Test MSE" results compares the decoded $\gamma$-AE output obtained from the MLP output, which MSE error is displayed as “Test ${\mathbf{H}}_{\mathbf{G}}$".

In Table VI the results for $M=3$ time instants are displayed, which were obtained with the 80-dim T-AE (therefore the $\gamma$-AE is also used with 80 dimensions). Similarly, Tables VII and VIII show the results for 5 and 7 time instants, with T-AE of 160-dim and 120-dim, respectively.

In Table VI, for $M=3$, the MLP size that yielded the lowest $\gamma$ MSE was the one with 200 neurons per hidden layer. Nevertheless, the 80 MLP size, which is equal as the used T-AE and $\gamma$-AE for this problem, generated the second best result. As it is possible to see, the epochs used for training decreases as the MLP size increases, which is indication that overfitting was avoided. Another important aspect to observe is that the best ${\mathbf{H}}_{\mathbf{G}}$ MSE was not observed for the same MLP size as the best $\gamma$ MSE, which indicates that the $\gamma$-AE accepts small differences in the latent spaces elements in order to decode it into the same output.

For $M=5$ and $M=7$ in Tables VII and VIII, the best found MLP size was the one with 40 neurons per hidden layer. Similarly to the results in Table VI, the training epochs needed also decreases when the MLP size increases, and the best ${\mathbf{H}}_{\mathbf{G}}$ MSE was not observed for the best $\gamma$ MSE. It is worth noting the the respective best $\gamma$ output MSE for 3, 5 and 7 time instants were 0.01396935, 0.01423047 and 0.01425704, respectively, indicating that the problem considering 3 instants generated the lowest error. Nevertheless, the three results are relatively similar.

Table VI
MSE for $M=3$, 80-dim AE, and different MLP sizes.

MLP size	Test MSE $\left({10}^{-2}\right)$	Test ${\mathbf{H}}_{\mathbf{G}}$ MSE $\left({10}^{-2}\right)$	Epochs
40	1.497972	0.108321	1947.0
80	1.421108	0.090003	1442.0
120	1.533937	0.108943	1107.0
160	1.492982	0.104258	828.0
200	1.396935	0.102659	723.0

Table VII
MSE for $M=5$, 160-dim AE, and different MLP sizes.

MLP size	Test MSE $\left({10}^{-2}\right)$	Test ${\mathbf{H}}_{\mathbf{G}}$ MSE $\left({10}^{-2}\right)$	Epochs
40	1.423047	0.069287	3382.0
80	1.569511	0.068726	993.0
120	1.490072	0.070474	720.0
160	1.462179	0.068498	612.0
200	1.513545	0.070189	549.0

Table VIII
MSE for $M=7$, 120-dim AE, and different MLP sizes.

MLP size	Test MSE $\left({10}^{-2}\right)$	Test ${\mathbf{H}}_{\mathbf{G}}$ MSE $\left({10}^{-2}\right)$	Epochs
40	1.425704	0.119196	2818
80	1.486218	0.117148	904
120	1.647438	0.129891	667
160	1.595111	0.129822	628
200	1.539906	0.125107	558

Detection of Contact Failures

The auxiliary function $\gamma$ estimation using the best ANN obtained in Tables VI, VII, and VIII are presented in this subsection. To illustrate how the model can generalize the input data with different noise levels, the results are considered for ${\sigma }_{\mathrm{\text{exp}}}$ of 0.1 °C and 0.2 °C.

In order to illustrate the T-AE noise reduction capability, the comparison among simulated experimental temperature data, the denormalized T-AE output, and the exact temperature is displayed in Fig. 5a, 6a, 7a, 8a, 9a, and 10a for different contact failures configurations. The simulated experimental temperature is generated with noise level of ${\sigma }_{\mathrm{\text{exp}}}=0.2$ °C and $M=3$ (three time instants). The T-AE output is obtained with the 20-dim latent space configuration. In such figures, the $\gamma \left(x\right)$ presented indicates the exact auxiliary function profile used to obtain the temperature displayed above each one. For the sake of visualization, the results with T-AE’s of 5 and 7 time instants are not displayed.

Although the noise level of ${\sigma }_{\mathrm{\text{exp}}}=0.2$ °C was the largest used in training, the approximation between the exact temperature profiles and the denormalized T-AE output is excellent, as it is possible to see for every configuration with one (Fig. 5a and 6a), two (Fig. 7a and 8a), and three (Fig. 9a and 10a) contact failures. This indicates that the AE has good dimensionality reduction qualities and it can, indeed, be used as a feature extractor tool, which stores the most important information necessary to reconstruct the temperature data without noise in the latent space.

Figure 5
T-AE output (a) and $\gamma \left(x\right)$ estimation (b) for one small contact failure configuration.

Figure 6
T-AE output (a) and $\gamma \left(x\right)$ estimation (b) for one large contact failure configuration.

Figure 7
T-AE output (a) and $\gamma \left(x\right)$ estimation (b) for two small contact failures configuration.

Figure 8
T-AE output (a) and $\gamma \left(x\right)$ estimation (b) for one small and one large contact failures configuration.

Figure 9
T-AE output (a) and $\gamma \left(x\right)$ estimation (b) for three small contact failures configuration.

Figure 10
T-AE output (a) and $\gamma \left(x\right)$ estimation (b) for one large and two small contact failures configuration.

Figure 5b and 6b presents the results for 1 single contact failure, being one small and one large, respectively. Both results shows a good approximation of the estimated $\gamma$ profile with the exact one. The small gap in Fig. 5b has 10 mm, which should be the hardest to detect due to lack of sensitivity, and shows very reasonable fit for all time instants considered. Therefore, this configuration with only one gap can be considered the easiest to estimate.

Now considering two defects in Layer 2, the results are presented in Fig. 7b and 8b for two small gaps and for one small and one large gap, respectively. Results in Fig. 7b shows two small gaps of 10 mm length and separated by another 10 mm. This configuration show very good results for $M=5$ and $M=7$ time instants, for both noise level considered. Although the results for $M=3$ are good, some undesired oscillation in the second gap is found. For one small and one large gaps, the results presented in 8b shows very good agreement to the exact values, considering that they are more far apart from each other. The large gap has 30 mm in length and the small 10 mm.

Finally, the results considering three gaps in Fig. 9b and 10b. The former is composed of three small gaps, and the latter with two small and one large. These configuration are the most difficult to estimate, due to the increase non-linearity of the problem: it makes the AE’s work harder to extract more information from the temperature data. All the gaps in Fig. 9b have 10 mm of length and are separated by 20 mm intervals. Here the agreements of results with exact values are not exactly centered, but, nevertheless, the three gaps positioning is well identified. Specially considering the result with $M=5$ and 7, which generated less oscillations. For two small and one large gaps, the results showed some difficult in estimation. The best agreement is obtained for $M=3$ for both noise levels, as indicated by results in the subsection entitled MLP Results. In this configuration, the model for $M=5$ presented some oscillations. Results with $M=7$ found the correct positioning of the two small gaps, but not the right size of them by estimating larger defects than the exact ones.

The computational times necessary for this investigation can be divided into three categories: training data generation, training of the models, and activation of the neural network. The system used to perform all executions is equipped with a AMD Ryzen™ 5 4600H processor, GeForce® GTX 1650 graphics processor, 8GB of RAM, and Microsoft Windows 10 operational system. Some aspects of computational times for each of the mentioned categories are the following:

• The training data generation is the most intensive computational task of this work. The time necessary to evaluate sequentially the 6000 different direct problem solutions, as described in the subsection General Aspects of the Artificial Neural Network, was 15.6 hours. Although this task was performed sequentially, it has a fixed number of iterations and can be easily parallelized in order to reduce this computational time.

• The computational times spent to train the neural networks are varied, given the different architectures and networks types. For the sake of illustration, the training of MLP’s with 160 and 200 neurons per layer of $M=7$ time instants took 3 minutes and 22 seconds, and 2 minutes and 57 seconds, respectively, with both using 628 and 558 epochs to reach the stopping criterion - see Table VIII.

• The computational cost of performing the final estimation of $\gamma \left(x\right)$ with the three networks is the main motivation for using the approach proposed. For instance, the estimation for $M=7$ and ${\sigma }_{\mathrm{\text{exp}}}=0.1$ °C, as presented in Fig. 6, takes an average of 0.113 seconds.

Conclusions

The present work used an ANN in order to estimate the positioning of defects that can occur in the bonding between two materials. To reduce the dimension of the 1D transient thermography experimental data, an AE was used. Different configurations of AE were tested considering data from 3, 5 and 7 time instants, which corresponds to 963, 1605 and 2247 temperature readings, respectively. The main aspects of the proposed approach for solving the defect positioning estimation problem may be summarized as the following items:

• Speed of estimation. When compared to classical approaches such as Maximum Likelihood or Bayesian Methods, the use of ANN’s for estimating parameters of the physical problem has a great advantage in computational efficiency. Usually, such classical methods have cumbersome iterative processes that are not practical in some real scenarios. On the other hand, the estimation with ANN’s can instantly be obtained.

• Reduced dimensions of temperature data. Since the problem solved in this work is transient, several time instants must be included as the ANN input. This work deals with this problem by encoding such transient temperature profiles in order to facilitate the work of estimating failures. Thus, the reduction in dimension was necessary to extract important features of the noisy data and work with a manageable ANN.

• Increased estimation resolution. Since an AE is also used to reduce the dimension of the output, a larger amount of points can be used to represent the varying thermophysical properties of the adhesive layer.

• Fixed computational cost for training data generation. Each training data is generated by solving the direct problem once. Therefore, the amount of examples needed to train the model is generated only once. For every result presented in this work, the same 6000 direct problem solutions were used.

Moreover, further investigations must continue in two main subdivisions: Validation of the proposed ANN model with real experimental data, and construction of a ANN model that considers the complete 2D transient thermography image as experimental data.

ACKNOWLEDGMENTS

The authors acknowledge the financial support provided by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, Finance Code 001), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ). LCSJ acknowledges CAPES also for the scholarship for his stay in the University of Granada (Grant CAPES/PrInt No. 88887.469279/2019-00). This work has been also partially funded by the projects PID2020-112754 GB-I00 (Spanish Ministry of Economy and Competitiveness and funds from the European Regional Developement Fund, ERDF) and B-TIC-640-UGR20 (Regional Govern of Andalusia, Spain).

• ABREU LAS, COLAÇO MJ, ORLANDE HRB & ALVES CJS. 2016. Thermography detection of contact failures in double layered materials using the reciprocity functional approach. Appl Therm Eng 100: 1173–1178. doi:10.1016/j.applthermaleng.2016.02.078
• ABREU LAS, ORLANDE HRB, KAIPIO J, KOLEHMAINEN V, COTTA RM & QUARESMA JNN. 2014. Identification of contact failures in multilayered composites with the Markov chain Monte Carlo method. J Heat Transf 136(10). doi:10.1115/1.4027364
• BALAJI C & PADHI T. 2010. A new ANN driven MCMC method for multi-parameter estimation in two-dimensional conduction with heat generation. Int J Heat Mass Transf 53(23-24): 5440–5455. doi:10.1016/j.ijheatmasstransfer.2010.05.064
• BECK JV, BLACKWELL B & CLAIR JR CRS. 1985. Inverse heat conduction: Ill-posed problems. John Wiley & Sons.
• CHOLLET F. 2017. Deep Learning with Python. New York, NY: Manning Publications.
• CZÉL B, WOODBURY KA & GRÓF G. 2014. Simultaneous estimation of temperature-dependent volumetric heat capacity and thermal conductivity functions via neural networks. Int J Heat Mass Transf 68: 1–13. doi:10.1016/j.ijheatmasstransfer.2013.09.010
• DANES F, GARNIER B & DUPUIS T. 2003. Predicting, measuring, and tailoring the transverse thermal conductivity of composites from polymer matrix and metal filler. Int J Thermophys 24(3): 771–784. doi:10.1023/A:1024096401779
• FANG Q & MALDAGUE X. 2020. A method of defect depth estimation for simulated infrared thermography data with deep learning. Appl Sci 10(19): 6819. doi:10.3390/app10196819
• GARCIA S, GUYNN J & SCOTT EP. 1998. Use of genetic algorithms in thermal property estimation: part II-simultaneous estimation of thermal properties. Num Heat Transf Part A Appl 33(2): 149–168. doi:10.1080/10407789808913932
• HAYKIN SS. 2009. Neural networks and learning machines. 3^rd ed. Upper Saddle River, NJ: Pearson Education
• HINTON GE & SALAKHUTDINOV RR. 2006. Reducing the dimensionality of data with neural networks. Science 313(5786): 504–507. doi:10.1126/science.1127647
• HU C, DUAN Y, LIU S, YAN Y, TAO N, OSMAN A, IBARRA-CASTANEDO C, SFARRA S, CHEN D & ZHANG C.. 2019. LSTM-RNN-based defect classification in honeycomb structures using infrared thermography. Infr Phys Technol. Infrared Physics & Technology 102: 103032. doi:10.1016/j.infrared.2019.103032
• IMANI A, RANJBAR A & ESMKHANI M. 2006. Simultaneous estimation of temperature-dependent thermal conductivity and heat capacity based on modified genetic algorithm. Inv Probl Sci Eng 14(7): 767–783. doi:10.1080/17415970600844242
• JAYASEELAN DD, XIN Y, VANDEPERRE L, BROWN P & LEE WE. 2015. Development of multi-layered thermal protection system (TPS) for aerospace applications. Comp Part B Eng 79: 392–405. doi:10.1016/j.compositesb.2015.04.017
• KAIPIO JP & FOX C. 2011. The Bayesian framework for inverse problems in heat transfer. Heat Transf Eng 32(9): 718–753. doi:10.1080/01457632.2011.525137
• KAUR K, MULAVEESALA R & MISHRA P. 2021. Constrained Autoencoder based Pulse Compressed Thermal Wave Imaging for Sub-surface Defect Detection. IEEE Sens J doi:10.1109/JSEN.2021.3056394
• KNUPP DC, NAVEIRA-COTTA CP, ORLANDE HRB & COTTA RM. 2013. Experimental identification of thermophysical properties in heterogeneous materials with integral transformation of temperature measurements from infrared thermography. Exper Heat Transf 26(1): 1–25. doi:10.1080/08916152.2011.631079
• MARANI R, PALUMBO D, GALIETTI U & D’ORAZIO T. 2021. Deep learning for defect characterization in composite laminates inspected by step-heating thermography. Opt Laser Eng 145: 106679. doi:10.1016/j.optlaseng.2021.106679
• MASCOUTO FS, KNUPP DC, ABREU LAS, SANCHES EL & SPHAIER LA. 2020. Detection of contact failures employing combination of integral transforms with single-domain formulation, finite differences, and Bayesian inference. Num Heat Transf Part A Appl 77(6): 599–618. doi:10.1080/10407782.2020.1713637
• MOURA NETO FD & SILVA NETO AJ. 2013. An introduction to inverse problems with applications. Springer Science & Business Media. doi:10.1007/978-3-642-32557-1
• ORLANDE HRB. 2012. Inverse problems in heat transfer: new trends on solution methodologies and applications. J Heat Transf 134(3). doi:10.1115/1.4005131
• ORLANDE HRB, DULIKRAVICH GS, NEUMAYER M, WATZENIG D & COLAÇO MJ. 2014. Accelerated Bayesian inference for the estimation of spatially varying heat flux in a heat conduction problem. Num Heat Transf Part A Appl 65(1): 1–25. doi:10.1080/10407782.2013.812008
• QUEIROZ RM, RIBEIRO EG, SANTOS ROSA N & GRASSI JR V. 2021. On deep learning techniques to boost monocular depth estimation for autonomous navigation. Robot Auton Sys 136: 103701. doi:10.1016/j.robot.2020.103701
• SARKAR D, HAJI-SHEIKH A & JAIN A. 2015. Temperature distribution in multi-layer skin tissue in presence of a tumor. Int J Heat Mass Transf 91: 602–610. doi:10.1016/j.ijheatmasstransfer.2015.07.089
• SHIGUEMORI EH, DA SILVA JDS & CAMPOS VELHO HF DE. 2004. Estimation of initial condition in heat conduction by neural network. Inv Probl Sci Eng 12(3): 317–328. doi:10.1080/10682760310001598599
• SOEIRO F, SOARES PO, CAMPOS VELHO HF & SILVA NETO AJ. 2004. Using neural networks to obtain initial estimates for the solution of inverse heat transfer problems. In: Inverse Problems, Design an Optimization Symposium, p. 358-363.
• SU J, CERQUEIRA DR & WANG S-Z. 2012. Thermal design of multi-layered composite pipelines for deep water oil and gas production. Int J Comp Appl Technol 43(3): 248–259. doi:10.1504/IJCAT.2012.046311
• VAKILI S & GADALA MS. 2009. Effectiveness and efficiency of particle swarm optimization technique in inverse heat conduction analysis. Num Heat Transf Part B Fund 56(2): 119–141. doi:10.1080/10407790903116469
• XU L & HU J. 2021. A Method of Defect Depth Recognition in Active Infrared Thermography Based on GRU Networks. Appl Sci 11(14): 6387. doi:10.3390/app11146387

Publication Dates

• Publication in this collection
  01 Aug 2022
• Date of issue
  2022

History

• Received
  13 Dec 2021
• Accepted
  17 Apr 2022