Computational Intelligence Conceptions to Automated Diagnosis: Feature Grouping for Performance Improvement

Abstract

The motivation of this work is to investigate two technological AI paths, evaluate the performance, and discuss the results. Using a covid-19 chest X-ray images databank, we address the two distinct experiments to this problem: (1) an investigation of feature extraction and classification using machine learning algorithms and (2) an approach based on transfer learning used in state-of-the-art applications. For the implementation of our proposal (1), an integrated framework consisting of 25 algorithms with different characteristics was developed to extract features from chest X-ray images. Following this path, we seek to focus on the spatial spectral signatures of shape, texture, local and global statistical quantities. The extraction of features based on information in Fourier and wavelet space-frequency domain was also implemented as part of the framework. On the other hand, several transfer learning CNN’s were also used to evaluate performance and to compere to the first technological path results. Furthermore, the performance of other results reported by various other works are provided. The comparative performance evaluation demonstrated that the two concepts for a computational intelligence tool can produce very good results even working in high-dimensional vector spaces.

Keywords:
feature extraction; machine learning; convolutional neural network; transfer learning; covid-19.

HIGHLIGHTS

• An integrated framework for feature extraction in medical images is proposed.

• It is investigated the automatic diagnosis in chest X-ray images using several algorithms.

• It is show that performance is increased when different feature extraction algorithms are merged.

• Experiments were performed with fourteen different IA algorithms and the results were compared.

INTRODUCTION

Humanity was surprised by the worldwide viral contamination that is now simply called COVID-19. This pandemic has caused a seemingly endless number of deaths and exposed the fragility of our healthcare institutions throughout the planet [¹]. Population isolation and early diagnosis can reduce viral spread. Immunizing the population through mass vaccination seems to be the strongest way to overcome this pandemic.

Since the end of 2019, the symptoms of COVID-19 have been explored and reported in the scientific literature. A large number of cases suffer acute respiratory syndrome [²]. The disease can be detected mainly by two types of tests: through the Reverse Transcription Polymerase Chain Reaction (RT-PCR) and serological tests (antibodies present in the blood). During this pandemic, radiologists have learned and gained experience from the multitude of situations they must assess [³-⁴].

To help healthcare systems and hopefully mitigate this pain that has befallen the planet, there is a strong motivation to invest in research and development automatic tools to support medical diagnosis [³]. Diagnosis can also be made by analyzing computer tomography (CT) images and/or by chest X-Ray images. This procedure can be performed by a radiologist as well as by an automatic tool to support medical diagnosis [³-⁷]. The resolution, texture quality, and discrimination of spectral components in CT images are superior to those of X-Ray [³-⁴]; thus, this approach allows a more accurate diagnosis by a radiologist or even by intelligent algorithms to support the diagnosis. However, the CT exam is more expensive and difficult to perform in many parts of the world. X-ray-based radiography found throughout the world is relatively inexpensive. Therefore, COVID diagnosis based on Chest X-ray images is the subject covered in this work. Two technological paths are investigated in this work: (1) grouping of features computed in different algorithms, and classification using machine learning algorithms, and (2) CNN transfer learning solution. The performance of the two approaches is shown and discussed.

We selected a set of works that we believe it reports the state of the art, although many other good articles can be found on this subject [⁸-⁴¹]. The scenario of deep learning research involves images as input information through convolutional neural networks (CNN). This can also be observed in research that addresses the diagnosis of COVID-19. Many works are found based on pre-trained model with transfer learning using known networks: AlexNet [³⁸], MobileNetV2 [⁸, ²⁰-²¹, ³¹], NASNet [¹¹], Inception - V3 [⁸, ²⁵], DarkNet - 53 [²⁸], DenseNet - 121 [⁹, ¹¹, ¹⁷, ²⁵, ³⁹], DenseNet - 201 [⁸, ¹⁴, ²¹], DenseNet - 210 [³⁸], Inception - V4 [³¹], InceptionResNet -V2 [⁸, ³¹], ReconNet [²⁵], ResNet - 8 [¹⁸], ResNet16 [¹⁸], ResNet - 18 [¹⁴-¹⁵, ¹⁷, ²⁷, ³⁷-³⁸], ResNet - 50 [⁸, ¹³, ¹⁵, ²², ²⁴, ²⁹, ⁴¹], ResNet -101 [¹⁷], Xception [⁸, ¹¹], EfficientNet [¹¹], SqueezeNet [¹⁴-¹⁵, ²⁰], VGG - 16 [¹⁰-¹³, ¹⁷, ²⁷, ³¹, ⁴⁰], VGG - 19 [⁸, ³⁰]. Works employing custom CNNs are also available [⁸, ¹⁰, ¹⁹, ²⁵, ²⁷, ³⁷, ³⁹].

For many solutions, CNNs can carry out the entire process up to classification [⁷, ⁹, ¹⁰-¹⁵, ¹⁷-¹⁹] or they can be part of a system that performs feature extraction [⁸, ¹⁰, ¹⁶, ²⁰, ²²-²⁷, ²⁹, ³⁰, ³², ³⁵, ³⁸, ⁴⁰, ⁴¹]. CNNs are also used in hybrid systems integrated with classic classifiers, such as Logistic Regression (LR) [³⁰, ³⁵], Support Vector Machine SVM [²⁰-²⁶, ³³, ³⁵, ³⁸], k-Nearest Neighbor (K-NN) [²⁰-²¹, ²³, ²⁵, ³⁰, ³⁵, ³⁸], Naïve Bayes [²¹], Random Forest [²¹, ²³, ³⁰], Multilayer Perceptron (MLP) [²¹, ²⁵], Decision Tree [²³, ³⁶], Adaptive Boosting (AdaBoost) [²³, ³⁵], Extreme Gradient Boosting (XGBoost) classifier [³⁰], and Stochastic Gradient Descent (SGD) [³⁵]. Deep learning techniques addressing Long Short-Term Memory (LSTM) network [³¹-³²], and even an approach using CNN integrated with LSTM [⁴²] is also found. In addition, COVID diagnosis methodologies have been reported that employ image features extraction based on radiomics, image statistics, texture, and shape integrated with classifier, such as Ensemble Bagged Model (EBM) trees, K-NN, and SVM for classification [³³-³⁴]. Dimensionality reduction [²⁵, ²⁹, ³⁹], feature selection [³¹, ³³, ³⁵], preprocessing as image resize [¹⁶, ¹⁷, ³⁹-⁴⁰], histogram equalization [³⁹-⁴⁰], segmentation [¹⁶, ¹⁹, ³³, ³⁷, ⁴⁰], data augmentation [⁹, ¹⁷, ³³, ³⁹-⁴⁰], and denoising [¹⁶] are addressed in many methodologies.

The data need to show the spectral signatures/patterns that characterize the studied problem. In our proposal, to extract features from chest X-ray images, we ignore the dimensionality problem of as the limitation. Even for machine learning algorithms, the reduction of complexity was not taken into account [²⁵, ²⁹].

Following this line of reasoning, we address to solve the problem in a higher order vector space, and the dimension was removed as a restriction of the investigated problem. Furthermore, many machine learning algorithms perform well in large dimensional spaces. Approaches that use Bayesian inference suffer from badly conditioned covariance matrix. However, most of the time it is possible to consider the random variables as independent and simplify the problem without significantly affecting algorithm performance.

We propose a scenario with a large number of features to be extracted (up to 4000 features) based on statistics, shape textures, and frequency domain. Results for COVID diagnosis were obtained in the chosen scenarios based on machine learning classification techniques. Furthermore, we provide results using pre-trained CNNs to compare to the machine learning techniques obtained and techniques addressed in other works.

This paper is organized as follows. The methodology adopted in the present work is shown in the next section. The results are presented in section 3. In section 4, the obtained results and the comparative performance evaluation are discussed. Section 5 provides the conclusions of the present work.

METHODS

Transfer learning based on CNN pre-trained models has been very successful in COVID-19 detection/diagnostic tools [⁸-⁴¹]. However, these methods require extensive computational resources such as GPUs (Graphics Processing Units), cloud processing, or specialized cluster hardware. Our work demonstrates other technological paths that can achieve similar performance, without costing so much computational effort. Figure 1 illustrates two paths taken in the construction of a computational medical diagnostic tool for COVID-19. In the first step of the block diagram of Figure 1, the dynamic range of the X-ray image is expanded to [0, 255] and then resized to (224 × 224) pixels. The thoracic X-ray images with its pixels represented in a gray scale are converted to an RGB format. One of the paths involves the development of a framework for feature extraction and classification based on machine learning techniques, and the other is based on transfer learning through pre-trained CNNs to produce a diagnosis. These two technological paths will be detailed below.

Overview of the proposed Feature Extraction Framework (FEF)

All In the block diagram shown in Figure 1 there are two solutions to the classification problem: one through the use of a feature extractor in conjunction with machine learning techniques and, the second solution, through the use of deep learning, more specifically pre-trained convolutional networks. In this section, an overview of the feature extraction framework is presented.

Figure 2 shows a block diagram of the Feature Extractor Framework (FEF). This is a flexible platform that allows the inclusion of other future feature extraction algorithms. The Figure 2 present the various algorithms that are part of it. The FEF input can be a grayscale or color image bank. In the way of RGB color images, there are algorithms for build histograms and global image statistics. RGB color space can be converted to HSV or CIELab color space. Features can be computed in any color representation space or in more than one if so configured.

In the path where we have the grayscale representation of the image, there is another scenario of global, local and texture processing algorithms. The images can be requantized by using a linear or k-Means based quantizer. The modules that appear at the top just below in Figure 2 present the cartesian moments, central moments, moments-HU and moments-Log-Hu. They can be configured together or individually. For example, if you specify the calculation of the log-Hu-moments (which are 7 in number) you will automatically need to calculate all the other moments whose blocks are connected in series in Figure 2. Table 1 shows the number of features computed in each algorithm.

Cartesian and central moments were calculated up to order n = 3 [⁴³]. This produces (n+1)² = 16 moments in each case. If the option "select moments" is chosen, only the five moments of order (0,1), (1,0) (1,1), (0,2), (2,0) will be returned. Seven invariants moments (Hu-moments) and seven and Log-Hu-moments can be computed, respectively. The invariant moments have been widely applied to image pattern recognition in a variety of applications due to their invariant characteristics in image translation, scaling and rotation [⁴³-⁴⁴]. Histograms can produce up to a maximum of 255 features per band (color images count the three bands).

The statistical features, which are a total of 32, are global spatial quantities computed directly on the image [⁴⁵].

We define the digital image as:

$\left[n,m\right],0\le n\le N-1and0\le m\le M-1$

where N defines the number of lines, M defines the number of columns of the image and x[.,.] informs the pixel amplitude value, x ∈ [0, 255]). The Probability Mass Function (PMF) is computed based on the relative frequency of the pixel amplitude, basically the histogram normalized by the number of pixels.

Figure 1
Framework block diagram. Note that there are two paths after the pre-processing module: in the path on the left shows the image feature extracting, these quantities are the inputs of the machine learning algorithms; the path on the right shows the solution through pre-trained convolutional networks (transfer learning). At the end of each technological conception, performance is evaluated according to objective metrics.

Local quantities such as those of Haralick descriptors [⁴⁶-⁴⁷], the Zernike moments [⁴⁸-⁴⁹], the Histogram of Oriented Gradients (HOG) [⁵⁰], the Local Binary Pattern (LBP) [⁵¹] and the Local Directional Patterns (LDP) [⁵²-⁵³] were also computed. The Haralick descriptors commonly used in image analysis are texture features and are computed from a gray scale image representation. The Haralick texture descriptors are computed from a matrix that counts the co-occurrence of neighboring gray levels in the image (GLCM - Gray Level Co-occurrence Matrix) [⁵³]. The GLCM matrixes counts the number of occurrences of the neighboring pixel values relative to the reference pixel. This is evaluated for all 4 directions {0, π/4, π/2, and 3π/4 radians}. See the example shown in Figure 3, in the direction of 0 radians if we take the reference pixel with value equal to 3, it is observed that the value with amplitude 1 immediately to the right occurs twice. This means that we have two occurrences of the pair {3, 1}. This information is indicated in the respective gray levels co-occurrence matrix.

The Zernike moments and the Histogram of Oriented Gradients (HOG) are descriptors that focus on the shape of an object. The Zernike moments, proposed by Frits Zernike [⁴⁸], are based on a set of complex polynomials that form a complete orthogonal basis. Similar to Hu invariant moments, the Zernike moments can also be used to describe the shape of an object, however, as the Zernike polynomials are orthogonal to each other, there is no redundancy of information between moments. To these features were added Histograms of Oriented Gradient (HOG) [⁵⁰]. The HOG is commonly used in computer vision and image processing for object detection purposes. The HOG shape features measure the intensity of the gradients or edge directions in an image cell. In this work, a cell is considered 32 × 32 pixels, but it is configurable.

The LBP [⁵¹] algorithm is characterized by being a texture operator that labels the pixels of an image, limiting the process to a neighborhood of each pixel. The result is encoded as a binary number [⁵¹]. The LDP algorithm, which has been successfully applied in face identification [⁵² ], uses changing the magnitude of the gradient in a specific direction around pixels to encode the local texture. Instead of comparing the intensity value of neighboring pixels, this method compares the gradient magnitude of the neighboring pixel along a specific direction and encodes it in the same way as LBP. In Figure 4 is shown a LDP example using Chest X-Ray as input image. Usually, the histogram is calculated from the LDP. However, other statistical parameters can be proposed.

The features based on texture matrixes [⁵³] are computed from Gray-Level Co-occurrence Matrices (GLCM), Gray Level Dependence Matrix (GLDM), Gray-Level Run Length Matrices (GLRLM), Gray-Level Size Zone Matrix (GLSZM) and Gray Tone Difference Matrix (NGTDM).

Figure 2
Feature extractor block diagram. Each block corresponds to a specific algorithm. The blocks in gray color refer to the algorithms applied in gray-levels images. The blocks in blue consider the three bands color images. The features computed in each algorithm can be concatenated into a single output file.

The addition of global features computed in the frequency domain are also proposed in the 2D-Discrete Fourier Transform (2D-DFT) domain and in the 2D-Discrete Wavelet Transform (2D-DWT) domain. For instance, we propose to investigate the frequency domain quantities such as the amount of information distributed in subband energy for the 2D-DFT. To construct the energy distribution features in the frequency domain, a two-dimensional FFT (Fast Fourier Transform) was used, and the magnitude spectrum (squared) was segmented into a set of circular disks. Each pair of circular disks are delimited by a lower and an upper frequency, corresponding to a subband in the discrete Fourier transform domain. The 2D-DFT can be computed by

$X\left[k,r\right]=\frac{1}{NM}{\displaystyle \sum }_{n=0}^{N-1}{\displaystyle \sum }_{m=0}^{N-1}x\left[n,m\right]e^{-\frac{2\pi }{N}kn}{e}^{-\frac{2\pi }{M}rm}$

Table 1
Feature extractor framework quantity features produced.

Algorithms	Number of features
cartezian moments {(0, 1), (1, 0), (1, 1), (0, 2), (2, 0)}	5
cartezian moments - moments order (default:3)	up to 64
central moments: {(0, 1), (1, 0), (1, 1), (0, 2), (2, 0)}	5
central moments of order - moments order (default:3)	up to 64
invariants Hu moments	7
invariants Log-Hu moments	7
gray-scale image histogram (default: 64 bins)	up to 256
BRG image color space histogram (default: 64 x 3 = 192 bins)	up to 768
HSV image color space histogram (default: 64 x 3 = 192 bins)	up to 768
CIELab image color space histogram (default: 64 x 3 = 192 bins)	up to 768
Gray-Scale Statistics	32
RGB Color Statistics	96
HSV Color Statistics	96
CIELab Color Statistics	96
Zernike Moments shape descriptors	25
Local Binary Patterns (LBP) descriptors (default: 16 features)	up to 256
Local Directional Patterns (LDP) descriptors (default: 16 features)	up to 256
Histograms of Oriented Gradients (HOG) (128 features)	up to 512
Haralick texture descriptors (4 directions x 13)	52
Gray-Level Co-occurrence Matrices (GLCM) (4 directions x 27)	108
Gray Level Dependence Matrix (GLDM)	20
Gray-Level Run Length Matrices (GLRLM) (4 directions x 20)	80
Gray-Level Size Zone Matrix (GLSZM)	20
Gray Tone Difference Matrix (NGTDM)	5
frequency energy from linear subband spectrum (default:16)	up to 64
frequency energy from logarithmic subband spectrum	up to 64
wavelet transform decompositions levels energy	image size log2 function

Figure 3
It shows the GLCM (Gray-Level Co-occurrence Matrices) computed in the 4 directions: 0, π/4, π/2, and 3π/4 radians. GLCMs have dimension NxN, where N is the number of bits with which each pixel is represented.

For illustrative purposes, Figure 5 is an example where the amplitude spectrum is represented in decibels. In the two-dimensional DFT (2D-DFT), the coefficient $X\left[\mathrm{0,0}\right]$ is centered in Figure 5, and the 2D-DFT is interpolated (this leads to Discrete-Space Fourier Transform - DSFT) for didactic effects.

$X{\left[k,r\right]}_{dB}=20{\log }_{10}\left|X\left[k,r\right]\right|$

where $0\le k\le N-1and0\le r\le M-1$. The energy of the information stored in each band $B_k$ is measured by computing

$E_k={\displaystyle \sum }_{k,r\in B_k}{\left|X\left[k,r\right]\right|}^2$

In Figure 5 the band $B_k$ projected on the frequency axes is emphasized. Note that since the Fourier transform modulus is symmetric to any of the frequency axes, it only suffices to calculate the energies over one of the quadrants (the gray area in Figure 5). In case $N\ne M$ the curve that defines the limits of each subband describes an ellipse, otherwise if $N=M$, the subband limit curve defines a circle. The DC coefficient $X\left[\mathrm{0,0}\right]$ is not included in the energy calculation. Two possibilities were implemented, in the first approach the bandwidth varies uniformly and in the second it varies logarithmically.

In the 2D-DWT domain, features are calculated as a function of the information energy at the transform decomposition level. The feature extraction framework (FEF) can be configured to produce up to 4500 features.

After building the database corresponding to the feature set for all images, features that are not random variables are eliminated (for example, if it is a constant value), as shown in Figure 2.

All computed features can now be concatenate and save as one data file. Feature selection and scaling algorithms are also included in the platform.

Observe that the set of features computed from many different algorithms are then placed together. These features concern to local and global quantities. Some features preserve the texture spectral signature, other are associated with shape. Some are obtained from the space domain and others from the frequency domain. We would like to show that this is an interesting strategy to obtain a set of features. Furthermore, therefore, an improvement in the performance of machine learning techniques.

Figure 4
Local Directional Patterns (LDP) example. The output of the LDP algorithm generates a binarized image as a function of the local texture behavior.

Figure 5
Frequency domain representation. The magnitude of the Fourier transform as a function of horizontal and vertical frequencies is represented in logarithmic terms (deciBeis) for ease of visualization.

Learning Framework

Two learning approaches were investigated (Figure 1). The first approach uses seven machine learning algorithms involving the data features file built by the feature extractor, and the second, is based on transfer learning through pre-trained CNNs. We also used seven pre-trained CNNs models.

In the second approach to the problem, as shown in Figure 1, pre-trained convolutional neural networks (CNN) were addressed. In this case, the chest radiography image dataset was used as input to the classification problem. Seven distinct algorithms were also investigated: ResNet, InceptonResNet, DenseNet, EffivientNet, VGG19, Xception, and MobileNet.

RESULTS

The following subsections will describe the input dataset used and the two scenarios investigated, with two or three diagnosis classes.

Input dataset description

In this work, the set of X-ray images used was extracted from different sources, available from the KAGGLE website version 4, with an update in March/06/2021. This image database contains 3616 COVID-19 positive cases, 10,192 normal X-ray images, 6012 X-ray images with lung opacity diagnosis, and 1345 viral pneumonia cases. All images are disposed in Portable Network Graphics (PNG) file format with a resolution of 299 × 299 pixels.

The 3616 images of COVID-19 positive cases were extracted from the Banco digital de Imagen Médica de la Comunidad Valenciana (BIMCV) as result of the Pathology Detection in Chest Radiology (PADCHEST) project; from the Institute of Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany; from the Società italiana di Radiologia Medica e Interventistica (SIRM); from the Eurorad COVID-19 Data Repository; and other publicly available sources of COVID-19 positive patients. In addition, a total of 6012 images diagnosed with lung opacity, along with 8851 normal chest radiograph images, were collected from the Radiological Society of North America (RSNA) database. Another 1341 normal X-ray images and the 1345 images diagnosed with viral pneumonia are pediatric images and were taken from a collection of X-ray images with 8 different pathologies called ChestX-ray8. An example with chest X-ray images with different diagnoses is shown in Figure 6.

Figure 6
Chest X-Ray images example.

In this work, we investigate two distinct scenarios involving the diagnosis of thoracic X-Ray images. In these scenarios, the algorithms need to discriminate between two and three distinct classes. The same preprocessing algorithm (Figure 1) was employed in the two scenarios, with 10% of the total instances used for technique performance evaluation and 90% for training and/or validation. The metrics used to assess the specific case of the two classes are different from the other scenarios studied, as will be detailed.

Table 2 shows the feature extractor configuration for the performance evaluation with the machine learning algorithms. The machine learning algorithms were implemented in Python language using Skit-learn libraries with the default classifier configuration. The following algorithms were chosen: Linear Discriminant Analysis (LDA), XGBoost Classifier (XGB), Histogram-based Gradient Boosting (HGB), Bagging Classifier (BC), Ensemble Extra-Tree Classifier (ETC), Gradient Boosting (GB) and Random Forest (RF). The set of chose machine learning were implemented to run in parallel way. It means that all algorithms were under the same training and testing conditions to produce the prediction results.

For all machine learning experiments, the model training and evaluation procedure was randomly repeated 30 times. The feature extractor framework was configured as shown in Table 2 to produce initially, 481 features. After eliminating the non-random variables and applying the feature selection algorithm based on Pearson's coefficient, 432 random variables remained. Features were discarded when the absolute value of the Pearson's coefficient is greater than 0.75. Furthermore, in the experiments carried out, no feature scaling was applied.

For all machine learning experiments, the model training and evaluation procedure was randomly repeated 30 times. The feature extractor framework was configured as shown in Table 2 to produce initially, 481 features. After eliminating the non-random variables and applying the feature selection algorithm based on Pearson's coefficient, 432 random variables remained. Features were discarded when the absolute value of the Pearson's coefficient is greater than 0.75. Furthermore, in the experiments carried out, no feature scaling was applied.

In each machine learning simulation (for a total of 30), a dataset for training and a dataset for performance evaluation were randomly selected for each simulation. The presented results correspond to the minimum, the average, and the maximum of the obtained performances. Using the pre-trained CNNs models, the approach was different, at the end of training, the best epoch validation performance model weights were chosen. The following networks based on transfer learning were evaluated: ResNet, InceptonResNet, DenseNet, EffivientNet, VGG19, Xception, and MobileNet.

Hyperparameters

The machine learning algorithms were implemented using Skit-Learn [⁵⁴] in Python programming language. For the boosting-based classifiers: Gradient Boosting (GB), Histogram-based Gradient Boosting (HGB), and XGBoost Classifier [⁵⁵] used in the experiment, the learning rate was set to 0.15. It was used the log-loss function to use to be optimized and the boosting process maximum number of iterations was set to 100. For Linear Discriminant Analysis (LDA), a Singular Value Decomposition (SVD) solver was chosen, since we are working with a large number of features. In this case the covariance matrix is not calculated. The determinant of this matrix tends to zero as the number of variables with statistical dependence increases. The absolute threshold for a singular value of a variable to be considered significant is set to 1x10^-5. The Ensemble Extra-Tree Classifier (ETC) was used with the number of trees in the forest equal to 180, the maximum depth of the tree 50 and log-loss was set as optimization criterium. The Random Forest (RF) classifier was configured as same as Ensemble Extra-Tree Classifier. The ensemble Bagging Classifier (BC) was configured to use the decision tree classifier as base estimator. The number of estimators was set to 15. All other hyperparameters used are scikit-learn defaults [⁵⁴].

On the other hand, in the proposed CNN experiments, all pretrained networks used the first-order gradient-based optimization algorithm of stochastic objective functions: ADAM (Adaptive Moment estimation) [⁵⁶]. The initial learning rate was set to 1x10^-3 and “softmax” activation function was chosen. Binary-crossentropy was used as loss function for binary classification model and categorical-crossentropy for three classes experiment. To improve performance when the metric has stopped the convergence, the system was configured with a reduce learning rate strategy. The factor by which the learning rate was reduced was set to ½. The minimum learning rate accepted was 1x10^-6. The system was also configured for early stopping in order to avoid overfitting by monitoring the validation loss.

The two-classes scenario

For the performance evaluation when there are two classes to discriminate, the following parameters were used as a metric: Accuracy, Specificity, Sensitivity, and Precision. The Accuracy (A_c) is defined as

$A_c=\frac{T_p+T_n}{P+N}=\frac{T_p+T_n}{T_p+T_n+F_p+F_n}$

The Specificity (S_p), selectivity or the true-negative rate (T_NR) can be written as

$S_p=T_{NR}=\frac{T_n}{F_p+T_n}=\frac{T_n}{N}$

The Sensitivity (Se), recall, hit-rate or the true-positive is formulated as

$S_e=T_{PR}=\frac{T_p}{T_p+F_n}=\frac{T_p}{P}$

The Precision (P_r) is defined as

$P_r=\frac{T_p}{T_p+F_p}$

where P is the total of positives, N is the total of negatives, T_p is the number of true positives, T_n is the number of true negatives, F_p is the number of false positives and F_n is the number of false negatives.

Table 2
Feature extractor framework experiment configuration.

FEF configuration	Number of features
Cartesian moments	16
Central moments	16
Hu-moments	7
Log-Hu-moments	7
Gray scale histogram	64
Gray scale statistics	32
Gray-level co-occurrence matrices (GLCM)	108
Zernike moments	25
Histogram of Oriented Gradients (HOG)	128
Local Binary Patterns (LBP)	32
Local Directional Patterns (LDP)	32
Frequency domain linear-subband energy	16
Scaling features algorithm	not used
Selection feature algorithm	correlation: 0.75

Table 3
FEF-MLF performance evaluation for Covid-19 vs. Pneumonia Viral.

Algorithm	Accuracy			Specificity			Sensitivity			Precision
	Max.	Avg.	Min.	Max.	Avg.	Min.	Max.	Avg.	Min.	Max.	Avg.	Min.
HGB	100.0	99.25	98.39	100.0	99.36	98.38	100.0	98.96	97.78	100.0	98.27	95.38
LDA	100.0	99.17	98.39	100.0	99.51	98.64	100.0	98.25	95.65	100.0	98.70	96.15
XGB	98.83	98.71	98.60	99.50	99.43	99.35	97.22	96.78	96.35	98.63	98.42	98.21
GB	99.60	98.23	97.38	99.73	98.31	97.32	100.0	98.02	95.45	99.19	95.43	92.31
ET	99.40	98.08	96.57	99.20	97.61	95.84	100.0	99.46	97.69	97.54	93.44	88.43
RF	99.19	97.75	96.17	99.20	97.32	95.49	100.0	99.02	97.64	97.54	92.61	86.78
BC	98.79	96.63	95.36	98.94	96.68	94.52	100.0	96.55	92.70	96.72	90.90	84.09

Table 4
TFL performance evaluation for Covid-19 vs. Pneumonia Viral.

Algorithm	Acc.	Spec.	Sens.	Prec.
Inception V3	100.0	100.0	100.0	100.0
MobileNet	99.80	100.0	99.72	100.0
DenseNet 121	99.80	99.36	100.0	99.71
XceptionNet	99.60	99.24	99.73	99.73
EfficientNet B7	99.60	99.73	99.24	99.24
VGG 19	99.40	98.43	99.73	99.46
InceptionResNeT V2	98.99	99.46	97.62	98.40

Table 5
Performance evaluation for Covid-19 vs. Pneumonia Viral results presented by others works.

Publication	Method	Acc.	Spec.	Sens.	Prec.
Panetta et al. [39]	Fibonacci Patterns	99.14	99.26	98.44	95.45
Panwar et al. [11]	nCOVnet - CNN	97.62	78.57	97.61	_____
Sheykhivand et al. [32]	CNN -LSTM (II)	96.90	96.80	97.10	96.75
Tamal et al. [35]	FE-ML (SVM)	95.20	85.00	99.60	_____
Karakanis et al. [19]	CNN - RESNET8	98.70	98.30	100.0	_____

In the two-classes scenario we have the classification between COVID-19 diagnosis versus viral pneumonia. Table 3 presents the results using the extracted features in conjunction with machine learning algorithms, Table 4 shows the performance results using the Pre-trained CNNs and in Table 5 is shown the results obtained from other works.

Scenario with more than two classes

In scenarios with more than two classes, the confusion matrix is used. The main diagonal of the confusion matrix indicates the ratio between the true positive rate of a class for the total instances of the respective class.

$P_r\left[k\right]=\frac{T_p\left[k\right]}{P\left[k\right]}$

The average accuracy is computed by the weighted average of the positive rate of each class.

$A_c=\frac{{{\displaystyle \sum }}_{k=1}^N{T}_p\left[k\right]}{{{\displaystyle \sum }}_{k=1}^{N}P\left[k\right]}$

In the three classes scenario we have to identify the correct class between COVID-19 diagnosis versus Viral Pneumonia versus Normal. Table 6 contains the results using the extracted features dataset in conjunction with machine learning algorithms and the results obtained using pre-trained CNNs. In Table 7 is presented, for a comparative analysis, the results obtained from other works. Finally, in Figure 7 presents one example of the confusion matrix evaluated on the test dataset for MobileNet approaching.

DISCUSSION

In the last 3 years the SARS-COVID-19 virus has been responsible for a pandemic. COVID-19 caused a fatal disease that spread rapidly around the world. Since the beginning of that pandemic the chest radiography (2D) has been an important screening tool that features a non-invasive approach. Addressing machine learning and deep learning approaching, technology has struggled to overcome the shortage of health professionals, especially when screening for the disease. In our proposed experiment, several machine learning algorithms were used for computer-assisted identification of COVID-19. To highlight that is a subject of extreme global importance, the authors of [⁵⁷] presented a survey of 1715 publications on the use of machine learning and deep learning techniques for disease screening of which, 265 articles were selected for construction of their review work.

Figure 7
Confusion matrix based on dataset for MobileNet: (a) two-classes scenario; and (b) three-classes scenario.

X-ray is a low-cost, rapidly produced exam and considered the gold standard for identifying changes in bones, lungs, heart and spinal cord [⁵⁸-⁵⁹ ]. Furthermore, it is used globally in hospitals and clinics, x-rays are undoubtedly the radiographic medium that produces the most exams. According to the cited references, researchers involved with the subject show greater interest in the CNN architecture in the investigation of COVID-19. Thinking about the sensitivity of medical data, in our work we were careful to use a database with 21,165 (see section Input dataset description) cases available in Kaggle and that gathers x-ray images from 8 different repositories, usually without adopting a protocol as to the orientation of the anteroposterior (AP) and lateral (lateral) incidences. Even so, with the contraposition of image acquisition, this number of cases is considered sufficient to avoid overfitting and consequently high false-positive values.

As shown in Table 3 and Table 4, the results presented in scenario 1 presents a good performance for both approaches (1) feature extraction in conjunction with machine learning techniques for classification and (2) transfer-learning based on pre-trained CNN networks.

CNNs have a very large degree of freedom that makes it possible to generalize a solution. On the other hand, they require a very large computational effort when compared to other machine learning techniques. Pre-trained CNNs networks have shown a very good classification performance [⁸-³⁹]. We also try to show that good results in COVID - 19 can be obtained without adding too much to the pre-trained network. In our example, a single model was conceived for all the evaluated CNN networks used, and one more layer was added to the existed pre-trained model. Very good results were obtained, as well as the best results reported in several works (Table 5 and Table 7).

Although the Inception V3 network (Table 4) presented 100% success for the COVID versus Viral Pneumonia discrimination, it does not necessarily mean that it is better than the others that obtained very similar results, as there is a certain dependence on the test dataset chosen. For the scenario with three classes, the MobileNet (Table 6) presented slightly higher results in terms of accuracy.

The machine learning (ML) based solutions are less generalist, then to obtain CNN’s similar results, a well-built feature database is needed. This means that is necessary a vector space of features that efficiently discriminates the spectral signatures embedded (somehow) in chest radiography images. We believe that a good performance in a pattern classification system depends more intensely on how the patterns are constructed than on the algorithm chosen for the classification. For the examples shown in this work, a dimension space of 432 features relevance were used to ML training. The XGB and the HGB presented a very good results for the three-class scenario, similar to pre-trained CNNs (Table 6). They have shown good convergence and stability even in large spaces of features.

Table 6
FEF-MLF/TLF Performance evaluation for Covid-19 vs. Pneumonia Viral vs. Normal.

Algorithm	Acc.	Covid	Normal	Pneu.
XGBoost Classifier	96.84	97.27	96.85	95.60
Hist. Grad. Boosting	96.79	94.40	98.70	88.76
Bagging Classifier	94.00	89.31	97.46	80.48
Linear Discriminant	93.95	89.30	96.08	90.39
Ensemble Extra-Tree	93.25	83.38	98.26	82.03
Gradient Boosting	94.34	87.93	97.80	85.46
Random Forest	93.21	83.78	98.27	80.43
MobileNet	98.48	96.94	99.22	95.64
XceptionNet	97.82	97.17	99.51	94.03
InceptionResNet V2	97.03	97.60	97.40	92.80
DenseNet 121	97.03	96.93	97.91	91.95
Inception V3	93.66	93.73	95.25	83.12
EfficientNet B7	92.74	84.70	95.81	85.90
VGG 19	91.56	75.52	97.62	83.13

Table 7
Performance evaluation for Covid-19 vs. Pneumonia Viral vs. Normal results presented by others work.

Publication	Method	Acc.	Cov.	Nor.	Pneu.
Panetta et al. [39]*	Fibonacci Patterns	98.71	98.00	98.00	100.0
Sheykhivand et al. [32]	LSTM (scen. V)	98.10	99.50	97.10	97.90
Dixit et al. [34]	CoV2 -Detect-Net	99.35	100.0	98.69	99.35
Demir [33]*	DeepCoro - Net	100.0	100.0	100.0	100.0
Karakanis & Leontidis [19]	CNN - ResNet8	98.70	99.00	98.00	97.00
Rajpal et al. [30]*	CNN - ResNet-50	97.88	98.73	99.36	95.54
Islam et al. [43]*	CNN - LTSM	99.12	99.34	100.0	98.03

• *The indices were calculated according to the metrics presented in section 3.4 using the results obtained from the confusion matrix

It is also clear to be observed that a specific problem involving the discrimination of two classes is, in most cases, easier to be solved than when adding more classes to the problem. Comparing Tables 3 and 6, an estimate of the increase in classification difficulty can be observed by the small decreasing in the measured performance quantities.

In general, when the number of classes is increased, it is observed that CNNs perform better. Figure 4 illustrates this point of view. To improve the performance of MLs in these circumstances, it is necessary to investigate an efficient approach for a feature selection given the statistical dependence of variables and the ability of a feature to report the spectral signatures of interest. It may also be interesting to have an adequate feature scaling, as they have a very large variation in the dynamic range between them. This was not the subject of this work, but it is certainly a topic of interest.

CONCLUSION

We believe that the main purpose of this work has been achieved. We sought to investigate two distinct approaches to handling COVID - 19 automatic diagnostics and have showed that machine learning-based approaches can produce results as good as the best deep learning techniques. However, to obtain this level of performance using machine learning (ML) approach, it is necessary to build a feature vector space that is, as much as possible, representative of the spectral signatures of interest. On the other hand, machine learning-based techniques require considerably less computational effort when compared to deep learning approaching. Looking at it from this point of view, ML could be more interesting than implementations using deep learning, for example, in applications involving embedded systems when there are computational restrictions.

The construction of a large feature space (> 400) from chest X-Ray images proved to be suitable for its use for ML training for COVID - 19 diagnoses. Since feature scaling selection techniques have not been investigated yet, this is one of the possible subjects for future work.

Acknowledgments

The present study was supported by CAPES and CNPq, two Brazilian government agencies for scientific and technological development.

REFERENCES

• ¹
  Wu F, Zhao S, Yu B, Chen Y, Wang W, Song ZG, et al. A new coronavirus associated with human respiratory disease in China. Nature. 2020 feb; 579(7798): 265-9. https//doi.org/10.1038/s41586-020-2008-3
  » https//doi.org/10.1038/s41586-020-2008-3
• ²
  Lai C, Shih T, Ko W, Tang H, Hsueh P. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): the epidemic and the challenges. Int. J. Antimicrob. Ag. 2020 mar; 55(3). https://doi.org/10.1016/j.ijantimicag.2020.105924
  » https://doi.org/10.1016/j.ijantimicag.2020.105924
• ³
  Duanmu H, Ren T, Li H, Mehta N, Singer AJ, Levsky JM, et al. Deep learning of longitudinal chest X-ray and clinical variables predicts duration on ventilator and mortality in COVID-19 patients. BioMed. Eng. OnLine. 2022 oct; 21(1): 1-15. https://doi.org/10.1186/s12938-022-01045-z
  » https://doi.org/10.1186/s12938-022-01045-z
• ⁴
  Tiwari S, Chanak P, Singh SK. A Review of the Machine Learning Algorithms for Covid-19 Case Analysis. IEEE T. Art. Intellig. 2023 feb; 4(1): 44-59. https//doi.org/10.1109/TAI.2022.3142241
  » https//doi.org/10.1109/TAI.2022.3142241
• ⁵
  Hussain AA, Bouachir O, Al-Turjman F, Aloqaily M. AI Techniques for COVID-19. IEEE Access. 2020 jul; 8: 128776-95. https://doi.org/10.1109/ACCESS.2020.3007939
  » https://doi.org/10.1109/ACCESS.2020.3007939
• ⁶
  Alahmari SS, Altazi B, Hwang J, Hawkins S, Salem T. A Comprehensive Review of Deep Learning-Based Methods for COVID-19 Detection Using Chest X-Ray Images. IEEE Access. 2022 sep; 10: 100763-100785. https://doi.org/10.1109/ACCESS.2022.3208138
  » https://doi.org/10.1109/ACCESS.2022.3208138
• ⁷
  Islam MM, Karray F, Alhajj R, Zeng J. A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19). IEEE Access. 2021 feb; 9:30551-72. https://doi.org/10.1109/ACCESS.2021.3058537
  » https://doi.org/10.1109/ACCESS.2021.3058537
• ⁸
  Cannata S, Paviglianiti A, Cirrincione G, Cirrincione M. Deep Learning Algorithms for Automatic COVID-19 Detection on Chest X-Ray Images. IEEE Access. 2022 nov; 10: 119905-13. https://doi.org/10.1109/ACCESS.2022.3221531
  » https://doi.org/10.1109/ACCESS.2022.3221531
• ⁹
  Hertel R, Benlamri R. COV-SNET: A deep learning model for X-ray-based COVID-19 classification. Inf. Med. Unloc. 2021 nov; 24:100620. https://doi.org/10.1016/j.imu.2021.100620
  » https://doi.org/10.1016/j.imu.2021.100620
• ¹⁰
  Fang Z, Ren J, MacLellan , Li H, Zhao H, Hussain A, et al. A Novel Multi-Stage Residual Feature Fusion Network for Detection of COVID-19 in Chest X-Ray Images. IEEE T. Mol. Biol. Mul-Scl. Communicat. 2022 jul; 8(1): 17-27. https://doi.org/10.1109/TMBMC.2021.3099367
  » https://doi.org/10.1109/TMBMC.2021.3099367
• ¹¹
  Nigam B, Nigam A, Jain R, Dodia S, Arora N, Annappa B. COVID-19: Automatic detection from X-ray images by utilizing deep learning methods. Expert Syst. Appl. 2021 aug; 176: 114883. https://doi.org/10.1016/j.eswa.2021.114883
  » https://doi.org/10.1016/j.eswa.2021.114883
• ¹²
  Brunese L, Mercaldo F, Reginelli A, Santone A. Explainable deep learning for pulmonary disease and coronavirus COVID-19 detection from X-rays. Comput. Meth. Prog. Bio. 2020 nov; 196: 105608. https://doi.org/10.1016/j.cmpb.2020.105608
  » https://doi.org/10.1016/j.cmpb.2020.105608
• ¹³
  Das AK, Kalam S, Kumar C, Sinha D. TLCoV - An automated Covid-19 screening model using Transfer Learning from chest X-ray images. Chaos Soliton. Fract. 2021 mar; 144:110713. https://doi.org/10.1016/j.chaos.2021.110713
  » https://doi.org/10.1016/j.chaos.2021.110713
• ¹⁴
  Alhudhaif A, Polat K, Karaman O. Determination of COVID-19 pneumonia based on generalized convolutional neural network model from chest X-ray images. Expert Syst. Appl. 2021 oct; 180: 1-9. https://doi.org/10.1016/j.eswa.2021.115141
  » https://doi.org/10.1016/j.eswa.2021.115141
• ¹⁵
  Minaee S, Kafieh R, Sonka M, Yazdani S, Soufie GJ. Deep-COVID: Predicting COVID-19 from chest X-ray images using deep transfer learning. Med. Image Anal. 2020 oct; 65: 101794. https://doi.org/10.1016/j.media.2020.101794
  » https://doi.org/10.1016/j.media.2020.101794
• ¹⁶
  Oyelade ON, Ezugwu AE, Chiroma H. CovFrameNet: An Enhanced Deep Learning Framework for COVID-19 Detection. IEEE Access. 2021 may; 9: 77905-77919. https://doi.org/10.1109/ACCESS.2021.3083516
  » https://doi.org/10.1109/ACCESS.2021.3083516
• ¹⁷
  Jain G, Mittal D, Thakur D, Mittal MK. A deep learning approach to detect Covid-19 coronavirus with X-Ray images. Biocybern. Biomed. Eng. 2020 oct; 40: 1391-1405. https://doi.org/10.1016/j.bbe.2020.08.008
  » https://doi.org/10.1016/j.bbe.2020.08.008
• ¹⁸
  Karakani S, Leontidis G. Lightweight deep learning models for detecting COVID-19 from chest X-ray images. Comput. Biol. Med. 2021 mar; 130: 104181. https://doi.org/10.1016/j.compbiomed.2020.104181
  » https://doi.org/10.1016/j.compbiomed.2020.104181
• ¹⁹
  Vidal PL, Moura J, Novo J, Ortega M. Multi-stage transfer learning for lung segmentation using portable X-ray devices for patients with COVID-19. Expert Syst. Appl. 2021 jul, 173: 1-13. https://doi.org/10.1016/j.eswa.2021.114677
  » https://doi.org/10.1016/j.eswa.2021.114677
• ²⁰
  Togac ar M, Ergen B, C¨omert Z. COVID-19 detection using deep learning models to exploit Social Mimic Optimization and structured chest X-ray images using fuzzy color and stacking approaches. Comput. Biol. Med. 2020 jun; 121: 103805. https://doi.org/10.1016/j.compbiomed.2020.103805
  » https://doi.org/10.1016/j.compbiomed.2020.103805
• ²¹
  Ohata EF, Bezerra GM, das Chagas JVS, Neto AV, Albuquerque ABB, Albuquerque VH, et al. Automatic Detection of COVID-19 Infection Using Chest X-Ray Images Through Transfer Learning. IEEE/CAA J. Autom. Sin. 2021 sep; 8(1): 239-248. https://doi.org/10.1109/JAS.2020.1003393
  » https://doi.org/10.1109/JAS.2020.1003393
• ²²
  Ismael AM, Sengur A. Deep learning approaches for COVID-19 detection based on chest X-ray images. Expert Syst. Appl. 2021 feb; 164: 1-11. https://doi.org/10.1016/j.eswa.2020.114054
  » https://doi.org/10.1016/j.eswa.2020.114054
• ²³
  Zhang Z, Chen B, Luo AY. Deep Ensemble Dynamic Learning Network for Corona Virus Disease 2019 Diagnosis. IEEE T. Neur. Net. Lear. 2022 sep; 1-15. https://doi.org/10.1109/TNNLS.2022.3201198
  » https://doi.org/10.1109/TNNLS.2022.3201198
• ²⁴
  Zhou C, Song J, Zhou S, Zhang Z, Xing J. COVID-19 Detection Based on Image Regrouping and Resnet-SVM Using Chest X-Ray Images. IEEE Access. 2021 jun; 9: 81902-12. https://doi.org/10.1109/ACCESS.2021.3086229
  » https://doi.org/10.1109/ACCESS.2021.3086229
• ²⁵
  Ahishali M, Degerli A, Yamac M, Kiranyaz S, Chowdhury MEH, Hameed K, et al. Advance Warning Methodologies for COVID-19 using Chest X-Ray Images. IEEE Access. 2021 jun: 9: 41052-65. https://doi.org/10.1109/ACCESS.2021.3064927
  » https://doi.org/10.1109/ACCESS.2021.3064927
• ²⁶
  Sharifrazi D, Alizadehsani R, Roshanzamir M, Joloudari JH, Shoeibi A, Jafari M, et al. Fusion of convolution neural network, support vector machine and Sobel filter for accurate detection of COVID-19 patients using X-ray images. Biomed. Signal Proces. 2021 jul; 68: 1-14. https://doi.org/10.1016/j.bspc.2021.102622
  » https://doi.org/10.1016/j.bspc.2021.102622
• ²⁷
  Fan Y, Liu J, Yao R, Yuan X. COVID-19 Detection from X-ray Images using Multi-Kernel-Size Spatial-Channel Attention Network. Pattern Recogn. 2021 nov; 119: 108055. https://doi.org/10.1016/j.patcog.2021.108055
  » https://doi.org/10.1016/j.patcog.2021.108055
• ²⁸
  Joshi RC, Yadav S, Pathak VK, Malhotra HS, Khokhar HVS, Pariah A, et al. A deep learning-based COVID-19 automatic diagnostic framework using chest X-ray images. Biocybern. Biomed. Eng. 2021 mar; 41(1): 239-54. https://doi.org/10.1016/j.bbe.2021.01.002
  » https://doi.org/10.1016/j.bbe.2021.01.002
• ²⁹
  Rajpal S, Lakhyani N, Singh A, Kohli R, Kumar N. Using handpicked features in conjunction with ResNet-50 improved detection of COVID-19 from chest X-ray images. Chaos Soliton. Fract. 2021 apr;145:110749. https://doi.org/10.1016/j.chaos.2021.110749
  » https://doi.org/10.1016/j.chaos.2021.110749
• ³⁰
  Das S, Roy SD, Malakar S, Vel´asquez JD, Sarkar R. Bi-Level Prediction Model for Screening COVID-19 Patients Using Chest X-Ray Images. Big Data Res. 2021 jul; 25: 1-10. https://doi.org/10.1016/j.bdr.2021.100233
  » https://doi.org/10.1016/j.bdr.2021.100233
• ³¹
  Sheykhivand S, Mousavi Z, Mojtahedi S, Rezaii TY, Farzamnia A, Meshgini S, et al. Developing an efficient deep neural network for automatic detection of COVID-19 using chest X-ray images. Alexandria Eng. J. 2021 jun; 60: 2885-903. https://doi.org/10.1016/j.aej.2021.01.011
  » https://doi.org/10.1016/j.aej.2021.01.011
• ³²
  Demir F. DeepCoroNet: A deep LSTM approach for automated detection of COVID-19 cases from chest X-ray images. Appl. Soft Comput. 2021 aug; 103: 107160. https://doi.org/10.1016/j.asoc.2021.107160
  » https://doi.org/10.1016/j.asoc.2021.107160
• ³³
  Dixit A, Mani A, Bansal R. CoV2-Detect-Net: Design of COVID-19 prediction model based on hybrid DE-PSO with SVM using chest X-ray images. Infor. Sciences. 2021 sep; 571:1-17. https://doi.org/10.1016/j.ins.2021.03.062
  » https://doi.org/10.1016/j.ins.2021.03.062
• ³⁴
  Tamal M, Alshammari M, Alabdullah M, Hourani R, Alola HA, Hegazi TM. An integrated framework with machine learning and radiomics for accurate and rapid early diagnosis of COVID-19 from Chest X-ray. Expert Syst. Appl. 2021 oct; 180: 1-8. https://doi.org/10.1016/j.eswa.2021.115152
  » https://doi.org/10.1016/j.eswa.2021.115152
• ³⁵
  Mohammed MA, Abdulkareem KH, Al-Waisy A, Mostafa S, Al-Fahdawi S, Dinar AM, et al. Benchmarking Methodology for Selection of Optimal COVID-19 Diagnostic Model Based on Entropy and TOPSIS Methods. IEEE Access. 2020 may; 8: 99115-31. https://doi.org/10.1109/ACCESS.2020.2995597
  » https://doi.org/10.1109/ACCESS.2020.2995597
• ³⁶
  Anter AM, Oliva D, Thakare A, Zhang Z. AFCM-LSMA: New intelligent model based on L´evy slime mould algorithm and adaptive fuzzy C-means for identification of COVID-19 infection from chest X-ray images. Adv. Eng. Inform. 2021 aug; 49: 1-13. https://doi.org/10.1016/j.aei.2021.101317
  » https://doi.org/10.1016/j.aei.2021.101317
• ³⁷
  Signoroni A, Savardi M, Benini S, Adami N, Leonardi R, Gibellini P, et al. BS-Net: Learning COVID-19 pneumonia severity on a large chest X-ray dataset. Med. Image Anal. 2021 jul; 72: 102046. https://doi.org/10.1016/j.media.2021.102046
  » https://doi.org/10.1016/j.media.2021.102046
• ³⁸
  Panetta K, Sanghavi F, Agaian S, Madan N. Automated Detection of COVID-19 Cases on Radiographs using Shape-Dependent Fibonacci-p Patterns. IEEE J. Biomed. Health. 2021 mar; 25(6): 852-1863. https://doi.org/10.1109/JBHI.2021.3069798
  » https://doi.org/10.1109/JBHI.2021.3069798
• ³⁹
  Quan H, Xu X, Zheng T, Li Z, Zhao M, Cui X. DenseCapsNet: Detection of COVID-19 from X-ray images using a capsule neural network. Comput. Biol. Med. 2021 jun; 133: 104399. https://doi.org/10.1016/j.compbiomed.2021.104399
  » https://doi.org/10.1016/j.compbiomed.2021.104399
• ⁴⁰
  Catal’a ODT, Igual I, P’erez-Benito F, Escriv’a D, Castell’o VLR, P’erez-Cort’es J. Bias Analysis on Public X-Ray Image Datasets of Pneumonia and COVID-19 Patients. IEEE Access. 2021 mar; 9: 42370-83. https://doi.org/10.1109/ACCESS.2021.3065456
  » https://doi.org/10.1109/ACCESS.2021.3065456
• ⁴¹
  Wang Z, Xiao Y, Li Y, Zhang J, Lu F, Hou M, et al. Automatically discriminating and localizing COVID-19 from community-acquired pneumonia on chest X-rays. Pattern Recogn. 2021 feb; 110: 1-9. https://doi.org/10.1016/j.patcog.2020.107613
  » https://doi.org/10.1016/j.patcog.2020.107613
• ⁴²
  Islam MZ, Islam MM, Asraf A. A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images. Inf. Med. Unloc. 2020 mar; 20: 100412. https://doi.org/10.1016/j.imu.2020.100412
  » https://doi.org/10.1016/j.imu.2020.100412
• ⁴³
  Flusser J, Suk T, Zitov´a B. Moment and Moment Invariants in Pattern Recognition. John Wiley and Sons. 2009. 296 p.
• ⁴⁴
  Hu MK. Visual pattern recognition by moment invariants. IRE T. Inform. Theor. 2009 apr; 8(2): 79-187.
• ⁴⁵
  Van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017 oct; 77(21): e104-e107. https://doi.org/10.1158/0008-5472.CAN-17-0339
  » https://doi.org/10.1158/0008-5472.CAN-17-0339
• ⁴⁶
  Löfstedt T, Brynolfsson P, Asklund T, Nyholm T, Garpebring A. Gray-level invariant Haralick texture features. PLoS ONE. 2019 feb; 14(2): 0212110. https://doi.org/10.1371/journal.pone.0212110
  » https://doi.org/10.1371/journal.pone.0212110
• ⁴⁷
  Haralick RM, Shanmugam K. Textural features for image classification. IEEE T. Syst. Sci. Cyb. 1973 nov; 6(6): 610-21. https://doi.org/10.1109/TSMC.1973.4309314
  » https://doi.org/10.1109/TSMC.1973.4309314
• ⁴⁸
  Zernike F. Beugungstheorie des Schneidenver-fahrens und seiner verbesserten form, der Phasenkontrastmethode. Physica. 1934 may; 1(8): 689-704. https://doi.org/doi: 10.1016/S0031-8914(34)80259-5
  » https://doi.org/doi: 10.1016/S0031-8914(34)80259-5
• ⁴⁹
  Braga C, Guardani R, Lopes E, Freitas A. Sparse Zernike Fitting for Dynamic LAS Tomographic Images of Temperature and Water Vapor Concentration. IEEE T. Instrum. Meas. 2022 apr; 71: 1-1. https://doi.org/10.1109/TIM.2022.3165802
  » https://doi.org/10.1109/TIM.2022.3165802
• ⁵⁰
  D´eniz O, Bueno G, Salido J, De la Torre F. Face recognition using Histograms of Oriented Gradients. Pattern Recogn. Lett. 2011 apr; 32(12): 1598-603. https://doi.org/10.1016/j.patrec.2011.01.002
  » https://doi.org/10.1016/j.patrec.2011.01.002
• ⁵¹
  Zhao G, Pietik¨ainen M. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE T. Pattern Anal. (2007) 29(6), 915-28. https://doi.org/10.1109/TPAMI.2007.1110
  » https://doi.org/10.1109/TPAMI.2007.1110
• ⁵²
  Zhong F, Zhang J. Face recognition with enhanced local directional patterns. Neurocomputing. 2013 nov; 119: 375-84. https://doi.org/10.1016/j.neucom.2013.03.034
  » https://doi.org/10.1016/j.neucom.2013.03.034
• ⁵³
  Shabat AM, Tapamo J. A comparative study of the use of local directional pattern for texture-based informal settlement classification. J. Appl. Res. Technol. 2017 jun; 15: 250-8. https://doi.org/10.1016/j.jart.2016.12.009
  » https://doi.org/10.1016/j.jart.2016.12.009
• ⁵⁴
  Zwanenburg A, Leger S, Vallières M, Löck S. Image biomarker standardization initiative - feature definitions. In eprint arXiv [Internet]. 2016 [cited 2023 mai 20]; 2016:1612.07003. Available from: https://arxiv.org/abs/1602.07003
  » https://arxiv.org/abs/1602.07003
• ⁵⁵
  Pedregosa M, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn Res. 2011; 12: 2825-30.
• ⁵⁶
  Chen T, Guestrin C. XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2016 August; [place unknown]. ACM; 2016. p. 785-794. https://doi.org/10.1145/2939672.2939785
  » https://doi.org/10.1145/2939672.2939785
• ⁵⁷
  Diederik PK, Jimmy LB. Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations (ICLR 2015); 2015 jan 30; San Diego, CA, USA: Cornell; 2015. p. 1-15. https://doi.org/10.48550/arXiv.1412.6980
  » https://doi.org/10.48550/arXiv.1412.6980
• ⁵⁸
  Liu C, Wang X, Liu C, Sun Q, Peng W. Differentiating novel coronavirus pneumonia from general pneumonia based on machine learning. BioMedi Eng. OnLine. 2020 Aug; 19(1): 1-14. https://doi.org/10.1186/s12938-020-00809-9
  » https://doi.org/10.1186/s12938-020-00809-9
• ⁵⁹
  Lasker A, Obaidullah SM, Chakraborty C, Roy K. Application of Machine Learning and Deep Learning Techniques for COVID-19 Screening Using Radiological Imaging: A Comprehensive Review. SN Comput. Sci. 2022 jul;4(1):65.
• ⁶⁰
  Narin A, Kaya C, Pamuk Z. Automatic Detection of Coronavirus Disease (COVID-19) Using X-ray Images and Deep Convolutional Neural Networks. arXiv Prepr. arXiv [Internet]. 2020 [cited 2023 mai 25]; 2003.10849. Available from: https://arxiv.org/abs/2003.10849
  » https://arxiv.org/abs/2003.10849

• Funding:

  This research received no external funding.

Edited by

Editor-in-Chief:

Alexandre Rasi Aoki

Associate Editor:

Fabio Alessandro Guerra

Publication Dates

• Publication in this collection
  30 Oct 2023
• Date of issue
  2023

History

• Received
  14 June 2023
• Accepted
  28 July 2023