A New Approach for Multivariate Data Analysis in Interlaboratory Comparisons Based on Multidimensional Scaling and Robust Confidence Ellipse

Abstract

Interlaboratory comparisons (IC) present a challenge related to multivariate data analysis. ISO 13528:2015 is a reference document for interlaboratory comparisons. This standard does not provide descriptions of statistical methods for multivariate analysis and, according to our best knowledge, there is no practical guidance for the organizing and evaluation of multivariate data analysis for interlaboratory comparisons available. Due to this reason, some researchers have made efforts to develop methodologies that make it possible to analyze multivariate data in IC. Generally, these approaches are based on dimensionality-reduction methods like principal component analysis. This paper proposes a new approach to reduce the dimensionality of large data set and check the performance of laboratories based on multidimensional scaling (MDS) and robust confidence ellipse/ellipsoid (RCE). MDS is a multivariate analysis technique that allows grouping laboratories according to their similarity in a Euclidean space. On the other hand, RCE is a statistical method for outlier detection in a multivariate data set. In this work, it is proposed to combine MDS and RCE to evaluate laboratory proficiency in interlaboratory comparison. This methodology was compared with the multivariate z-score and both methodologies identified the same outlying laboratories. This preliminary result indicates that MDS/RCE is promising for classifying IC results.

Keywords:
multidimensional scaling; robust confidence ellipse; interlaboratory comparisons; multivariate analysis

Introduction

Interlaboratory comparison (IC) seeks to assess the performance of a given measurement method. To organize an IC, the same or similar items test are sent to each participating under predetermined conditions.¹ In some cases, the reported results by participants are multivariate data and univariate techniques are not suitable to analyze this kind of data.² For this situation, multivariate analysis techniques can be used for investigating the inherent structure of data without losing valuable information of measurement.

Multivariate analysis, in a general way, refers to all statistical methods that simultaneously analyze multiple measures in each individual or object (in this case laboratory).³ The official document for interlaboratory comparison, ISO 13528:2015,¹ does not provide descriptions for multivariate analysis. Due to this gap left by the official document, some researchers have been proposed many ways to realize multivariate data analysis in interlaboratory comparison. Sheen et al.² have proposed multivariate z-score to identify outlying laboratory results.

In the proposed methodology by Sheen et al.,² the spectra should be grouped so that the cluster consists of multiple nuclear magnetic resonance (NMR) spectra of the k-th sample provided by the participants. From these clusters, the interspectral distance matrix D_k is calculated, whose elements are the distances D_ij,k = d(s_i,k, s_j,k) where s_i,k is the spectrum of laboratory i, s_j,k is the spectrum of laboratory j (both belonging to the conglomerate S_k) and d(.) is a multivariate distance measure. The authors suggest Euclidean, Mahalanobis, Hellinger, Kullback-Leibler, Jensen-Shannon and Jeffreys distances.²

Based on the values D_ij,k, the average distance $${\hat{D}}_{\mathrm{i},\mathrm{k}}=(1/\mathrm{n})\sum _{\mathrm{j}}{\mathrm{D}}_{\mathrm{i}\mathrm{j},\mathrm{k}}$$ is computed. The $${\hat{D}}_{\mathrm{i},\mathrm{k}}$$ values need to be fitted to a given probability distribution for each laboratory i. After this step, the matrix Z is obtained, where Z_i is the z-score vector of the i-th laboratory. The vector Z_i is obtained by $${\mathrm{Z}}_{\mathrm{i},\mathrm{k}}={\mathrm{C}}^{\ast -1}\left({\mathrm{C}}_{\mathrm{k}}\left({\hat{\mathrm{D}}}_{\mathrm{i},\mathrm{k}}\right)\right)$$ where C_k is the cumulative distribution function after being fitted to the conglomerate k and C* is the corresponding standard distribution function.²

In this approach, the next step is to perform the principal component analysis (PCA) on the Z matrix. In the PCA model, T = ZP^t where T is the matrix of the scores of the principal components and P is the matrix of the loadings. The most significant L principal components are identified by obtaining T_L = ZP_L^t. For each participant, the Euclidean norm ||T_i,L|| is calculated and these statistical distances are adjusted to a new probability distribution with distribution function $$\hat{\mathrm{C}}$$.²

This new distribution has a z-score associated with it. The authors called the projected statistical score ($${\hat{Z}}_{\mathrm{i}}$$). This score is calculated using $${\hat{\mathrm{Z}}}_{\mathrm{k}}={\hat{\mathrm{C}}}^{\ast -1}\left(\hat{\mathrm{C}}\left(\parallel {\mathrm{T}}_{\mathrm{i},\mathrm{L}}\parallel \right)\right)$$. If any $${\hat{Z}}_{\mathrm{i}}$$ value falls outside the 95% confidence interval, the corresponding laboratory is considered an outlier and removed from the data set. The process is repeated until no dataset falls outside the 95% confidence interval.² In this study, 10 participants provide a one-dimensional ¹H NMR spectrum and three of them were identified as outlying by the proposed methodology.

Another approach to deal with multivariate data was proposed by Viant et al.⁴ It was proposed principal component analysis (PCA) to clustering participating results. The items test sent to laboratories were synthetic metabolite mixtures and European flounder liver extracts (biological) from clean and contaminated sites. The goal of the intercomparison exercise was to evaluate the effectiveness of ¹H NMR metabolomics to generate comparable data sets from environmentally derived samples and each participant provides a one-dimensional spectrum. The associated PCA scores plots were used to visually assess if individual laboratories could reveal which metabolites discriminated the synthetic mixtures and biological samples. In both cases, the PCA approach allows concluding that there was a high degree of similarity across all laboratories. The comparability and precision of the laboratories participating were good, reflecting good results from NMR spectra.⁴

Gallo et al.⁵ have proposed a new performance index, named Q_p-score, to assess the laboratory performance in multi-component analyses. Eight nuclear magnetic resonance signals (3 for aldicarb, 1 for methamidophos, 2 for oxadixyl, and 2 for pirimicarb) were obtained by 36 participants in interlaboratory comparison. It was proposed the following parameter: $$Q_p=(a_i\cdot \overline{a})/{\sigma }_{\text{slope}}$$ where a_i is the slope of the calibration line determined by the i-th participant, $$\overline{a}$$ is the consensus slope value, and σ_slope is the inter-laboratory standard deviation on slopes. The Q_p -score is considered satisfactory when |Q_p| ≤ 2, questionable when 2 < |Q_p| < 3 and unsatisfactory when |Q_p| ≥ 3. The proposed methodology allows classifying the reported results by participants in each NMR signal. In general, 9 laboratories were classified as unsatisfactory, 2 were classified as questionable and 25 were classified as satisfactory.⁵

Other authors reported multivariate data analysis by PCA in interlaboratory comparisons. Danzer et al.⁶, Henrion⁷ and, Škrbić et al.⁸ suggested PCA to identify outlying laboratories in interlaboratory comparison. Minkkinen⁹ proposes principal components score plot to visualize the variation of the results between and within laboratories. Heininger et al.¹⁰ have proposed PCA to group the laboratories and identify the type of method used to analyze the samples by each participants. Aoki et al.¹¹ proposed multiple hypothesis testing to assess the equivalence among the laboratories measurements with respect to the reference laboratory. The authors suggest build a confidence regions between each participating and reference laboratory based on Wald statistic.¹¹

This paper proposes a new procedure to assess the performance of interlaboratory comparisons that are different from the methodologies mentioned above. A new methodology is derived from the concepts of outlying detection in two- and three-dimensional Euclidean space. The methodology is based on following steps.

Initially the data (Figure 1a) is arranged as matrix X_n×p where n represents the number of variables, that is, the chemical shift of the spectrum provided by i-th laboratory and p represents the number of participants in the interlaboratory comparison (Figure 1b).

Figure 1
The proposed methodology: (a) multivariate reported spectrum by each participating; (b) multivariate data set organized in matrix where the first column contains the chemical shift and the remaining columns contain the spectra reported by each participant; (c) multidimensional scaling combined with robust confidence ellipse (MDS/RCE 2D); (d) multidimensional scaling combined with robust confidence ellipsoid (MDS/RCE 3D).

The second step refers to performing a multidimensional scaling (MDS) on the multivariate data to transform them into a dimension that can be used to visualize the results of the laboratories in two or three dimensions. After that, a confidence limit based on a robust confidence ellipse/ellipsoid (RCE) is plotted to identify outlying laboratory result (Figures 1c and 1d). Points outside of robust confidence ellipse/ellipsoid provide evidence that laboratory result is an outlier with a specific confidence level (95%, for example). On the other hand, points inside of RCE are not considered outlying result with same confidence level.

Multidimensional scaling is a multivariate technique that allows revealing “hidden” structures in a multivariate data set.¹² In other words, it is a method that allows visualizing the similarity/dissimilarity among laboratories participating in an interlaboratory comparison which are represented as points in a two- or three-dimensional space.¹³ The proximity/distance between the points represents the similarity/dissimilarity among laboratories.¹³

Robust confidence ellipse/ellipsoid is a multivariate analysis technique for outlier detection. A confidence region is built from the variance-covariance matrix of original data set, which allows identifying if there are laboratories that differ statistically from the others at some specific significance level.¹⁴

The combination of these two techniques (Figure 1) constitutes the proposal for evaluating the performance of the results reported by the participants.

Methodology

Data set

The data used in this study can be obtained from the available data set described by Sheen et al.² In this data set, an interlaboratory comparison was carried out to assess the effectiveness of ¹H NMR metabolomics. Seven laboratories from the United States, one from Canada, one from the United Kingdom, and one from Australia participated in the intercomparison. Mixtures of synthetic metabolites and samples of biological origin from liver extracts of European flounder from clean and contaminated sites were analyzed.⁴

The data described by Sheen et al.² refers to adult female European flounder collected from Tyne in the United Kingdom. This is a biological sample selected from a polluted site. The fish were sacrificed, liver tissues were dissected, snap-frozen in liquid nitrogen, and stored at -80 °C until extraction by IC participating.⁴ The samples were extracted using methanol:chloroform:water method and Precellys-24 bead-based homogenizer (Stretton Scientific Ltd., U.K.).⁴

Each participating laboratory obtained a one-dimensional ¹H NMR spectrum. The spectra are reported as chemical shift frequencies with a range from 10.0 to 0.2 ppm. The region from 4.7 to 5.2 ppm was excluded due to water solvent suppression artifacts and the NMR spectra were renormalized. The spectra were binned with a bin width of 0.005 ppm, for a total of 1860 variables in each spectrum.²

In this work, the multivariate techniques of multi­dimensional scaling and robust confidence ellipse/ellipsoid (Figures 1c and 1d) were explored to assess the performance of laboratories participating in a laboratory intercomparison classifying them as outlier or not. The basis for the development of this methodology is described below.

Multidimensional scaling

Let p be the number of different laboratories (Figure 1b) and δ_ij the dissimilarity between laboratories i and j. The coordinates are gathered in the matrix X_n×p where n is the dimensionality of the solution to be specified. Thus, the column i of X_n×p provides the coordinates of the laboratory i (Figure 1b). Let d_ij(X) be the Euclidean distance (most used)¹⁵,¹⁶ between columns i and j defined as:

(1) $${\mathrm{d}}_{\mathrm{i}\mathrm{j}}(\mathbf{X})={\left(\sum _{s=1}^n{(x_{is}-x_{js})}^2\right)}^{1/2}$$

which is the shortest distance between laboratories i and j. In the equation 1, x_is is the spectrum of laboratory i and x_js is the spectrum of laboratory j.

The Euclidean distance is more favorable in visual representations because a more isotropic display is obtained using it.¹⁷ The purpose of multidimensional scaling is to find an X_n×p matrix that d_ij(X) is equal to δ_ij as much as possible.¹³

To obtain this matrix, the least squares MDS model is used, which consists of minimizing the equation:

(2) $${\sigma }^2(\mathbf{X})=\sum _{\mathrm{i}=2}^{\mathrm{p}}\sum _{\mathrm{j}=1}^{\mathrm{i}-1}{\mathrm{w}}_{\mathrm{i}\mathrm{j}}{({\delta }_{\mathrm{i}\mathrm{j}}-{\mathrm{d}}_{\mathrm{i}\mathrm{j}}(\mathbf{X}))}^2$$

where w_ij is a user defined weight which must be nonnegative. The minimization problem of σ²(X) is quite complex and it is necessary to use interactive algorithms to find the matrix X_n×p that minimizes σ²(X). The most used for the solution of this is the SMACOF algorithm.¹²,¹³,¹⁸

Robust confidence ellipse

After applied MDS technique, the dimensionality data was reduced for two or three dimensions. The following step is identifying the outlying laboratories by robust confidence ellipse (Figure 1c).

The robust confidence ellipse is built from the matrix equation:

(3) $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}+\left(\sqrt{2\cdot {\mathrm{F}}_{2;(\mathrm{n}-1);(1-\alpha )}}\right)\cdot \mathrm{U}\cdot \mathrm{Q}$$

where $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}=\left[\begin{array}{ll}{\overline{\mathrm{X}}}_{\mathrm{r}\mathrm{o}\mathrm{b}}& {\overline{\mathrm{y}}}_{\mathrm{r}\mathrm{o}\mathrm{b}}\end{array}\right]$$ is the vector of robust means; F_{2;(n-1);(1-α)} is the quantile of the Fisher-Snedecor distribution with 2 and (n - 1) degrees of freedom and confidence level of (1 - α)%; Q is the Cholesky decomposition of the robust variance-covariance matrix S_rob.¹⁸,¹⁹ Both $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}$$ and S_rob are estimated by the iterative process described in the next section.

The matrix U is the unit circle defined by

(4) $$\mathbf{U}=\left[\begin{array}{ll}\cos (\mathbf{a})& \sin (\mathbf{a})\end{array}\right]$$

where a = [a₁ … a_m] is a vector of size m (0 ≤ a ≤ 2π).

Robust means ($${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}$$) and variance-covariance matrix (S_rob)

The vector of robust means $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}$$ and the variance-covariance matrix S_rob, for bi-dimensional data set, mentioned in the previous sub-section, are fitted from the following iterative process.¹⁹ Let (x; y) a bi-dimensional data set.

(5) $$(\mathbf{x};\mathbf{y})=\left[\begin{array}{cc}{\mathrm{x}}_1& {\mathrm{y}}_1\\ {\mathrm{x}}_2& {\mathrm{y}}_2\\ ⋮& ⋮\\ {\mathrm{x}}_{\mathrm{n}}& {\mathrm{y}}_{\mathrm{n}}\end{array}\right]$$

Step (i): $$\hat{\mu }=\left[\begin{array}{ll}\overline{x}& \overline{y}\end{array}\right]$$ and wi = 1 + p/υ ∀ i = 1, …, n where p is the number of variables and υ is degree of freedom of multivariate t distribution.

Step (ii): compute the matrix:

(6) $$\mathbf{X}=\left[\begin{array}{cc}({\mathrm{x}}_1-\overline{\mathrm{x}})& ({\mathrm{y}}_1-\overline{\mathrm{y}})\\ ⋮& ⋮\\ ({\mathrm{x}}_{\mathrm{n}}-\overline{\mathrm{x}})& ({\mathrm{y}}_{\mathrm{n}}-\overline{\mathrm{y}})\end{array}\right]$$

Step (iii): compute the singular value decomposition of matrix A, svd(A) = USV^T, where:

(7) $$\mathbf{A}=\left[\begin{array}{cc}\sqrt{\frac{{\mathrm{w}}_1}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}}}({\mathrm{x}}_1-\overline{\mathrm{x}})& \sqrt{\frac{{\mathrm{w}}_1}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}}}({\mathrm{y}}_1-\overline{\mathrm{y}})\\ ⋮& ⋮\\ \sqrt{\frac{{\mathrm{w}}_{\mathrm{n}}}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}}}({\mathrm{x}}_{\mathrm{n}}-\overline{\mathrm{x}})& \sqrt{\frac{{\mathrm{w}}_{\mathrm{n}}}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}}}({\mathrm{y}}_{\mathrm{n}}-\overline{\mathrm{y}})\end{array}\right]$$

Step (iv): compute the matrix:

(8) $${\mathbf{w}}^{\mathrm{\prime }}={\mathrm{X}\mathrm{V}\mathrm{S}}^{\mathrm{\prime }}=\left[\begin{array}{cc}{\mathrm{W}}_{11}^{\mathrm{\prime }}& {\mathrm{W}}_{12}^{\mathrm{\prime }}\\ ⋮& ⋮\\ {\mathrm{W}}_{\mathrm{n}1}^{\mathrm{\prime }}& {\mathrm{w}}_{\mathrm{n}2}^{\mathrm{\prime }}\end{array}\right]$$

where,

(9) $${\mathbf{S}}^{\mathrm{\prime }}=\left[\begin{array}{cc}1/s_1& \\ & 1/s_2\end{array}\right]$$

Step (v): compute the vector Q = [Q₁ … Q_n] = [(w’₁₁)² + w’₁₂)² … (w’_n1)² + w’_n2)]

Step (vi): compute the new weights:

(10) $${\mathrm{w}}_{\mathrm{i}}^{\ast }=\frac{(\mathrm{v}+\mathrm{p})}{(\mathrm{v}+{\mathrm{Q}}_{\mathrm{i}})}\mathrm{\forall }\mathrm{i}=1,\dots ,n$$

Step (vii): compute the robust vector of means:

(11) $${\hat{\mathit{\mu }}}_{\text{rob}}=\left[\begin{array}{ll}{\overline{\mathrm{x}}}_{\text{rob}}& {\overline{\mathrm{y}}}_{\text{rob}}\end{array}\right]=\left[\begin{array}{ll}\frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }{\mathrm{x}}_{\mathrm{i}}}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }}& \frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }{\mathrm{y}}_{\mathrm{i}}}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }}\end{array}\right]$$

Step (viii): the new fitted vector $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}$$ and the weights w_i^* are used to obtain new values in step (i) ($$\hat{\mathit{\mu }}={\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}$$ and w_i = w_i^* ). This procedure is repeated until the values of w_i^* converge, that is, |w_i - w_i^*| < ε.

Step (ix): compute the robust variance-covariance matrix S_rob.¹⁹

(12) $${\mathbf{S}}_{\mathrm{r}\mathrm{o}\mathrm{b}}=\left[\begin{array}{cc}\frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }{({\mathrm{x}}_{\mathrm{i}}-{\overline{\mathrm{x}}}_{\mathrm{r}\mathrm{o}\mathrm{b}})}^2}{\mathrm{n}}& \frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }({\mathrm{x}}_{\mathrm{i}}-{\overline{\mathrm{x}}}_{\mathrm{r}\mathrm{o}\mathrm{b}})({\mathrm{y}}_{\mathrm{i}}-{\overline{\mathrm{y}}}_{\mathrm{r}\mathrm{o}\mathrm{b}})}{\mathrm{n}}\\ \frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }({\mathrm{x}}_{\mathrm{i}}-{\overline{\mathrm{x}}}_{\mathrm{r}\mathrm{o}\mathrm{b}})({\mathrm{y}}_{\mathrm{i}}-{\overline{\mathrm{y}}}_{\mathrm{r}\mathrm{o}\mathrm{b}})}{\mathrm{n}}& \frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{i}}^{\ast }{({\mathrm{y}}_{\mathrm{i}}-{\overline{\mathrm{y}}}_{\mathrm{r}\mathrm{o}\mathrm{b}})}^2}{\mathrm{n}}\end{array}\right]$$

Robust confidence ellipsoid

It is recommended to investigate through the robust confidence ellipsoid (Figure 1d) the possible presence of outliers that were not identified in the 2D analysis mentioned in previous sub-section.

The robust confidence ellipsoid is built from an equation analogous to the robust confidence ellipse:

(13) $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}+\left(\sqrt{3\cdot {\mathrm{F}}_{3;(\mathrm{n}-1);(1-\alpha )}}\right)\cdot \mathrm{U}\cdot \mathrm{Q}$$

where $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}=\left[\begin{array}{lll}{\overline{\mathrm{x}}}_{\mathrm{r}\mathrm{o}\mathrm{b}}& {\overline{\mathrm{y}}}_{\mathrm{r}\mathrm{o}\mathrm{b}}& {\overline{\mathrm{z}}}_{\mathrm{r}\mathrm{o}\mathrm{b}}\end{array}\right]$$ is the vector of robust means; F_{3;(n-1);(1-α)} is the quantile of the Fisher-Snedecor distribution with 3 and (n - 1) degrees of freedom and confidence level of (1 - α)%; Q is the Cholesky decomposition of the robust variance-covariance matrix. Furthermore, $${\hat{\mathit{\mu }}}_{\mathrm{r}\mathrm{o}\mathrm{b}}$$ and S_rob are estimated analogous to iterative process described in previous sub-section.

The m × 3 matrix U is the sphere of radius 1 defined by:

(14) $$\mathbf{U}=\left[\begin{array}{lll}\cos (\mathit{\theta })\sin (\mathit{\phi })& \sin (\mathit{\theta })\sin (\mathit{\phi })& \cos (\mathit{\phi })\end{array}\right]$$

where θ = [θ₁ … θ_m] is a vector of size m (0 ≤ θ ≤ 2π) and φ = [ϕ₁ … ϕ_m] is a vector of size m (0 ≤ φ ≤ π).¹⁸

Software

All statistical analyses were performed using the R statistical software, an open-source free environment for statistical computing and graph creation.¹⁸ The 2D plot analysis (Figure 1c) was built using the R packages (CAR and STATS).¹⁸ In addition, an R code was built to define which points obtained in the multidimensional scaling technique were outside the robust confidence ellipse. The MASS R package¹⁸ was necessary for this purpose.

Regarding the 3D analysis (Figure 1d), the plot was built using PLOTLY R package¹⁸ which is a graphing library that makes interactive plots. It is worth pointing out that to build the robust confidence ellipsoid it was necessary to develop an R code based on the dataEllipse R function from CAR R package.¹⁸ Moreover, an R code was built to identify outlying results. This code also depends on the MASS R package.¹⁸

Results and Discussion

The results of the participants (Figure 1a) were arranged in a matrix with 1860 rows and 10 columns (Figure 1b). The columns contain the results of the participants in which each column is the spectrum reported by the i-th participant. The lines represent the variables (chemical shift in ppm).

It should be noted that this is the first study combining multidimensional scaling and robust confidence ellipse/ellipsoid (MDS/RCE) to evaluate laboratory proficiency in interlaboratory comparison. In order to validate the methodology suggested in this article, the MDS/RCE results will be compared with multivariate z-score results obtained by Sheen et al.²

In the proposed methodology, the two-dimensional multidimensional scaling provides a location in Euclidean space of each laboratory participating according to similarity/dissimilarity between ¹H NMR spectra. Additionally, the two-dimensional robust confidence ellipse allows identifying if there is an outlier spectrum (Figure 1c). The Figure 2 presents the results obtained by two-dimensional approach.

Figure 2
Multidimensional scaling from ¹H NMR spectra reported by participating and 95% robust confidence ellipse (2D plot).

It can be seen in Figure 2 that the laboratory 7042 is located out of ellipse. This provides evidence that its result differs statistically from other participants. At 5% significance level (Figure 2) the spectrum reported by laboratory 7042 is classified as an outlier when compared to other participants. On the other hand, laboratories that are located within the robust confidence ellipse do not differ statistically from each other. In this case, there is no evidence that the results reported by these participants are outliers.

The following step is analyzing the three-dimensional limit according to Figure 1d. The 3D plot allows seeing an outlier result that is “hidden” in the 2D plot. In other words, the three-dimensional approach provides more information about participating performances.

It can be seen from Figure 3 that more laboratories show results that differ from the others (at the 5% significance level) when compared to the bi-dimensional plot showed in Figure 2. Altogether there are three laboratories (0714, 7042, and 9541) out of ellipsoid which means that their results may be considered outliers, that is, statistically different from the others. The rest, inside the ellipsoid, do not differ statistically from each other. By an analogous reasoning followed for the univariate z-score suggested by ISO 13528:2015,¹ performance assessment by the MDS/RCE is considered acceptable when results are inside of ellipse/ellipsoid and unacceptable when results are outside.

Figure 3
Multidimensional scaling from ¹H NMR spectra reported by participating and 95% robust confidence ellipsoid (3D plot).

The confidence level adopted to consider the reported result as an outlier was 95% in 2D (Figure 2) and 3D (Figure 3) plot analysis.

In some cases, two (Figure 1c) and three (Figure 1d) dimensional analysis can provide the same conclusions, however, there are situations where this does not occur (such as Figures 2 and 3). In this situation, the two-dimensional analysis differs from the three-dimensional analysis, thus the latter must be adopted because it offers a more comprehensive (and reliable) analysis of the participants’ results.

In the multivariate z-score approach suggested by Sheen et al.,² the interspectral distance matrix D_k was obtained from the Kullback-Leibler, Mahalanobis, Hellinger and Jensen-Shannon distances. The values were fitted to a lognormal distribution. The choice of this probability distribution was based on the Q-Q plot. Additionally, according to the authors, the lognormal is the maximum entropy distribution for a specified mean and standard deviation.²

Each Z_i,k value is an indication of where the s_i,k spectrum is in relation to the others in the S_k cluster. In the case of the lognormal distribution, Z_i,k(1/2) = 1 and Z_i,k(0.95) ca. 5. In this context, Z_i,k = 1 indicates that s_i,k is closer to the center of S_k while Z_i,k greater than 5 indicates that s_i,k is outside the range 95% confidence in the conglomerate S_k.²

The ||T_i,L|| values were fitted to a lognormal distribution and the projected statistical score was calculated for each data set. The scores calculated from the Kullback-Leibler, Mahalanobis, Hellinger, and Jensen-Shannon distances showed evidence that participants 0714, 7042 and 9541 reported results (spectra) are outliers.²

The methodology based on multidimensional scaling combined with the robust confidence ellipse/ellipsoid (Figure 1) identified the same outlying participant results as the multivariate z-score method proposed by Sheen et al.² The main advantage of the methodology proposed in this article is that it does not depend on choosing a probability distribution. This provides evidence that MDS/RCE has potential to be considered as performance evaluation method in interlaboratory comparisons.

Conclusions

In this paper, it was proposed a new approach to analyze multivariate data from interlaboratory comparisons. The methodology combines multidimensional scaling and robust confidence ellipse/ellipsoid to identify an outlying laboratory result. The results obtained by the methodology suggested in this work were compared with the results obtained by the multivariate z-score method described by Sheen et al.² The MDS/RCE method found the same three outlying laboratories (0714, 7042, and 9541) identified in the multivariate z-score method. The proposed methodology has the advantage in relation to the latter of not depending on the choice, sometimes subjective, of a probability distribution.

This approach proved to be promising as a performance evaluation method for multivariate data analysis in interlaboratory comparison. It should be noted that this methodology can be used, in a similar way, to evaluate the performance of laboratories participating in proficiency testing scheme. In this approach, it is suggested classifying the participating laboratory results located outside of robust confidence ellipse/ellipsoid as unacceptable and inside as acceptable like univariate z-score showed in ISO 13528:2015.

The proposed method therefore constitutes a valuable tool that contribute to filling a gap in the literature regarding the multivariate data analysis in interlaboratory comparisons and proficiency trials. Other multivariate data analysis techniques like factor analysis and Kohonen’s self-organizing map, for example, are multivariate techniques that can be considered in future works for performance assessment tool in interlaboratory comparisons.

References

• ¹
  ISO 13528: Statistical Methods for Use in Proficiency Testing by Interlaboratory Comparisons, ISO: Geneva, 2015.
• ²
  Sheen, D. A.; Rocha, W. F. C.; Lippa, K. A.; Bearden, D. W.; Chemom. Intell. Lab. Syst. 2017, 162, 10. [Crossref]
• ³
  Hair Jr., J. F.; Black, W. C.; Babin, B. J.; Anderson, R. E.; Multivariate Data Analysis, 7^th ed.; Prentice Hall: New Jersey, USA, 2009.
• ⁴
  Viant, M. R.; Bearden, D. W.; Bundy, J. G.; Burton, I. W.; Collette, T. W.; Ekman, D. R.; Ezernieks, V.; Karakach, T. K.; Lin, C. Y.; Rochfort, S.; De Ropp, J. S.; Teng, Q.; Tjeerdema, R.S.; Walter, J. A.; Wu, H.; Environ. Sci. Technol. 2009, 43, 219. [Crossref]
• ⁵
  Gallo, V.; Intini, N.; Mastrorilli, P.; Latronico, M.; Scapicchio, P.; Triggiani, M.; Bevilacqua, V.; Fanizzi, P.; Acquotti, D.; Airoldi, C.; Arnesano, F.; Assfalg, M.; Benevelli, F.; Bertelli, D.; Cagliani, L. R.; Casadei, L.; Marincola, F. C.; Colafemmina, G.; Consonni, R.; Cosentino, C.; Davalli, S.; De Pascali, S. A.; D’aiuto, V.; Faccini, A.; Gobetto, R.; Lamanna, R.; Liguori, F.; Longobardi, F.; Mallamace, D.; Mazzei, P.; Menegazzo, I.; Milone, S.; Mucci, A.; Napoli, C.; Pertinhez, T.; Rizzuti, A.; Rocchigiani, L.; Schievano, E.; Sciubba, F.; Sobolev, A.; Tenori, L.; Valerio, M.; Anal. Chem. 2015, 87, 6709. [Crossref]
• ⁶
  Danzer, K.; Wank, U.; Wienke, D.; Chemom. Intell. Lab. Syst. 1991, 12, 69. [Crossref]
• ⁷
  Henrion, R.; Chemom. Intell. Lab. Syst. 1994, 25, 1. [Crossref]
• ⁸
  Škrbić, B.; Héberger, K.; Ðurišić-Mladenović, N.; Anal. Bioanal. Chem. 2013, 405, 8363. [Crossref]
• ⁹
  Minkkinen, P.; Chemom. Intell. Lab. Syst. 1995, 29, 263. [Crossref]
• ¹⁰
  Heininger, P.; Pelzer, J.; Henrion, R.; Henrion, G.; Fresenius’ J. Anal. Chem. 1998, 360, 344. [Crossref`]
• ¹¹
  Aoki, R.; Leão, D.; Bustamante, J. P. M.; Vilca, F.; J. Appl. Stat. 2021, 1. [Crossref]
• ¹²
  Leeuw, J.; Mair, P.; J. Stat. Software 2009, 31, 1. [Crossref]
• ¹³
  Groenen, P. J. F.; van de Velden, M.; J. Stat. Software 2016, 73, 1. [Crossref]
• ¹⁴
  ISO 13528: Statistical Methods for Use in Proficiency Testing by Interlaboratory Comparisons, ISO: Geneva, 2005.
• ¹⁵
  Agrafiotis, D. K.; Rassokhin D. N.; Lobanov, V. S.; J. Comput. Chem. 2001, 22, 488. [Crossref]
• ¹⁶
  Morrison, A.; Ross, G.; Chalmers, M.; Inf. Vis. 2003, 2, 68. [Crossref]
• ¹⁷
  Asan, U.; Ercan, S. In Computational Intelligence Systems in Industrial Engineering; Kahraman, C., ed.; Atlantis Press: Paris, France, 2012, ch. 14. [Link]
• ¹⁸
  Bates, D.; Chambers, J.; Dalgaard, P.; Gentleman, R.; Hornik, K.; Ihaka, R.; Kalibera, T.; Lawrence, M.; Leisch, F.; Ligges, U.; Lumley, T.; Maechler, M.; Meyer, S.; Murrell, P.; Plummer, M.; Ripley, B.; Sarkar, D.; Lang, D. T.; Tierney, L.; Urbanek, S.; R Project, version 4.2.0; R Foundation, Austria, 2022.
• ¹⁹
  Kent, J. T.; Tyler, D. E.; Vard, Y.; Commun. Stat. Simul. Comput. 1994, 23, 441. [Crossref]

Edited by

Editor handled this article: Maria Cristina Canela (Associate)

Publication Dates

• Publication in this collection
  10 Mar 2023
• Date of issue
  Mar 2023

History

• Received
  02 May 2022
• Published
  16 Sept 2022