Statistics of the Pareto front in Multi-objective Optimization under Uncertainties

In this paper we address an innovative approach to determine the mean and a confidence interval for a set of objects analogous to curves and surfaces. The approach is based on the determination of the most representative member of the family by minimizing a Hausdorff distance. This method is applied to the analysis of uncertain Pareto frontiers in multi-objective optimization MOO . The determination of the Pareto front of deterministic MOO is carried by minimizing the hypervolume contained between the front and the utopia point. We give some examples and we apply the approach to a truss-like structure for which conflicting objective functions such as the structure mass and the maximum displacement are both to be minimized.


INTRODUCTION
In real-life situations, it is frequent to consider contradictory objectives to be satisfied simultaneously in order to furnish an acceptable solution belonging to a set of possible choices.For instance, in Engineering, it is usual to look for solutions that maximize the performance while minimizing the cost -what is generally contradictory.A consumer chooses the best bundle of goods that he can afford but looks for the minimal expenses, while a producer maximizes his income and minimizes both production time and total cost Varian 2006, Varian 2009and Mankiw 2011 .Traders seek for investments making the expected portfolio returns as high as possible with the lowest risk possible Hurson and Zopounidis 1997, Zopounidis 1999, Pätäri et al. 2018and Craven and Islam 2005 .In such a situation, compromises must be determined between the objectives -it is usual to look for the Pareto frontier associated to the multi-objective problem, which synthetizes the possible compromises and trade-offs between the objectives.
Multi-objective optimization MOO is deeply applied to furnish more realistic solutions to improve economic activity or industrial process Amodeo et al. 2007 and Ivanov and Ray 2014 .In addition, real problems are also characterized by uncertainty: in practice, parameters defining objectives and constraints may be subjected to variability or simply badly known.Thus, considering uncertainty becomes essential and we may find in the literature many works devoted to uncertainties in multi-objective optimization.For instance, in the field of economics, stochastic dominance has been introduced Hadar and Russell 1969, Bawa 1975, Bawa and Goroff 1983 and is widely exploited in Economics, Finance and Social Sciences see, for instance, a few among many works: Ji and Lejeune 2018, Light 2018, Yager 2018 .Another approach often found in the literature concerns the determination of robust solutions, id est, solutions remaining stable for a given range or known scenarios of perturbation see, for instance, a few among many works: Navabi and Mirzaei 2017, Bachur et al. 2017, Xidonas et al. 2017, Moreira et al. 2016 .
The efforts to consider uncertainty in optimization have a long history Sahinidis 2004 , but it is rare to find works concerning statistics of the Pareto frontier, such as its mean, its variance or the determination of a confidence interval.Indeed, under uncertainty, Pareto frontier becomes uncertain and, when the uncertainties are modeled as random variables, Pareto frontier becomes stochastic, so that we may look for its mean, variance and a confidence interval.Although natural, such an analysis appears as difficulty, since a Pareto frontier is an object belonging to an infinite dimensional vector space: for instance, when considering a bi-objective problem, the Pareto frontier is, in general, a curve in ℝ , which must be described by a vector map or an algebraic equation, id est, a vector function associating an interval I ⊂ ℝ to a set of points in ℝ I ∋  →   ∈ ℝ on an algebraic equation   0 ,  ∈ S .A first approximation may consider the Pareto frontier as a cloud of points, but even in this case, difficulties arise, since each point is a variate from a distribution dependent on a parameter  ∈ 0,1 and the value of  associated to each point is unknown -thus, the evaluation of statistics of the points request a previous procedure for the indexation of the points by , what remains arbitrary.In this paper, we address this difficulty by an alternative approach, by considering that the median object is the most representative one in the family.This approach allows considering random objects that can be modeled by continuous geometric forms instead of a cloud of a data points.The approach is applied to Pareto frontiers, which are determined by the variational approach introduced in Zidani et al. 2013, Souza de Cursi 2015 to solve deterministic MOO problems and that leads to the determination of Pareto frontier by minimizing a hypervolume, but other methods of determination may be used instead.
In section 2, we illustrate the difficulty about the determination of statistics of families of curves and the proposed approach.The rest of the paper is organized as follows.In section 3, the mathematical model of deterministic multi-objective optimization problems is presented, and then uncertainties are introduced in section 4 for the MOO problems with constraints.In section 5 we explain the process we follow to quantify the uncertainties, and how the link is established between Statistics and Geometry.Three academic problems are solved in section 6 in both deterministic and uncertain cases before to study a 5-bar truss structure problem in section 7.In these one two exogenous variables are made random.At last, a summary concludes this paper in section 8.

STATISTICS OF CURVES
In this section we illustrate the difficulties in the determination of statistics of families of curves and the proposed approach.Analogous difficulties arise in higher dimensional situations.
As previously observed, a curve in the plane is a set of points which may be described by an algebraic equation   0,  ∈  or a map :  → ℝ , where  ,  ⊂ ℝ.We are interested in the situation where the curve depends upon a random variable  ∈ ℝ : the equation becomes  |  0 ,  ∈   and the map reads as  | :   → ℝ  .For instance, let us consider the family defined by:  |   0,   : | |  1 We have: | ,  ,  ∈ ,  . 2 Assume that  ∈ 0,1 is uniformly distributed.The mean value of the parameter is   1/2 and, for a given , the mean value of  | is   | , 1 || /2 Notice that  ∈ ||, 1 .As shown in Figure 1, the means evaluated by this way do not correspond to the family: in fact, they generate a curve similar to the envelope of the family.a A family of curves b The mean curve determined by examining the points Figure 1: An example where the mean generated by the points is not a member of the family so that the mean is a member of the family.Nevertheless, the approach introduced by Croquet and Souza de Cursi 2010 requests that the family is composed of parameterized curves.If the parameterization is missing, this method cannot be applied -it is necessary to determine a parameterization previously.In this work, we examine an alternative approach which may be applied without parameterization: we look for one of the elements of the family having a median position, id est, for a member of the family which occupies a central position and may be considered as a good representative of the family.In such a case, we look for the element of the family which is the nearest one for all the others.This is performed by minimizing the distance between nonparameterized curveswe use the Hausdorff distance HD defined in equation 17 .Figure 2 shows the obtained result when this method is applied to the previous example.Once the median curve is determined, we may look for a "confidence interval" by finding a region including the mean curve and containing a given percentage of the family: usually, confidence intervals use a parameter  -the risk -and a confidence level  .For instance, we may look for a "confidence interval" having a level  % thus,   % by finding a region containing the 90% members of the family which are closer to the median curve, as shown in Figure 2.More generally, the difficulty exposed concerns the determination of the mean or the median of families of curves.Let us illustrate the situation by considering different families of random curves.

A family of random circles
Let us consider a family of circles having a random radius and a random initial phase:    * cos      * sin   5 with  ∈ 0,2 ,  is uniformly distributed on 1,3 and  is uniformly distributed on 0,2 .If the pointwise mean is considered, we have       0, so that the mean is a point: the origin 0,0 .However, the expected mean is a circle of radius  2. In practical situations, we have a finite sample of the family: for instance, let us consider a sample of  circles from this family, shown in Figure 3.We represent the empirical pointwise mean of the sample in black and the median curve in red, for samples of  100 at left and  1000 at right .Green circles correspond to the confidence interval with a risk  10%.Blue circles lay outside the confidence intervalas expected these are the outermost ones, symmetrically distributed on the interior and exterior boundaries of the family -the blue circles represent 10% of the sample.Analogously, the pointwise mean furnishes a small circle near the origin, with a radius that goes to zero when the size of the sample increases.where  ∈ 0 , /4 ,  is uniformly distributed on 0,2 ,  and  are uniformly distributed on 1,1 .For independent variables,       0. As previously done with circles, we consider samples of  100 at left in Figure 4 and  1000 at right in Figure 4 .Green arcs lay in the confidence interval with a risk  10%.Blue segments are outside this confidence interval -as expected these are the outermost ones.As in the preceding example, the blue curves represent 10% of the sample and the pointwise mean furnishes a small curve near the origin, that goes to zero when the size of the sample increases.exhibit the statistical characteristics of interest from this family.In this case again, the mean curve is not a trajectory of the system, while the median is a member of the family.The results are displayed in Figure 5 considering the confidence interval at the same level used before.It is interesting to notice that, in this case, the confidence interval may be considered as unilateral.
Figure 5: The results for the Van der Pol oscillator with random initial position: pointwise mean black curve is not a trajectory, while the median red curve is a feasible trajectory.Green curves correspond to the confidence interval.

A Duffing oscillator with random parameters
A last example is given by the Duffing oscillator:    ,  0  ;  0 1 .8 We consider  is uniformly distributed on 0.5, 1.5 , and  uniformly distributed on 0.1, 0.2 .The results are exhibited in Figure 6.We observe that the pointwise mean does not correspond to a trajectory in the phase space, while the median is a feasible trajectory.
Figure 6: Results for the Duffing oscillator with random parameters: pointwise mean black curve is not a trajectory, while the median red curve is a feasible trajectory.Green curves correspond to the confidence interval.
As established in the preceding examples, this approach is effective to furnish the median and, then, to generate a confidence interval of a family of curves, so that we may consider its application to the determination of the mean and a confidence interval of Pareto's fronts.In the sequel, this method is used in order to quantify uncertainties in MOO problems.Initially section 6 , it is applied to three classical test problems and then section 7 , to the analysis of a 5 bar truss structure MOO problem.

DETERMINISTIC MULTIOBJECTIVE OPTIMIZATION
As usual, a standard MOO problem is modeled with a system of equalities and inequalities equations as follows: Numerous methods can be used to solve deterministic MOO problems, such as the weighting method Gass andSaaty 1955 andZadeh 1963 , the å-constraint method Haimes et al. 1971, Geoffrion-Dyer-Feinberg Method Geoffrion et al. 1972 , the Keeney-Raiffa method Keeney and Raiffa 1994 and more see for instance: Miettinen 1999 and Collette and Siarry 2002 and in this work we use a variational method called here the Zidani-Souza's method that has been proposed by Zidani et al. 2013 and which is presented in the appendix A. It consists in minimizing the hypervolume between the Pareto front and the utopia point.In this method, the decision variables are developed in polynomials, what allows to get a piecewise continuous Pareto front.

MULTIOBJECTIVE OPTIMIZATION UNDER UNCERTAINTIES
To take the uncertainties into account, we introduce the random vector  of exogenous variables, in replacement of the deterministic vector .As a consequence, both objective functions and constraints become random.In this case, equality and inequality constraints become probabilistic and the standard MOO problem becomes: As previously mentioned, the main goal of this work is to focus on uncertainties in MOO; more precisely, a link is established between Geometry and Statistics when the randomness of an exogenous variable lead to a set of Pareto fronts instead of one front.In fact, our objective is to evaluate statistical quantities, namely confidence intervals of a set from the single data furnished by the set itself, without the use of an external probability -they are generated by using "means" or "medians" .Then, considering randomness on a MOO problem inputs, we explore the outputs, namely the trade-offs, and extract from the family its "median" and other "quantile curves", more generally the hypersurfaces that belong also to the same set.In other words, we search for a mean of a set of subjects that is one of its members.

IMPLEMENTATION
In practice, the approach under consideration is implemented as follows: • A sample  1 ,  2 , … ,   of the random vector  is generated • Each problem is solved by using the Zidani-Souza's method and  Pareto fronts denoted   * are obtained.• We compute the sum of the distances between each Pareto front and all the other ones:
In this work, we consider two distances: 1) The L 2 distance given by: 2) The Hausdorff distance given by:   * ,  * max sup where  * and  * are closed bounded non-empty subsets of the metric space ℝ ,  .
In a first step, we considered both the distances: tests running with each one provided results that were compared and showed to be almost identical.Then, in a second step, we considered the single Hausdorff distance and a larger sample  200 to determine the quantities of interest.

TEST FUNCTIONS
In this section, three academic test functions are considered.All are solved as deterministic problems before uncertainties are considered.All the problems are bi-objective, involving constraints, as previously mentioned.

Binh and Korn function
The original problem is reported in Binh and Korn 1997     Since there are no uncertainties in constraints, they must be satisfied at a 100% level of probability for the problem solution.Figure 8 shows the 200 Pareto fronts obtained, with the mean front in red, the 180 nearest fronts to the mean in green and the 20 farthest ones -in the sense of Hausdorff's distance -in blue.Thus, the green curves correspond to the 90% closer to the median in the sense of Hausdorff's distance and may be considered as belonging to a confidence interval with risk į 10%.We observe that, as expected, the median is a central curve and the curves laying outside the confidence interval are the outermost ones.

Fonseca and Fleming function
The Fonseca and Fleming problem Fonseca and Fleming 1995 is: Here yet, polynomials of the 6 th .degree with coefficients on c , c 10,10 are considered.The numerical results are the following, and the Pareto front is shown in Figure 9: Three uncorrelated random variables  ,  and  that are all uniformly distributed on 0,0.1 are added to this system in order to make it uncertain, then  200 deterministic problems derived from the initial one are obtained as follows: Figure 10 shows the 200 curves we get and the mean that minimizes both of the distances that we used is in the middle of the curves set as expected.In this case    is a concave function, furthermore, all of the Pareto fronts ends are located together is a small region of space.In this case, the Hausdorff distance exhibits a larger variation in the middle of the family of curves, so that the curves beyond the 90%-quantile in blue appear more clearly when compared to the preceding situation.Notice that the median appears as a mean curve, in the center of the family.In this case, the Pareto's front is discontinuous.This fact makes that Hausdorff's distances between curves that seem close to the eye are, in fact, large in the sense of Hausdorff's distance.This fact is due to the fact that some parts of a curve may be isolated from the other curve, so that the distance of these points to the other one is large.For instance, let us consider the fronts shown in Fig. 13: the blue front may appear to the eye as being closer to the red one than the green front, but the respective Hausdorff's distances are 0.4132 and 0.1975, so that the green front is closer to the red one in the Hausdorff's sense.Observe that the red front has points which are far from the blue one.
Figure 13: The blue front may appear as closer to the median in red than the green one, but its Hausdorff's distance is the greatest one.
An alternative consists in evaluating the distances only on the first 5 parts of the front thus, in neglecting the last one .In this case, we obtain the result shown in Figure 14: in this case, the results better fit the eye's expectations, but the confidence interval appears as unilateral.

APPLICATION ON A 5-BAR TRUSS STRUCTURE
In this section we study the five-bar truss structure sketched in Figure 15, where we minimize its total mass denoted , simultaneously with its maximum displacement denoted 𝑢 Ellaia et al. 2013 .Then we introduce uncertainties on some parameters to see how the solution set behaves when the system values change.
It is assumed that the structure will be modeled by linear, two nodes, bar elements in linear elasticity, subjected only to axial forces and free from imperfections  With a 6 degree polynomial  and  10, we get the Pareto front in Figure 16.In the next step we consider: • the force  that becomes uncertain (denoted ) following a normal distribution with 10% for the coefficient of variation.
• the Young modulus  that becomes uncertain too (denoted ) following a truncated normal distribution defined on 60.68 , 77.22 GPa with 3% for the coefficient of variation.
As done with the test functions, ns 200 problems are generated with a MCS and the result we obtained is shown in Figure 17: In Figure 17, results are "as expected": the mean is in the middle of the curves set, while curves beyond the 90%-quantile in blue are located at the exterior of the curves set.

Summary
In this work, the MOO problems with constraints and uncertainties are addressed.Instead of analyzing a cloud of data points, the adopted point of view consists of analyzing the randomness of objects that can be modeled by continuous geometric forms, thanks to the Zidani and Souza de Cursi's method which leads to a piecewise continuous Pareto front for the MOO problems, and curves distances measures.Hence, by using a Monte Carlo simulation, a sample of Pareto fronts is generated and the Hausdorff's distance leads to a Pareto front quantile analysis, from the link that we made between Statistics and Geometry.Three academic problems are first modified to handle uncertainties and then solved.Next, an application to a 5-bar truss structure with two exogenous random variables is considered to demonstrate the applicability of the proposed method with a more difficult problem.All results obtained appear satisfactory when observing the location of the median curve and quantiles.
A possible perspective of this work is to apply the approach presented in this paper to a MOO problem under uncertainties where objective functions and constraints are expended with Generalized Fourier Series Bassi et al. 2016 .The use of approximated functions instead of the initial ones aims to reduce the algorithms running time.
Example 6.3 shows that Hausdorff's distance may lead to results that may be considered as unexpected from the eye's point of view.We may find in the literature modifications of Hausdorff's distance see, for instance, Dubuisson and Jain, 1994 -different distances may be used with the procedure exposed in this work.The comparison between the existing distances and the definition of criteria for the selection of the adequate one will be matter of further work.

Figure 2 :
Figure 2: The median curve red and the curves defining the "confidence interval" cyan are members of the family

18 Figure 3 :
Figure 3: The pointwise mean circle in black in near the origin, the median generated by examining the Hausdorff distances in red is central.green circles correspond to the confidence interval

Figure 4 :
Figure 4: The pointwise mean arc of circle in black reduces to a point, the median one generated by examining the Hausdorff distances in red , and the confidence interval green arcs of circle . ⟩ being the scalar product, rot  * .the rotational of  * and  is the area or hypervolume that was minimized and that is bounded by the utopia point and the Pareto front given by  *  * ,  * .The corresponding Pareto front is presented in Figure7.

Figure 7 :
Figure 7: Pareto front of the Binh and Korn test function

Figure 8 :
Figure 8: Binh and Korn function under uncertainties for ns 200 sample size: the median Pareto front appears in red, the confidence interval in green and the Pareto fronts beyond the 90%-quantile in blue

Figure 9 :
Figure 9: Pareto front of the Fonseca and Fleming test function

Figure 10 :
Figure 10: Fonseca and Fleming under uncertainties for  200: the median Pareto front appears in red, the confidence interval in green and the Pareto fronts beyond the 90%-quantile in blue

6. 3
Zitzler-Deb-Thiele's function 3 ZDT3 Let us consider the function  of   ,  , … ,  defined such as: ZDT3 Problem, which is reported in Zitzler et al. 2000 reads as follows: the case  2. The Pareto front of this problem is given in Figure 11.

Figure 11 :
Figure 11: Pareto front of the ZDT3 test function Figure 14: ZDT3 median Pareto front red and confidence interval green when the last arc is ignored.
. The geometric and material parameters used are length  9.3144 m, area  0.01419352 m , load  448.2 kN, Young ′ s modulus  68.95 GPa, density  2,768 kg m ⁄ and yield stress  172.4 MPa.

Figure 15 :
Figure 15: 5 bar truss structure schemaDenoting  ∈ ℝ the vector of the topological and sizing optimization parameters, such that 0  1 for  ∈ 1, 2, … ,  where  5 is the number of elements, the problem to solve is:

Figure 16 :
Figure 16: Pareto front of the 5 bar truss structure

Figure 17 :
Figure 17: Pareto fronts of the 5 bar truss structure with uncertainties: the median Pareto front appears in red, the confidence interval in green and the Pareto fronts beyond the 90%-quantile in blue Statistics of the Pareto front in Multi-objective Optimization under Uncertainties Statistics of the Pareto front in Multi-objective Optimization under Uncertainties  1 ,  2 , … ,   are the objective functions •  1 ,  2 … ,   are the inequality constraints • ℎ 1 , ℎ 2 … , ℎ  are the equality constraints •   ,  , … ,  ∈ ℝ is the decision variables vector •   (resp.  ) is the lower boundary (resp.upper boundary) •   ,  , … ,  ∈ ℝ represents the values of the exogenous parameters •   is the feasible space including all the constraints and boundaries above • Prob is the probability operator,  has a given joint probability distribution, and  and  are the reliability probabilities that are imposed by the decision maker.Then, each realization   , ,  , , … ,  , of  leads to a unique deterministic MOO problem that generates a piecewise continuous Pareto front, and a sample  ,  , … ,  of  leads to  distinct MOO problems and a set of  different curves or hypersurfaces .

•
By using a Monte Carlo simulation, we get for each m-tuple    ,1 ,  ,2 , … ,  , t such as 1   a new MOO problem.In this work we focus on the bi-objective optimization problems with inequality constraints, then, theoretically, each value of the sample, leads to a new problem defined as follows: and is as follows:Statistics of the Pareto front in Multi-objective Optimization under Uncertainties By applying Zidani-Souza's method of the appendix A, this problem is efficiently solved by expanding  and  as polynomials of degree 6, having their coefficients in Latin American Journal of Solids and Structures, 2018, 15 11 , e130 8/18 *  ∇ *  d