Markov Chains and Reliability Analysis for Reinforced Concrete Structure Service Life

From field studies and the literature, it was found that the degradation of concrete over time can be modelled probabilistically using homogeneous Markov Chains. To confirm this finding, this study presents an application of Markov Chains associated with the reliability analysis of experimental results of the degradation of concrete by chlorides. Experimental results were obtained for chloride penetration originating from non-accelerated tests in concretes in which the water/binder ratio was variable (0.40, 0.50 and 0.60) and that were produced with Pozzolanic Portland cement that was exposed for six months to the action of NaCl. Using a simulation process, the failure and safety probabilities were calculated by reliability and using Markov Chains, a service life project was estimated (a period of corrosion initiation). Compared to a concrete structure itself, the average error of service life predicted using Markov was approximately 14%. The results show a promissory methodology, in combination with the determination of concrete cover thickness, according to the required service life.


Introduction
The degradation of reinforcement by corrosion is one of the greatest problems found in concrete structures, causing high annual maintenance, repair and rehabilitation costs.Chloride ions play an important role in this process because they are one of the main causes of this problem.The precise determination of the factors that influence this degradation is a complex issue.In the engineering field, many problems are related to natural processes and phenomena that are inherently random, and some variables related to them may not be considered as constant in time.Balaji Rao and Anoop 1 reported that the corrosion state of a reinforcing bar in a specified bridge girder, exposed to given nominal environmental conditions, is a random variable.The corrosion state of the reinforcing bar also varies along the length of the bridge.Thus, the corrosion state of the reinforcing bar in a bridge has to be modeled as a random process.
Therefore, many decisions made during the planning and project phases of engineering processes are invariably accomplished under uncertain conditions 2 .In some cases, the use of stochastic processes, such as the Reliability Theory of Markov Chains, may be advantageous to account for the variability in the main parameters that have a significant influence on the degradation process.
Reliability theory is one of the first stochastic methods to be used for probabilistic predictions of the service life (SL) (Appendix 1) of reinforced concrete structures (RC), whose probability reliability is related to the perfect operation of a certain component during a specific period of time in its normal conditions of use 3 .The reliability may reflect the safety level of the structure, and it is given by the reliability index β 2 , which is associated with a failure probability p f related to the limiting states of the structure.
Other probabilistic approaches are currently employed for studies of engineering systems, such as Fuzzy Logic, Neural Networks and Markov Chains 1,[4][5][6][7] .In this study, due to the short bibliography about the theme, the basic principles of Markov Chains are discussed.Higher considerations about the reliability theory and its applications may be found in the studies of Stewart and Melchers 8 and Ang and Tang 2 .
The Markov Chain is a special case of a stochastic process of discreet parameters whose development may be conducted through a series of transitions between the states of a system.According to Ang and Tang 2 , take a system with m possible states, called 1, 2,..., m, whose state changes may only occur in discreet parameters, for example, at times t 1 , t 2 ,..., t n .Thus, X n+1 represents the state of the system at time t n+1 .The probability of the system is in a future state depending exclusively on the present state of the system, thus creating a Markov Chain, whose conditional probability is given by Equation 1.
X n+1 is the random variable that corresponds to the state assumed by the system at time t n+1 , X 0 = x 0 ,..., X n = x n *e-mail: epossan@gmail.comrepresent all of the former states of the system, and P(X n+1 =I | X n =x n ) represents the probability that the system is in the state i, after the occurrence of n+1 given intervals that were in the current state of the system x n at the time n.This type of process is also known as memoryless process because the past is "forgotten"; only the present time is actually taken into consideration.
For a Markov Chain of discreet parameters, the probability that the system takes the state j at the time t n once it was in the state i at the given time t m is called the transition probability (p ij ), and it is represented by Equation 2.
Because m represents the number of possible states for the system over time, the transition probabilities between the states of the system may be represented by a matrix (m x m) called the transition probability matrix (P) (Equation 3).
Because the system states are mutually exclusive and collectively exhaustive after each transition, the values that compose the matrix must be non-negative and lie between 0 and 1, and the addition of the entrance of each line must equal 1.
The probabilities of the initial state of a system (P(0)) may be represented by a line matrix (Equation 4), also called the initial condition vector in which p i (0) is the probability of the system in the initial state i.
After a transition, the probability of the system in state j is given by Equation 5, which is represented in matrix denomination by Equation 6.
, (1)  (0) (1) (0) Therefore, the probability of the system being in state j after n transitions is given by Equation 7.
( 2) (0) n P n P n P P n For engineering applications, the Markov Chains models are based on cumulative probabilities of the degradation of a determined system or component of the system.These probabilities, in general, are obtained by means of visual inspections, degradation models or experts' knowledge and reliability theory 9,10 .Attention must be given to the limitations related to the determination of the initial probabilities of the degradation system, which are usually made by visual inspections of a structure and taken as constant for the system 11 .
From information derived from experts, Balaji Rao et al. 6 used Markov Chains to study the corrosion degradation of elements of concrete structures.In the methodology used by the researchers, the experts evaluated the damage state of the structure, which is expressed in terms of probability of the structure to achieve a given level of degradation.An example problem of a reinforced concrete bridge girder is presented, which illustrates the usefulness of the proposed methodology in facilitating decision making regarding repair.
The project Lifecon 5 employed Markov Chains for SL studies, emphasizing the benefits of this application when combined with degradation models.Some authors 7,9 also used Markov Chains for durability studies and SL prediction in RC, proving them advantageous for engineering studies.Markov Chains may be used at any time of an SL of a structure, considering the possible actions of maintenance and repair and the costs inherent to them.Moreover, one of the most important properties of this method is that it provides a system that is mathematically linear, enabling linear optimization.
Before these facts, and as a result of field observation, for the initiation period of corrosion due to degradation by chlorides, reliability analysis and Markov Chains are jointly employed for the purposes of the SL prediction of RC following the Brazilian normative specifications.

Methodology
The decision to employ Markov Chains to predict the SL of RC, together with the reliability theory, aims to consider the uncertainties of the degradation process until the structures reach the durability limit state (DLS).This state precedes the occurrence of the Serviceability Limit State (SLS) and the Ultimate Limit State (ULS).In this study, for the period of corrosion initiation, the DLS is given by the critical concentration of chlorides (C cr ) on the steel bar level (if C cr ≥ 0.40% in cement mass, the durability limit is surpassed).It can be said that the period of corrosion initiation corresponds to the time (t) that the chloride ions take to reach the steel bars.
Through natural aging tests (Figure 1), data about chloride penetration into concrete over time were obtained.The solution of Fick's Second Law was used for the adjustment of the penetration profiles to determine the values of the diffusion coefficient (D) and the superficial concentration (C s ).
The cover thicknesses related to the environmental classification adopted by national Brazilian standards 12 were used.For each variable of study, the uncertainties of the degradation process, as shown in Table 1, were included.The coefficient of variation (CV) and the probability density function (p.d.f.) were compiled from the literature 13 .
Monte Carlo Simulations employing the Fick's error function solution were used as the DLS function.From this simulation, probability values distributed in two regions were obtained: one that characterizes the failure, where the durability limit is surpassed (C cr ≥ 0.40%), and the other one is the safety (C cr ≤ 0.40%).
Considering a system with two possible states depending on the C cr value, the initial probabilities were determined by reliability analysis.The experimental data of a six-month exposure to chlorides were considered in Markov as p ij in the P, also called the probability matrix of system degradation.
The p ij values of the system were estimated using Markov Chains of discreet processes, by means of simulation, thus determining the DLS for the studied concretes.

Experimental procedure
The data used in this paper were derived from a study conducted by Rizzon 15 , who investigated the natural penetration of chlorides into concrete for different water/ cement (w/c) ratios (0.40; 0.50 and 0.60), for two, four and six months, produced with a Brazilian Portland Pozzolanic cement (similar at ASTM Type IP cement) called CP IV.In this study, only the data from six months of tests for all levels of the w/c ratio were used.
For the production of concretes, a cement with γ = 2.82 g/cm 3 , fine aggregate from quartz origin (γ = 2.62 g/cm 3 , φ máx = 4.8 mm and fineness modulus of 2.54) and coarse aggregate from basaltic origin (γ = 2.65 g/cm 3 , φ máx = 19 mm) was used.For the natural penetration of chlorides, concrete beams were manufactured (one for each w/c ratio), whose dimensions were 100×200×500 mm.After seven days of humid curing and 21 days in a laboratory environment, the beams were submerged in a NaCl solution with a concentration of 3.5% (1 Mol) until the correct age for the test.A cylindrical concrete specimen for compression strength testing at 28 days was also manufactured.
In the natural test of chloride penetration, the flux of solution was directed to the sides of larger dimensions (200×500 mm), according to Figure 2a.
The other sides of the beams (100×500 and 100×200 mm) were waterproofed with epoxy resin (Figure 2b).The chloride amount was determined by titulometry based on powdery samples collected at depths of 5, 10, 15, 20 and 25 mm in relation to the surface (Figure 2c).With these data, it was possible to make penetration profiles of the chloride.Through adjustments using the least square method (Equation 8), the values of C s and D for the analyzed concretes were calculated.

Application of reliability analysis
In this study, concrete was considered a homogenous and isotropic material.The only transportation mechanism is the diffusion, and it is a one-dimensional analysis.The possible local variations in the surfaces of the beams were not considered, assuming that penetration is uniform and that the concrete structure was well executed.For simulation purposes, it was assumed that two structures with coating thicknesses (x) corresponding to 30 mm and 20 mm would be constructed in the same degradation environment (sea environment of moderate aggression).It is assumed that a structure standing between 100 and 750 meters away from the sea is found in an environment of moderate aggressiveness.Despite the fact that these values are not absolute because they depend on the wind, temperature, humidity and other factors, they may be taken as a reference.
A C cr value equal to 0.40% in cement mass was adopted as necessary for the depassivation of the steel bars.It was also assumed that the values of CV for x, D, C s and C cr would be the ones presented in Table 1.The lowest variation coefficients were adopted because the degradation data came from a non-accelerated experiment, whose control level is elevated.
For this study, it was assumed that the DLS would be reached when the C s close to the bars at a determined time t − represented by C(x,t) − was equal to the critical concentration necessary for depassivation (C cr ).Considering such assertions, the function of the limit state established through Fick's Second Law can be represented by Equation 8. Table 1.Variability adopted in the reliability analysis (adapted from Meira et al. 14 ).Finally, the Monte Carlo method was employed with 5000 simulations for each analyzed case.As a result, the failure probabilities (p f ) and safety (p s ) for each level of w/c ratio due to coating thicknesses specified by the normative law were determined.

Markov Chains application
The application of Markov Chains for the purposes of SL prediction first demands that the states (m) that the system can assume are defined.In this work, two states are assumed (Table 2): the "zero" state represents the concrete in initial conditions of use (DLS is not surpassed), and state number "one" represents the concrete in failure conditions (DLS is surpassed).
After defining the states of the system, it is necessary to identify all of the possible transitions between the states.Thus, the probability of transition of step (p ij ) may represent the probability that each group remains in the current state (represented by the probability that characterizes the safetyp s -determined by reliability) or passing to a future state (represented by the probability that characterizes the failurep f -determined by reliability) in the variation period of ½ years.The probabilities of the states of the system may be represented by the transition matrix It was assumed that it is impossible for a structure to return to a former state without maintenance or repair intervention.Thus, the states that precede the current state all correspond to the "zero" state.Considering that, the values of the diagonals of the matrix of transition present only the probability of the matrix to "remain at a state of the system" and the probability to "pass to the following step".Another consideration is the definition of the vector of the initial conditions for the system, called P(0).For a new structure, there is P(0)= [1, 0] because it is assumed that the structure is free from damage and in safety conditions.

Results and Discussions
Figure 4 show the chloride profiles obtained from the adjustment of the data.The values corresponding to the D and the C s , adjusted using Fick's Second Law, are shown in Table 3.
D is very different over time (see Table 3), especially for concretes whose w/c ratio are 0.50 and 0.60.In two months of immersion, D is more elevated but is reduced at 4 and 6 months.This behavior is due mainly to the reduction of the porosity of concrete over time, which is a function of the continuous hydration processes of Portland cement and the action of pozzolans that are present in the composition of the cement 16 .
Cs increases over time and with the elevation of the w/c ratio.The higher Cs is, the higher D will be 17 .According to Swamy, Hamada and Laiw 18 , the effect of Cs on the adjacent surfaces of concrete has an influence on the D value.
Considering the variations of x of 30 mm and 20 mm and the data for six months of chloride exposition, the

States of the system
State Classification 0 C cr < 0.40% -durability limit is not surpassed -safety conditions 1 C cr ≥ 0.40% -durability limit is surpassed -failure conditions simulations of the degradation process were accomplished employing the solution of Fick's Second Law as a function of the DLS.The answers were the probabilities of the system to be found in failure or safety conditions (see Table 4).The elevated results of p f in concretes whose w/c ratio equals 0.50 and 0.60 were already expected because the C close to the steel bars is already higher than C cr (Figure 4).For the concrete with the lowest w/c ratio (0.40), in the penetration profiles, the C cr was not reached at 2 and 6 months of immersion, which is also evidenced in the p f (Table 4) for the two analyzed x (in the time of 6 months).Regarding the importance of the w/c ratio on the SL of the structures, it is possible to verify that its increment brought about a higher chloride penetration, an increase in the D and Cs and, consequently, a higher p f , considerably reducing the SL.Tuutti 19 and Mangat, Khatib and Molloy 20 verified that D is strongly influenced by the w/c ratio; thus, the more elevated the w/c is, the easier the transportation of chlorides into the concrete becomes.
The p f and p s were defined as being the p ij in the Markov.Therefore, the P of the degradation process is defined according to Figures 5a and 5b.In this way, the third matrix of Figure 5a represents P for the concrete degradation when produced with a w/c ratio of 0.60 for a minimum coating thickness of 30 mm.The probabilities 0.8438 and 0.1562 represent, respectively, the p ij of the system, in this case, the concrete element that is being studied, remaining in its current state (p 00 ) and its probability of changing states (leaving current state p 00 and passing to the following state p 01 ).The other matrixes present the same characteristics.Comparing these data with the probabilities shown in the third matrix of Figure 5b, whose x was reduced to 20 mm, it is observed that the probability of the system changing states (p 01 =0.8064) is higher than the probability of its remaining (p 00 =0.1936) at the current state of the system.
According to the formulation shown in Equation 7, the vector of the initial conditions P(0) = [ 1 , 0 ] with each P shown in Figure 5 is multiplied.In this way, it was possible to determine the transition matrixes of the system (P(n)) and their probabilities (p (n) ij ) in time intervals of 0.50 years, according to the example below: It is also shown that the average degradation level of the structure (DL) is determined by multiplying the scale vector of the states of the system (S = [0, 1]) by the distributions of the degradation conditions of each state.The DL for each time interval is obtained by adding the results of these multiplications, with the precision of a decimal fraction.
For the analysis accomplished via Markov, it is assumed that the project SL is reached at the time in which the degradation level is equal to "one" (C cr ≥ 0.40%).In Tables 5 and 6, this interval is highlighted in boldface, and the x of 30 mm and 20 mm are equivalent.
The influence of the w/c ratio and the x in the SL can be observed in the simulation of the degradation process using Markov (Tables 5 and 6).A higher concrete cover (30 mm) and lower w/c ratio (0.40) lead to a longer SL (≥ 50 years).If the x is reduced by 10 mm, the SL estimated using Markov is approximately 5 years.For 30 mm of x, if the w/c ratio is changed to 0.50, the project SL is approximately 26 years.Such data demonstrate the importance of definitions and specifications of a project, showing that slight variations in the x and w/c ratio can reduce or considerably increase the required SL.In this way, considering the x of 30 and 20 mm, respectively, Figures 6 and 7 were designed to show the evolution of the SL along the time through the average values of the DL.
Regardless of the adopted x, it is possible to observe in Figures 6 and 7 that the degradation level increases with the elevation of the w/c ratio.For the aggressive level that is being analyzed, concrete with a w/c ratio of 0.60 presents an SL of approximately 9.5 years, for an x of 30 mm.If this x is reduced to 20 mm, the SL is reduced as well, to approximately 1 year.For concrete with a w/c ratio of 0.40, the estimated SL is more than 50 years for an x of 30 mm and approximately 5 years for a thickness of 20 mm.
As shown in Figure 8, considering a w/c ratio of 0.50, the necessary time for this concrete to reach the degradation  state "one" is approximately 27 and 2.5 years for the x of 30 and 20 mm, respectively.As reported in the literature 14,21 , these results also evidence the importance of the thickness in the SL of RC, showing that a variation of 10 mm in this protective layer may considerably reduce the SL.For a coating of 30 mm, it is observed via Markov analysis that a reduction of the w/c ratio of the concrete from 0.50 to 0.40 increases the SL 100%.Considering the same w/c ratio (0.40), the estimated SL for a structure with an x of 30 mm is ten times as high as a structure with 20 mm of coating.Project definitions (concrete cover specifications due to the environmental aggression) and the execution (process control to reassure the project definitions) may be evaluated in a quantitative way, inserting the costs (acquisition and maintenance) of each alternative -through the analysis of the costs of the life cycle -having previously determined the most viable alternative following the criteria of economic viability versus SL.
Aiming to evaluate the estimation of SL obtained using Markov and to calculate the associated error, it is necessary to compare it with a real structure.Thus, the data of a case study by Guimarães 22 were used.The author investigated a structure exposed to a sea environment of degradation, aged 22 years after construction at the time of the investigations, located in the harbor of Rio Grande, RS, in the South region of Brazil (Figure 9).
The profile of the penetration of chlorides of the investigated structural element (a beam exposed to the spray zone, made with Pozzolanic Portland cement (ASTM Type IP) with a w/c of 0.44 and compressive strength of 23 MPa) is shown in Figure 10.From the adjustment of this profile through the Fick's error function, it was possible to obtain C s = 3.45% and D = 0.06 cm²/year.For the 22 years of exposure of the structural element to sea breeze, considering the C cr of 0.40%, the measurement in situ of the depth of penetration of chlorides obtained by Guimarães 22 was 31 mm.With this value as the x of the concrete, it was possible to accomplish the simulation of SL prediction and compare with the estimation using Markov.Because the w/c ratio of the structural element is different from the ones employed in this research (0.40, 0.50 and 0.60), it was accomplished using a regression model with the data of Rizzon 15 aimed at obtaining the data (D and C s ) concerning the w/c ratio of the real structure (0.44).The SL estimated using Markov was approximately 19 years and 22 years for the real structure, corresponding to an error of 14%.
It is verified in the current literature 7,22 that the variability associated with the estimation of the SL, both by reliability and Markov, is between 17 and 27%.Morcous 7 used historical data of degradation obtained from visual inspection of structures and observed an error of approximately 24% in the estimation of SL using Markov.Applying the reliability theory, Guimarães 22 found errors of 17 and 26% in the estimation of the depth of penetration of chlorides through the concrete for a failure probability of 0.50.

Conclusions
Considering the probabilistic estimation of SL through the conjoint application of reliability analysis and Markov Chains, it was observed quantitatively that the increase in x is favorable to durability and consequently to the SL of RC.By making use of probabilistic simulation, it was possible to verify that for a concrete with a w/c ratio 0.50, an increase of 10 mm in the x leads to an increase in the SL of approximately 25 years.Considering the w/c ratio (0.40), the estimated SL for a structure with a coating of 30 mm is nearly 10 times as high as a structure with 20 mm of coating.
Compared to a real structure with 22 years of age at the time of the investigation, the average error of the calculated SL estimation in this study using Markov Chains associated with the reliability analysis was 14%, demonstrating the advantages of the presented technique.
The Markov method is an auspicious mathematical tool for modeling and optimizing systems, as well as for SL predictions of RC.Markov methods are characterized by the simplicity of use, and in addition to demanding low computational costs, they do not require advanced software to be analyzed.Software such as Excel® may be employed successfully.
One of the main advantages of employing Markov Chains to predict the SL of RC is the possibility of conducting the treatment of the effects and causes of degradation in a totally probabilistic way, enabling the description of parallel processes depending on time (for example, the corrosion process of steel bars versus systems of protection of the structure by coating).Moreover, the Markov Chains may be employed at any time of a structure's SL and for several degradation processes (corrosion, expansion, cracking).These chains also enable technical and economic evaluation of the possible actions of maintenance and repair inherent to RC.Additionally, one of the most important properties of this chain is that it produces a mathematically linear system, which allows linear optimization.
As for the application of Markov Chains for SL prediction of RC, it is observed that the results are very sensitive to p ij , which makes the system strongly dependent on the entrance variables.Incorrect or misleading   determinations may lead to non-representative results of the studied system, demanding special attention when obtaining these probabilities.The adoption of adequate degradation models, the association of probabilistic techniques such as reliability theory, and the creation of data banks of degradation of RC may aid the determination of these probabilities of transition and the reduction of the uncertainties of the process.

Figure 1 .
Figure 1.Simulation process of degradation of concrete due to chloride degradation.D=Chloride diffusion coefficient.Cs = Chloride superficial concentration.
Coefficient of Variation; II f.d.p: function of probability density.
diagram of Figure 3.

Figure 2 .
Figure 2. (a) Sketch of solution flowing through the concrete specimen (b) specimen in immersion (c) collecting depths in mm.

Figure 4 .
Figure 4. Chloride penetration profile for a saturated concrete.

Figure 9 .
Figure 9. Location of the structure in study.

Table 2 .
States of the system.

Table 3 .
C s and D of chlorides in the concrete.

Table 4 .
p f and p s in function of x, after six months of exposition.

Table 6 .
Evolution of degradation along the time for x =20 mm.
*DL = degradation level of the structure.