Meta-analysis of the experimental coefficient of variation in wheat using the Bayesian and Frequentist approaches

ABSTRACT A large set of variables is assessed for progeny selection in a plant-breeding program and other agronomic fields. The meta-analysis of the coefficient of variation (CVe) produces information for researchers and breeders on the experimental quality of trials. This analysis can also be applied in the decision-making process of the experimental plan regarding the experimental design, the number of repetitions, and the treatments and plants/progenies to be measured. In this study, we evaluated the dataset distribution and the descriptive statistics of CVe through the Frequentist and Bayesian approaches, aiming to establish the credibility and confidence intervals. We submitted CVe data of ten wheat (Triticum aestivum L.) traits reported in 1,068 articles published to the Bayesian and Frequentist analyses. Sample data were analyzed via Gamma and normal models. We selected the model with the lowest Akaike Information Criterion (AIC) value, and then we tested three link functions. In the Bayesian analysis, uniform distributions were used as non-informative priors for the Gamma distribution parameters with three ranges of q~U (a,b,). Thus, the prior probability density function was given by: p θ = 1 β - α , θ ∈ α , β . The Bayesian and Frequentist approaches with the Gamma model presented similar results for CVe; however, the range Bayesian credible intervals was narrower than the Frequentist confidence intervals. Gamma distribution fitted the CVe data better than the normal distribution. The credible and confidence intervals of CVe were successfully applied to wheat traits and could be used as experimental accuracy measurements in other experiments.


Introduction
In many countries, governmental authorities require the evaluation of a new cultivar in officially registered trials before its release.The estimated coefficient of variation (CVe) is usually essential for this process and has been used as a parameter of experimental quality (Piepho and Möhring, 2006).The CVe must present adequate levels depending on the species and traits available (Resende and Duarte, 2007;Arnhold and Milani, 2011).This measurement considers the experimental estimate error ( σe 2 is the relation mean overall of the experiment and can be easily obtained as Classifications for the CVe magnitude have been proposed for several crops (Albert and Zhang, 2010;Fritsche-Neto et al., 2012;Couto et al., 2013;Aerts et al., 2015).The Frequentist approach has been used to estimate the CVe (Fritsche-Neto et., 2012;Mora and Arriagada, 2016;Nardino et al., 2020); nevertheless, few proposals take into account the CVe distribution for a trait that follows a non-normal distribution, that is, robust methods are necessary, since the distribution is unknown in many cases.
The Bayesian inference can be very useful to evaluate the CVe classification, since it allows estimating parameters and relating measurements of association in non-normally distributed data or where asymptotic assumptions are not appropriate, due to sparse data or small sample sizes.However, no studies were found in the literature for wheat crop (Triticum aestivum L.) that use the Bayesian approach for the CVe classification.The advantages of the Bayesian approach are mainly related to the independence of normally distributed data, considering that space parametric CVe (%) is > 0 and the normal distribution is -∞ + ∞.Moreover, this method offers flexibility to choose the dataset distribution and to incorporate prior knowledge on model parameters (Silva et al., 2013).
In the literature, no studies on cross-information with the Frequentist and Bayesian approaches were observed on dataset distributions or descriptive statistics of the CVe in wheat.Therefore, we searched in the leading Brazilian journals for the CVe values in wheat traits via the meta-analysis.The variables studied are relevant for breeding programs to select and estimate genetic gain, as well as in the plant science field for studies on cultivar characterization.Research on experimental quality in these and other fields via different statistical approaches is scientifically relevant.Here, we evaluated the distribution of the CVe and descriptive statistics through the Frequentist and Bayesian approaches to establish credibility and confidence intervals for ten wheat traits.

Data source
We researched 1,068 articles on wheat published between 1970 and 2020 in all editions of the most renowned Brazilian scientific journals (Table 1).These data strongly support statistical tests to establish criteria of CVe classification for the most evaluated traits in wheat.We collected experimental CVe values from all journals listed in Table 1 and accessed all articles on the journal's online page.The following search terms were used: Triticum aestivum L., wheat, trigo, coefficient of variation, CV %, Triticum.

SEM
. Estimated values are presented in Table 1.

Database reviews
The initial analysis for data inspection revealed that 90.7 % was obtained from experiments arranged in a randomized complete block design (RCBD), 6.17 % was obtained from completely randomized design (CRD), 2.53 % was obtained from an experiment conducted in lattice design (DLAT), and 0.6 %, in the design of tracks.The data were tabulated in an MS Excel spreadsheet containing the categorical traits of the journal, publication year, number of treatments, number of replications, and the experimental CVe values of traits, prior to preparation and organization for the statistical analyses.

Model selection
The goodness-of-fit of the models to the data was tested by the Akaike information criterion (AIC), as follows: where: p is the number of parameters, and LogL is the logarithm of the maximum value of the likelihood function.The best model has the smallest AIC or less information loss (Casella and Berger, 2002;Cavanaugh and Neath, 2019).We also tested three link functions in the Gamma model that used identity, log and inverse, and AIC to select the best function link.
Initially, the CVe data of each variable were used to obtain data distributions and projections of the normal and Gamma distribution.Then, a generalized linear model (GLM) was fitted (intercept only), assuming a Gamma distribution.We tested three link functions, using the GLM function of the R software system, as follows: We computed the lower confidence interval (LCi) and estimated mean and upper confidence interval (UCi).
The statistical analysis was carried out in the R software system (R Core Team, version 4.0.2),using the metan (Olivoto and Lúcio, 2020) and ggplot2 packages (Wickham, 2016).The scripts that were used to carry out the analysis are given in Appendix I.

Bayesian approach
Uniform distributions were used as non-informative priors for the Gamma distribution parameters, where three ranges were evaluated: r ~ U(0 -5) and mu ~ U(0 -5), r ~ U(0 -10) and mu ~ U(0 -10) and r ~ U(0 -20) and mu ~ U(0 -20) with q~U (a, b).The values of deviance information criterion (DIC) were computed.Thus, the prior probability density function was given by: p( ) . The uniform distributions have been used in the Bayesian analysis for both conceptual and practical reasons (Gelman, 2006).When assuming the Gamma distribution for the data and the uniform distribution for Gamma parameters, the posterior density function was given as p(q|y)~Gamma(q|a,b), with the following density function: , where the mean (expectated) is equal to E( ) θ α β = (Gelman et al., 2004).Iterations of 10,000, with burn-in and thin given, respectively, by 1,000 and 10 iterations were used.
The Highest Posterior Density interval (HPD) was used to obtain LCi and UCi with probability = 0.95.For the Bayesian analyses, we used boa package (Bayesian Output Analysis) (Smith, 2007).The OpenBUGS and package R R2OpenBUGS (Sturtz et al., 2005) were also used.The scripts for the Bayesian analyses are reported in Appendix I.

Convergence and model fit
We used 10,000 iterations for burning and we realized a cut of the first 1,000 iterations.The results for all 9,000 iterations of the parameters of deviance, r, and mu of Gamma distribution are demonstrated in the supplementary material (Appendix II A and II B), considering all the wheat traits evaluated in Table 2.
We used three models with different ranges for the parameters r and mu of uniform distribution in the script in Table 2, with a range of 0 -5 in DIC_1, 0 -10 in DIC_2, and 0 -20 in DIC_3.Thus, for each model, we generated the DIC of the traits.The results of the DIC analysis for three models, considering ten traits, revealed that model 1, with 0 -5 range of uniform distribution, has lower DIC, indicating the model with the best fit from which we obtained the descriptive statistics and credible intervals.
We obtained the results of the posterior mean of three models and the credibility intervals (LCI and UCI, 95 %) (Figure 1).The smallest range between the credibility intervals was observed for Model 1 (r ~ U(0 -5) and mu ~ U(0 -5)).For the posterior mean, the most significant difference between the three models was found in the trait grain yield plant (GYP), 0.07 %.
The Akaike Information Criterion (AIC) for the Gamma model of the Frequentist approach of the ten traits is shown in Table 2.The AIC for GY: 6,233.3,DF: 408, GYP: 433.4,HGW: 355, TGW: 731.1, HW: 636.7,PH: 1,090.1,LS: 249.4,NSPS: 356.4,and NGS: 666.4,respectively.In the Gamma model, three link functions were tested using the AIC criterion to select the model best link function.No differences were observed between the link functions used, namely identity, Log, and inverse.Thus, we selected the identity link function due to the easiness to interpret the parameters.
We used a normal model and a Gamma model in data of ten traits and obtained the AIC (Table 2).All AIC values were higher in the normal model, where the AIC for GY was: 6,462.8,DF: 448.6, GYP: 448.6, HGW: 377.84,TGW: 871.7,HW: 894.6, PH: 1398.6,LS: 259.3, NSPS: 375.4,and NGS: 671.9.The Gamma model presented lower AIC values; thus, it adjusted better to the CVe values than the normal model.

Comparison Bayesian/Frequentist approaches
The results of posterior Bayesian distribution for the ten wheat traits evaluated regarding CVe are presented in Figure 2A.The posterior distribution range is short.GYP showed a greater CVe range, between 10 % and 17.5 %, and a density above 0.4.The traits with shorter CVe range in the posterior distribution were DF, GY, and HW with values 3 -4.5 %, 12 -13.5 %, and 2 -3.3 %, besides all traits with a density equal to or above 2. Figure 2B shows that the grouping of the traits in the same figure demonstrates that each variable has a shape of specific distribution or variability.
The data results -salmon color, Gammablue and normal line -dotted line distribution -are presented in Figure 3 for the ten wheat traits.The traits revealed that the CVe data evaluated were better fitted with the Gamma distribution in relation to the normal distribution.We highlight the fit for the traits DF, TGW, HW and GY of the Gamma distribution also the different forms of the trait distribution for CVe.
Table 3 shows the results for the descriptive statistics of CVe of the ten traits evaluated in wheat by the Bayesian approach.Initially, we considered describing the results for descriptive statistics: quantile 2.5, 1 st quartile, posterior mean, 3 rd quartile and quantile 97.5.
Therefore, the credible interval LCi and UCi) was used to refer to the Bayesian approach, while the confidence interval (CI) was used to refer to the Frequentist approach (Figure 4).In both approaches DF and HW had the lowest CVe values, with a range between 3 and 4 % for DF and 2 and 3 % for HW.GY and GYP showed the highest CVe values, ranging between 12 and 13 % for GY, and 12 and 16 % for GYP; in both approaches, GY and GYP require attention in the experimental planning.
The traits TGW, PH, NSPS, LS, HGW, and NGS revealed similar magnitudes for CVe, with values between 5 and 11 % for both approaches, considering the credible and confidence interval of 95 %.The variable GYP presented the highest values for the interquartile range of credible and confidence interval of CVe.For the ten traits evaluated, the Bayesian approach showed a shorter range of credible intervals for posterior mean in relation to the confidence interval for the mean estimate, considering the model Gamma for the Frequentist and Gamma model with prior noinformative for the Bayesian approach.

Discussion
The DIC values for the Bayesian approach and the AIC for the Gamma model and the CVe of the ten traits in wheat as well as the values of the normal distribution are presented in Table 2. Based on the three DIC values, we selected the DIC 1 model with the uniform distribution parameters 0-5, due to the smaller deviance values.We observed differences between DIC 1 and AIC of the Gamma model, whose deviance values for the Bayesian model were lower than three units in eight of the ten traits evaluated.
On the other hand, the AIC values of the normal model were surprisingly higher than the AIC values of the Gamma model for all wheat variables evaluated.These results directly impact on the CVe classification methods that use properties of the normal distribution.For the HW, PH, and TGW traits, for example, the difference between the AIC values was above 100 units.The AIC is a ubiquitous tool in statistical modeling and is an estimate for the out-of-sample error based on information theory.The AIC estimates the relative amount of information lost by a model, that is, the less information a model loses, the higher the quality of that model and the lower the AIC score.The criteria for model selection provide a valuable tool to identify a model of appropriate structure and dimension among candidates and are used to compare models based on different probability distributions for the outcome variable.A selection criterion assesses whether a fitted model offers an optimal balance between the goodnessof-fit and the parsimony (Cavanaugh and Neath, 2019).
Posterior and Frequentist distributions of the CVe of the evaluated traits presented some differences, mainly for the descriptive statistics.The CVe distribution for different traits is an interesting and partially conclusive aspect, since the absence of normal distribution of CVe of traits in wheat can be visually represented, regardless of the sample size.The literature presents many methods and studies on CVe with different species and most presuppose that the data is normally distributed.This is not always true, since the Gamma distribution demonstrated a more similar fit for the CVe data distribution for the ten traits evaluated here.
The distributions of CVe data presented in Figures 2 and 3 indicate a wide variability of CVe in wheat crops.The Bayesian and Frequentist approaches demonstrated contrasting distributions, while the estimate and posterior means as well as the credible and confidence intervals were similar.This significant variation is observed to justify the classification coefficient variation for these traits individually (Costa et al., 2002, Nardino et al., 2020).
Non-normally distributed CVe is commonly observed, but this information is frequently neglected.The Bayesian approach has some advantages, such as flexibility in selecting the distributions for sample data and unknown parameters as well as the possibility of incorporating the prior knowledge about the parameters of the model (Sorensen and Gianola, 2002;Silva et al., 2013).
The values of the a posteriori mean and the mean of the Gamma model were similar.Associated to the average of the Bayesian and Frequentist models, we added the a posteriori standard deviation statistic and the standard error estimate.The values for these  statistics generated are lower in the Bayesian model in seven of the ten traits studied.However, the mean magnitude is highly contrasting among the variables, with the lowest CVe mean for HW = 2.6 % and the highest CVe mean for GYP = 13.6 %.This demonstrates that the CVe magnitude is directly associated to the trait nature and its distribution.Some variables, such as GYP, NGS, and GY presented high CVe magnitudes.Therefore, we should increase the number of repetitions and/or plants per plot to reduce the magnitude of the experimental error.
In terms of the credible interval, the Bayesian approach revealed a shorter range than the frequentist.Credible intervals or credible regions are built to qualify in terms of final precision, that is, for validation of the data observed rather than repetitions or hypothetical results (Resende et al., 2014).In this respect, significant criticism has been made about the Frequentist confidence intervals (Murteira, 1995) since the experiments are not likely to be thoroughly repeated.
We opted to use non-informative prior due to the different approaches observed in the literature on the CVe distribution.We also identified that CVe distribution was highly dependent on the species studied and the variable measured, hindering the establishment of general intervals for CVe classification.This has been reported in the literature.Nevertheless, few studies have reported on the distribution of the CVe by comparing the confidence and credible intervals between the Bayesian and Frequentist approaches.The Bayesian approach provided shorter credible intervals than the Frequentist approach for most wheat traits studied.We obtained the standard deviations using Bayesian inference on the CVe study with the posterior and exact credible intervals for the parameters of each variable obtained through the meta-analysis.
The CVe of the ten variables studied is frequently used in wheat research for cultivar phenotyping, phenotypic diversity, the selection of progenies and families in breeding programs, and the final screening for the release of new wheat cultivars.These traits are associated to grain yield (TGW, NGS, HGW, GYP and GY), plant morphology (PH, NSPS, LS and DF) and, indirectly, industrial quality (HW).In this sense, the results of this study also have applicability in different agronomic areas, including plant breeding.This study demonstrates the experimental quality of the trials based on the CVe magnitude.Besides, the results assist in the decision-making process for an experimental plan, such as the experimental design, number of repetitions, and the treatments of the plants/progenies to be measured.

Figure 3 -
Figure 3 -Frequentist distribution of experimental CVe for ten wheat traits.The data distribution is in salmon color, the Gamma distribution is in blue line, and the normal distribution is in the dotted line, where: days for the flowering (DF), grain yield (GY), grain yield per plant (GYP), hundred-grain weight (HGW), hectoliter weight (HW), spike length (LS), number of grains per spike (NGS), number of spikelets per spike (NSPS), plant height (PH), and thousand grain weight (TGW).

Table 1 -
Database of the experimental coefficient of variation (CVe) used in the Bayesian and Frequentist analyses.

Table 2 -
Results of deviance information criterion (DIC) analysis with three ranges for uniform distributions in the r and mu prior parameters DIC 1: 0 -5, DIC 2: 0 -10 and DIC 3: 0 -20 and Akaike Information Criterion (AIC) for the Gamma model fit and for the normal model fit.

Table 3 -
Results of descriptive statistics of analysis CVe data published for ten traits of wheat.q25 = first quartile; LCi = lower credible interval; UCi = upper credible interval; Mean: posteriori mean ± standard deviation a posteriori, q75 = third quartile, and mean ± standard error.