Development and validation of a standard area diagram set to evaluate bacterial blight on yellow passion fruit leaves

Bacterial blight is one of the most important diseases of yellow passion fruit. The aim of this study was to develop and validate a set of standard area diagram (SADs) to improve the accuracy and precision of bacterial blight severity assessments on yellow passion fruit leaves. The proposed SADs, composed of illustrations of symptomatic leaves with nine severity levels (1, 2.5, 5, 10, 20, 30, 40, 50 and 62%), were evaluated by 15 raters. Accuracy, precision and reliability of the SADs were validated by quantifying and comparing the assessments performed by 15 raters with and without the use of SADs. The SADs improved accuracy (coefficient of bias, C b = 0.940 without SADs and Monzani, R.M.; Silva, G.A.; Nutter Junior, F.; Duarte, H.S.S. De Mio, L.L.M. Development and validation of a standard area set diagrams to evaluate bacterial blight on yellow passion fruit leaves. Summa Phytopathologica, v.44, n.4, p.332-337, 2018. 0.967 with SADs), precision (correlation coefficient, r = 0.901 without SADs and 0.949 with SADs) and overall agreement (Lin’s concordance correlation coefficient, ρ c = 0.863 without SADs and 0.935 with SADs) of severity estimates. Inter-rater reliability also significantly improved when the SADs were used (coefficient of determination, R = 0.717 without and R = 0.880 with the SADs; intra-class correlation, ρ = 0.798 without and ρ = 0.926 with the SADs). The developed SADs improved accuracy, precision and reliability of disease severity assessments and are recommended as an aid to assess bacterial blight on yellow passion fruit leaves.

Yellow passion fruit (Passiflora edulis Sims) is the most widely cultivated and consumed passion fruit species produced in Brazil.This country has been the worlds largest producer for more than two decades, accounting for approximately 80% of the global production (18).However, a gradual decrease in production area and yield has been observed since 2010.In the 2015 season, there was a reduction of 33% in production area, 45% in yield, and 17% compared to the 2010 season (14).
Although the crop has potential to reach 50 t ha -1 (18), phytosanitary issues related to the yellow passion fruit constitute one of the greatest limitations to increase yields and production area.Bacterial blight, caused by the bacterium Xanthomonas axonopodis pv.passiflorae (Pereira) Gonçalves & Rosato, causes chlorosis, necrosis and plant death (13).Diseased leaves exhibit translucid and anasarcous small spots, which become necrotic and dark brown with chlorotic halos encircling spots.As the disease develops, the entire leaf is affected, causing wilt with subsequent defoliation, thereby reducing productivity (27).
the vascular system and spread to all plant tissues, leading to plant wilt and even plant death in susceptible cultivars.The disease can cause drastic reductions in the lifespan of affected plants (27).
Disease assessment is an essential tool to conduct appropriate research, study epidemiology, quantify crop loss, screen for resistance and effective bactericides, evaluate the efficacy of treatments in field experiments and conduct plant disease survey (15); therefore, a standardized method to accurately assess disease severity is crucial.Severity is a visual estimation of the percentage of the diseased sampling unit area in relation to the total sampling unit area and it is a subjective assessment, influenced by rater bias (23).In this pathosystem, severity is estimated as the percentage of diseased leaf tissue in relation to the total leaf area.Ideally, these estimates should be accurate (estimated values close to the true values) (21), precise (lack of variation in disease estimates when evaluating the same sampling units) (17) and reliable (consistent disease estimates between or among raters) (21).The product of precision and accuracy provides a measure of agreement, which is a variable often used to compare the estimated values with the true values (17).Standard area diagrams (SADs) have been used to improve accuracy, precision and reliability of disease severity assessments by providing a standard reference of illustrated images of diseased plants, leaves or other plant parts, with distinct severity levels that can be used in comparison to actual diseased leaves (or their images) (5).
A SADs for assessment of bacterial blight on passion fruit leaves was developed and validated by Miranda (19) during her PhD thesis, using mono-lobed leaves.However, the leaves of yellow passion fruit are mono-lobed, especially at the seedling stage, and are three-lobed during most of the crop cycle (28).The SADs proposed by Miranda (19) can be recommended and useful for assessing seedlings but not for epidemiological research in the field.Therefore, it is important to develop and validate SADs to quantify the severity of bacterial blight in three-lobed leaves.Similarly, a SADs for assessment of bacterial blight was developed and validated by Castillo (6) in three-lobed leaves, published in a non-indexed journal in Colombia, but it presented only a set of six diagrams and started with 5% severity, failing to contemplate minor severity found in the field, considering that in this disease there is a tendency to overestimate the evaluation of the disease.
The aim of this study was to develop and validate a SADs that includes the full range of bacterial blight severities encountered in the field in order to assist raters in more accurately estimating disease severity on yellow passion fruit leaves.

Development of SADs
Three hundred yellow passion fruit leaves that exhibited a wide range of bacterial blight severities were arbitrarily collected from commercial orchards in the municipality of Araquari, Brazil.Infected leaves were obtained between January 2013 and July 2015.Leaves were individually scanned using an HP scanner (Scanjet 3800 model) with an image resolution of 300 dpi.The images were processed, and each one was rated using the image analysis software QUANT v.1.0.1 (26) to determine the percentage of diseased leaf area in relation to the total leaf area, i.e. the actual disease severity.These images were used as a reference for evaluating the accuracy, precision and reliability of rater estimates with and without the aid of SADs.Based on the image analysis, a set of nine images spanning the minimum and maximum disease severity was used to develop a nearly linear SADs.Using a digitalized yellow passion fruit leaf shape as template, a SADs with nine distinct disease severities was created by using the image editing software PAINT.NET (http://www.getpaint.net).The software designed lesions with shapes and distribution of actual diseased leaves.

Validation of SADs
For assessment purposes, a total of 50 images of yellow passion fruit leaves with distinct levels of disease severities were projected, one at a time, onto a projector screen using a MS Power Point file.A group of 15 raters with no experience in quantifying disease severity were asked to estimate the disease severity of each of the 50 diseased leaf images set without and with the use of SADs.Firstly, the raters evaluated the leaf images without the SADs.Then, after a 15-min break, a new randomized sequence of images was displayed, and the raters estimated severity using the SADs as an assessment aid.The raters were instructed to compare each leaf image with the SADs to estimate the percentage of diseased area considering both symptoms of chlorosis and necrosis, which were included in the estimates of disease severity.

Data analyses
Accuracy and precision of the estimates by each rater, with and without the use of SADs, were determined based on Lin's concordance correlation coefficient (LCCC, ρc), as described by Nita et al. (2003) and Bock et al. (2010).The absolute errors (estimated severity minus actual severity for each leaf × rater) were determined to show the bias of data points from the actual value.
Inter-rater reliability of the estimates was assessed based on the coefficient of determination (R²) from linear regression analysis of relationships between severity estimates for all pairs of raters ( 21) and on intracluster (or intra-class) correlation (ρ) for all raters combined, as described by Nita et al. (20).
An equivalence test was used to verify the difference between means of the treatments (with SADs and without SADs) for all variables of the LCCC analysis (r, Cb, υ, u and ρc) and for the inter-rater reliability (R²).The equivalence test assessed whether there was a significant difference in means and calculated the 95% confidence intervals (CIs) for each statistic (the difference between means) by bootstrapping using the percentile method (in an equivalence test, the null hypothesis (H 0 ) is nonequivalence, the opposite of H 0 ) (1, 29, 30).All analyses were based on 10000 balanced bootstrap samples, and the 95% CIs were calculated based on the difference between the means of groups; if the CIs included zero, there was no significant difference (α = 0.05).
All statistical analyses were calculated in R. The epi.ccc function of the epiR package (25) was used to obtain LCCC statistics.The built-in boot.sampleR function was used for the equivalence test.The ρ was calculated with the icc function of the irr R package (11).

RESULTS
The SADs of nine images of yellow passion disease severity on fruit leaves ranged from 1 to 62%.Leaves exhibited characteristic lesions of bacterial blight: translucid and anasarcous small spots, which become necrotic and dark brown with chlorotic halos encircling spots (13,27).
All assessments performed with and without the use of SADs resulted in linear relationships between estimated and actual disease severity of bacterial blight.Lin's concordance correlation analysis demonstrated that the raters' assessments of disease severity were more similar to the actual severity when the SADs were used, compared to the assessments performed without using the aid.The equivalence test showed that the statistics of most parameters (ρc, C b and r) were significantly improved when the SADs were employed, indicating that accuracy and precision of the estimated values were significantly improved.Although there was a slight numerical decrease in the scale bias (v) and location bias (u) (v = 0.988 and 1.052, and u = 0.217 and 0.241, with and without the SADs, respectively), it was not significant.
Accuracy (C b ), precision (r) and agreement (ρ ) of disease severity assessments were significantly improved when SADs were used.Without the SADs, accuracy (C b ) was 0.940, the value improved to 0.967 when assessments included the SADs as an aid.Precision (r) was 0.901 in the absence of SADs, but when the SADs was employed, precision was 0.949.Agreement (ρ c ), which combines measures of accuracy and precision, was 0.863 when the SADs was not used, and was increased to 0.935 when SADs were used as an assessment aid.There was a reduction in absolute error, with most estimates falling within ± 20% of the actual value compared to +49/-33 when estimates were made without SADs.There was a tendency to overestimate disease severity with the use of the SADs, as 75% raters showed positive u values (data not shown).This trend was further proved since most estimates of the majority of raters had positive absolute errors.

Coefficient of determination (R2) Frequency (count of inter-rater comparisons)
comparisons had an R 2 < 0.80.The inter-rater reliability of the estimates was used as another indicator of the importance of the use of SADs.
The equivalence test revealed that the statistical parameters, based on the coefficient of determination (R 2 ) and intracluster correlation coefficient (ρ), were significantly improved when raters used SADs (R 2 = 0.880 and 0.717, and ρ = 0.926 and 0.798 , with and without the SADs, respectively).

Discussion
The main objective of this study was to develop and validate a standard area diagram set to assess bacterial blight severity on yellow passion fruit leaves.The diagrams reflected the range of severity found in the field and demonstrated an improvement in accuracy, precision and reliability of estimates.
Visual ratings are the most often used methods to estimate disease severity.Other technologies have been proposed, for example, image analysis and hyperspectral imaging, which can make great contributions to assessments, but visual quantification will continue to be the main approach for assessing plant disease (3).The use of SADs as an aid to visually estimate disease severity can considerably enhance the quality of assessment data, which is helpful to choose the most appropriate disease control (24).
Disease quantification is an important procedure in epidemiological study and management of bacterial blight on yellow passion fruit.The SADs proposed in this report demonstrated to be an effective tool to estimate disease severity, as they improved accuracy, precision and reliability of the data, reducing the chances of drawing incorrect conclusions, i.e. failing to reject the null hypothesis when the null hypothesis was false (type II error) (3,22).Standard area diagram sets have recently been proved to be useful for several diseases of important crops, such as coffee leaf rust (5), potato early blight (10), wheat spot blotch (9), loquat scab (12) and soybean frogeye leaf spot (8).
The nine diagrams used in this SADs similar to the number used in SADs of other pathosystems, which is of practical use and sufficient to guide estimates of disease severity (29).An increase in the number of diagrams has not resulted in more accurate or reliable disease estimates, as demonstrated by Bock et al. (2), when three-, five-, seven-or 10-diagram SADs were compared.The maximum disease severity and the disease severity range likely have a greater effect on accuracy and reliability.It is understood that the use of a high number of diagrams can be time-consuming and therefore can affect the quality of assessments, especially under field conditions.Severity evaluations using an interval scale instead of the nearest percentage might be negatively affected by a low number of diagrams (less than six) (2, 29).
Similar to what was found in other studies, the nine diagrams, representing the range 1 -62% disease severity, were linearly arranged, to ensure more precise and accurate assessments (7, 8).Some SADs have used a logarithmic arrangement based on the Weber-Fechner law, which assumes a logarithmic relationship between estimated and actual disease.But this methodology has been questioned (3, 23) because the Weber-Fechner law has been found to be partially invalid (16,17) and may lead to false conclusions (3, 23).
Accuracy, precision and reliability of the SADs to estimate bacterial blight severity were determined using an equivalence test (based on 95% CIs by bootstrapping of the difference between means), which is an appropriate statistical method to be used for agreement studies (30).In the present study, the equivalence test demonstrated that most of the statistics variables (ρ c , C b and r) were improved when raters used the diagrams to estimate severity, compared to when SADs were not used.Agreement and reliability values were also improved in other SADs that were statistically analyzed by the equivalence test (2, 7, 8, 9, 10).
In most studies of SADs, disease severity has been overestimated by raters (5, 29).However, some reports show the severity being underestimated by raters (7,8,12).The tendency to over or underestimate disease severity might be affected by lesion size, shape and number, as much as disease distribution might be (3).In the present study, the location bias (u) values were greater than zero for most of the raters' estimates using the SADs, indicating a tendency for overestimation.
The use of the SADs to assess bacterial blight severity contributed to a reduction in the absolute errors.In order to reduce the absolute errors and standardize results, computerized disease assessment training programs and human guidance (1), the SADs used in this report can be used to train evaluators to assess disease severity.Speed and standardization that result from the use of SADs can counterbalance the presence of some level of absolute error in the estimates (2).
Inter-reliability of estimates between raters was used to indicate the effectiveness of the SADs.Our findings show that the inter-rater reliability was higher with the use of the diagram to estimate severity, compared to when the SADs were not used, which is congruent with previous reports (12,29).Reliability between evaluators was measured by the linear relationship between disease severity estimation of the same leaf image assessed by different individuals.Preferably, the coefficient of determination (R 2 ) value should approach 1.0, indicating that the estimates made by different raters are similar or identical when R 2 is 1.0.For the evaluations made without using the SADs, the R 2 values were lower than 0.80 for 66.67% comparisons between raters.Using the SADs, R 2 values were higher than 0.80 for 97.14%, indicating high reliability of estimates when the SADs is used.
Liner regression remains a useful analysis for evaluating how well estimates for disease severity provided by one rater agree with the severity assessments of one or more other raters, when the same set of disease leaf samples (experimental units) are assessed (22).The R 2 generated by this analysis provides a quantitative measure of the amount of variation in one rater's severity assessments, which can be explained by another rater's severity estimates (i.e., inter-rater agreement).
The SADs developed and validated in this study will be a valuable tool for raters as an aid to estimate disease severity in epidemiological investigations, comparative studies on disease control methods, pathotype characterization, germplasm screening, and other applications that require accurate, precise and reliable assessment of bacterial blight severity on yellow passion fruit leaves in the field.The proposed SADs proved to be useful for obtaining more accurate, precise and reliable estimates of bacterial blight on yellow passion fruit leaves and consequently minimizing errors in results.

Figure 1 .
Figure 1.Standard area diagram set for bacterial blight (Xanthomonas axonopodis pv.passiflorae) severity on yellow passion fruit leaves (Passiflora edulis).The numbers represent percent (%) leaf area showing symptoms (necrosis and chlorosis) of the disease.

Figure 2 .
Figure 2. Relationship between actual and estimated severity (A, B) and the absolute error (estimated minus actual severity) (C, D) of assessments of a set of 50 images of bacterial blight (Xanthomonas axonopodis pv.passiflorae) of yellow passion fruit leaves (Passiflora edulis) without (A, C) and with (B, D) the use of a set of standard area diagram (SADs), 15 raters.The solid line (A, B) represents the best-fitting line, and the dotted line is the concordance line, which represents perfect agreement between actual and estimated severity (slope of 1, intercept of 0).Low absolute errors indicate that the estimated severity was similar to the actual severity.

1
Standard deviation in parentheses.2Mean of the difference between each rating.Standard errors are in parentheses (bootstrap calculated values).3Confidence intervals (CIs) based on 10.000 bootstrap samples.If the CIs embrace zero, the difference is not significant (α=0.05).Significant differences in bold.4Scale bias, or slope shift (υ, 1 = no bias relative to the concordance line). 5Location bias, or height shift (u, 0 = no bias relative to the concordance line). 6Coefficient of bias (C b ) is a measure of accuracy.It measures how far the best-fit line deviates from the concordance line. 7Correlation coefficient (r) is a measure of precision. 8Lin's concordance correlation coefficient (ρ c ) combines precision (r) and accuracy (C b ) to measure agreement with the true value.

Figure 3 .
Figure 3. Frequency of inter-rater reliability using the coefficient of determination (R 2 ) of visual assessments of bacterial blight on 50 images of diseased leaves of yellow passion fruit performed by 15 raters with and without the use of the standard area diagram (SADs).

Table 1 .
Effect of standard area diagram set (SADs) assessment aid on the bias, accuracy, precision and agreement of assessments of bacterial blight severity on 50 yellow passion fruit leaves by 15 raters.