Development and validation of a standard area diagram set to evaluate bacterial blight on yellow passion fruit leaves

ABSTRACT

Bacterial blight is one of the most important diseases of yellow passion fruit. The aim of this study was to develop and validate a set of standard area diagram (SADs) to improve the accuracy and precision of bacterial blight severity assessments on yellow passion fruit leaves. The proposed SADs, composed of illustrations of symptomatic leaves with nine severity levels (1, 2.5, 5, 10, 20, 30, 40, 50 and 62%), were evaluated by 15 raters. Accuracy, precision and reliability of the SADs were validated by quantifying and comparing the assessments performed by 15 raters with and without the use of SADs. The SADs improved accuracy (coefficient of bias, C_b = 0.940 without SADs and 0.967 with SADs), precision (correlation coefficient, r = 0.901 without SADs and 0.949 with SADs) and overall agreement (Lin’s concordance correlation coefficient, ρ_c = 0.863 without SADs and 0.935 with SADs) of severity estimates. Inter-rater reliability also significantly improved when the SADs were used (coefficient of determination, R² = 0.717 without and R² = 0.880 with the SADs; intra-class correlation, ρ = 0.798 without and ρ = 0.926 with the SADs). The developed SADs improved accuracy, precision and reliability of disease severity assessments and are recommended as an aid to assess bacterial blight on yellow passion fruit leaves.

Keywords
Passiflora edulis; Xanthomonas axonopodis pv. passiflorae; Disease severity; Phytopathometry

RESUMO

A mancha-bacteriana é uma das principais doenças do maracujazeiro amarelo. O objetivo deste estudo foi desenvolver e validar uma escala diagramática para melhorar a acurácia e precisão na avaliação da severidade da mancha-bacteriana em folhas de maracujazeiro-amarelo. A escala diagramática proposta, composta de ilustrações de folhas sintomáticas com nove níveis de severidade (1, 2.5, 5, 10, 20, 30, 40, 50 e 62%), foi validada por 15 avaliadores. A acurácia, precisão e a reprodutibilidade da escala diagramática foram validadas pela quantificação e comparação das avaliações realizadas por 15 avaliadores com e sem o uso da escala diagramática. A escala melhorou a acurácia (fator de correção, C_b= 0,940 sem escala e 0,967, com escala), precisão (coeficiente de correlação, r = 0,901 sem escala e 0,949 com escala) e concordância (coeficiente de correlação de concordância de Lin, ρ_c = 0,863 sem escala e 0,935 com escala) das estimativas de severidade. A reprodutibilidade entre avaliadores melhorou com o uso da escala (coeficiente de determinação, R² = 0,717 sem e R² = 0,880 com a escala, e coeficiente correlação intra-classe ρ = 0,798 sem e ρ = 0,926 com a escala). A escala diagramática desenvolvida melhorou a acurácia, precisão e reprodutibilidade das avaliações da severidade da doença, sendo recomendada como auxiliar na avaliação da mancha-bacteriana em folhas de maracujazeiro-amarelo.

Palavras-chave
Passiflora edulis; Xanthomonas axonopodis pv. passiflorae; Severidade de doença; Fitopatometria

Yellow passion fruit (Passiflora edulis Sims) is the most widely cultivated and consumed passion fruit species produced in Brazil. This country has been the worlds largest producer for more than two decades, accounting for approximately 80% of the global production (¹⁸). However, a gradual decrease in production area and yield has been observed since 2010. In the 2015 season, there was a reduction of 33% in production area, 45% in yield, and 17% compared to the 2010 season (¹⁴).

Although the crop has potential to reach 50 t ha^-1 (¹⁸), phytosanitary issues related to the yellow passion fruit constitute one of the greatest limitations to increase yields and production area. Bacterial blight, caused by the bacterium Xanthomonas axonopodis pv. passiflorae (Pereira) Gonçalves & Rosato, causes chlorosis, necrosis and plant death (¹³). Diseased leaves exhibit translucid and anasarcous small spots, which become necrotic and dark brown with chlorotic halos encircling spots. As the disease develops, the entire leaf is affected, causing wilt with subsequent defoliation, thereby reducing productivity (²⁷).

Bacterial blight occurs in all production regions in Brazil, particularly during the warmer and more humid periods, and causes significant yield losses in yellow passion fruit. The bacteria may invade the vascular system and spread to all plant tissues, leading to plant wilt and even plant death in susceptible cultivars. The disease can cause drastic reductions in the lifespan of affected plants (²⁷).

Disease assessment is an essential tool to conduct appropriate research, study epidemiology, quantify crop loss, screen for resistance and effective bactericides, evaluate the efficacy of treatments in field experiments and conduct plant disease survey (¹⁵); therefore, a standardized method to accurately assess disease severity is crucial. Severity is a visual estimation of the percentage of the diseased sampling unit area in relation to the total sampling unit area and it is a subjective assessment, influenced by rater bias (²³). In this pathosystem, severity is estimated as the percentage of diseased leaf tissue in relation to the total leaf area. Ideally, these estimates should be accurate (estimated values close to the true values) (²¹), precise (lack of variation in disease estimates when evaluating the same sampling units) (¹⁷) and reliable (consistent disease estimates between or among raters) (²¹). The product of precision and accuracy provides a measure of agreement, which is a variable often used to compare the estimated values with the true values (¹⁷). Standard area diagrams (SADs) have been used to improve accuracy, precision and reliability of disease severity assessments by providing a standard reference of illustrated images of diseased plants, leaves or other plant parts, with distinct severity levels that can be used in comparison to actual diseased leaves (or their images) (⁵).

A SADs for assessment of bacterial blight on passion fruit leaves was developed and validated by Miranda (¹⁹) during her PhD thesis, using mono-lobed leaves. However, the leaves of yellow passion fruit are mono-lobed, especially at the seedling stage, and are three-lobed during most of the crop cycle (²⁸). The SADs proposed by Miranda (¹⁹) can be recommended and useful for assessing seedlings but not for epidemiological research in the field. Therefore, it is important to develop and validate SADs to quantify the severity of bacterial blight in three-lobed leaves. Similarly, a SADs for assessment of bacterial blight was developed and validated by Castillo (⁶) in three-lobed leaves, published in a non-indexed journal in Colombia, but it presented only a set of six diagrams and started with 5% severity, failing to contemplate minor severity found in the field, considering that in this disease there is a tendency to overestimate the evaluation of the disease.

The aim of this study was to develop and validate a SADs that includes the full range of bacterial blight severities encountered in the field in order to assist raters in more accurately estimating disease severity on yellow passion fruit leaves.

Material and Methods

Development of SADs

Three hundred yellow passion fruit leaves that exhibited a wide range of bacterial blight severities were arbitrarily collected from commercial orchards in the municipality of Araquari, Brazil. Infected leaves were obtained between January 2013 and July 2015. Leaves were individually scanned using an HP scanner (Scanjet 3800 model) with an image resolution of 300 dpi. The images were processed, and each one was rated using the image analysis software QUANT v.1.0.1 (²⁶) to determine the percentage of diseased leaf area in relation to the total leaf area, i.e. the actual disease severity. These images were used as a reference for evaluating the accuracy, precision and reliability of rater estimates with and without the aid of SADs. Based on the image analysis, a set of nine images spanning the minimum and maximum disease severity was used to develop a nearly linear SADs. Using a digitalized yellow passion fruit leaf shape as template, a SADs with nine distinct disease severities was created by using the image editing software PAINT.NET (http://www.getpaint.net). The software designed lesions with shapes and distribution of actual diseased leaves.

Validation of SADs

For assessment purposes, a total of 50 images of yellow passion fruit leaves with distinct levels of disease severities were projected, one at a time, onto a projector screen using a MS Power Point file. A group of 15 raters with no experience in quantifying disease severity were asked to estimate the disease severity of each of the 50 diseased leaf images set without and with the use of SADs. Firstly, the raters evaluated the leaf images without the SADs. Then, after a 15-min break, a new randomized sequence of images was displayed, and the raters estimated severity using the SADs as an assessment aid. The raters were instructed to compare each leaf image with the SADs to estimate the percentage of diseased area considering both symptoms of chlorosis and necrosis, which were included in the estimates of disease severity.

Figure 1
Standard area diagram set for bacterial blight (Xanthomonas axonopodis pv. passiflorae) severity on yellow passion fruit leaves (Passiflora edulis). The numbers represent percent (%) leaf area showing symptoms (necrosis and chlorosis) of the disease.

Data analyses

Accuracy and precision of the estimates by each rater, with and without the use of SADs, were determined based on Lin’s concordance correlation coefficient (LCCC, ρc), as described by Nita et al. (2003) and Bock et al. (2010). The absolute errors (estimated severity minus actual severity for each leaf × rater) were determined to show the bias of data points from the actual value.

Inter-rater reliability of the estimates was assessed based on the coefficient of determination (R²) from linear regression analysis of relationships between severity estimates for all pairs of raters (21) and on intracluster (or intra-class) correlation (ρ) for all raters combined, as described by Nita et al. (²⁰).

An equivalence test was used to verify the difference between means of the treatments (with SADs and without SADs) for all variables of the LCCC analysis (r, Cb, υ, u and ρc) and for the inter-rater reliability (R²). The equivalence test assessed whether there was a significant difference in means and calculated the 95% confidence intervals (CIs) for each statistic (the difference between means) by bootstrapping using the percentile method (in an equivalence test, the null hypothesis (H₀) is nonequivalence, the opposite of H₀) (1, 29, 30). All analyses were based on 10000 balanced bootstrap samples, and the 95% CIs were calculated based on the difference between the means of groups; if the CIs included zero, there was no significant difference (α = 0.05).

All statistical analyses were calculated in R. The epi.ccc function of the epiR package (²⁵) was used to obtain LCCC statistics. The built-in boot.sample R function was used for the equivalence test. The ρ was calculated with the icc function of the irr R package (¹¹).

Results

The SADs of nine images of yellow passion disease severity on fruit leaves ranged from 1 to 62%. Leaves exhibited characteristic lesions of bacterial blight: translucid and anasarcous small spots, which become necrotic and dark brown with chlorotic halos encircling spots (¹³, ²⁷).

All assessments performed with and without the use of SADs resulted in linear relationships between estimated and actual disease severity of bacterial blight. Lin’s concordance correlation analysis demonstrated that the raters’ assessments of disease severity were more similar to the actual severity when the SADs were used, compared to the assessments performed without using the aid. The equivalence test showed that the statistics of most parameters (ρc, C_b and r) were significantly improved when the SADs were employed, indicating that accuracy and precision of the estimated values were significantly improved. Although there was a slight numerical decrease in the scale bias (v) and location bias (u) (v = 0.988 and 1.052, and u = 0.217 and 0.241, with and without the SADs, respectively), it was not significant.

Accuracy (C_b), precision (r) and agreement (ρ_c) of disease severity assessments were significantly improved when SADs were used. Without the SADs, accuracy (C_b) was 0.940, the value improved to 0.967 when assessments included the SADs as an aid. Precision (r) was 0.901 in the absence of SADs, but when the SADs was employed, precision was 0.949. Agreement (ρ_c), which combines measures of accuracy and precision, was 0.863 when the SADs was not used, and was increased to 0.935 when SADs were used as an assessment aid. There was a reduction in absolute error, with most estimates falling within ± 20% of the actual value compared to +49/-33 when estimates were made without SADs. There was a tendency to overestimate disease severity with the use of the SADs, as 75% raters showed positive u values (data not shown). This trend was further proved since most estimates of the majority of raters had positive absolute errors.

Figure 2
Relationship between actual and estimated severity (A, B) and the absolute error (estimated minus actual severity) (C, D) of assessments of a set of 50 images of bacterial blight (Xanthomonas axonopodis pv. passiflorae) of yellow passion fruit leaves (Passiflora edulis) without (A, C) and with (B, D) the use of a set of standard area diagram (SADs), 15 raters. The solid line (A, B) represents the best-fitting line, and the dotted line is the concordance line, which represents perfect agreement between actual and estimated severity (slope of 1, intercept of 0). Low absolute errors indicate that the estimated severity was similar to the actual severity.

Table 1
Effect of standard area diagram set (SADs) assessment aid on the bias, accuracy, precision and agreement of assessments of bacterial blight severity on 50 yellow passion fruit leaves by 15 raters.

Variables	Means1					Difference2 between means		95% CIs3 of the difference
		With SADs		Without SADs
Scale (υ)4		(0.039) 0.988		(0.199) 1.052		(0.001) -0.063		-0.023 -0.155
Location (u)5		(0.155) 0.217		(0.249) 0.241		(0.001) -0.025		-0.064 -0.119
Coefficient of bias (Cb)6		(0.037) 0.967		(0.085) 0.940		(0.001) 0.027		-0.067 0.001
Correlation coefficient (r)7		(0.039) 0.949		(0.069) 0.901		(0.001) 0.043		-0.074 0.019
LCCC (ρc)8		(0.039) 0.935		(0.108) 0.863		(0.001) 0.071		-0.116 0.034

• ¹
  Standard deviation in parentheses
• ²
  Mean of the difference between each rating. Standard errors are in parentheses (bootstrap calculated values).
• ³
  Confidence intervals (CIs) based on 10.000 bootstrap samples. If the CIs embrace zero, the difference is not significant (α=0.05). Significant differences in bold.
• ⁴
  Scale bias, or slope shift (υ, 1 = no bias relative to the concordance line).
• ⁵
  Location bias, or height shift (u, 0 = no bias relative to the concordance line).
• ⁶
  Coefficient of bias (_b) is a measure of accuracy. It measures how far the best-fit line deviates from the concordance line.
• ⁷
  Correlation coefficient (r) is a measure of precision.
• ⁸
  Lin’s concordance correlation coefficient (ρ_c) combines precision (r) and accuracy (_b) to measure agreement with the true value.

Figure 3
Frequency of inter-rater reliability using the coefficient of determination (R²) of visual assessments of bacterial blight on 50 images of diseased leaves of yellow passion fruit performed by 15 raters with and without the use of the standard area diagram (SADs).

Table 2
Inter-rater reliability of visual assessments by 15 raters of bacterial blight on 50 leaves of yellow passion fruit with and without the use of standard area diagram set (SADs) aid. Inter-rater reliability is measured by the intracluster correlation coefficient (ρ) and coefficient of determination (R²).

Statistics		With SADs		Without SADs
Intra-class correlation coefficient (ρ)		0.926 F, P > F: L= 262 (<0.0001); R= 22 (<0.0001)		0.798 F, P > F1: L= 83 (<0.0001); R= 21(<0.0001)
Mean inter-rater coefficient of determination (R2)2		0.880 (0.783 - 0.943)		0.717 (0.299 - 0.917)
	Mean difference3 = 0.164 (0.001), 95% CIs 0.140 - 0.187

• ¹
  F-value for L = leaf, R = rater. P-value in parentheses.
• ²
  Mean coefficients of determination estimated from pairwise comparisons of assessments by all raters.
• ³
  Mean of the difference between each rating, with standard errors in parentheses (bootstrap calculated value), confidence intervals (CIs) were based on 10.000 bootstrap samples. If the CIs embrace zero, the difference is not significant (α=0.05).

Without the use of the SADs, 66.67% pairwise comparisons had an R² < 0.80, but when the SADs was employed, only 2.86% pairwise comparisons had an R² < 0.80. The inter-rater reliability of the estimates was used as another indicator of the importance of the use of SADs. The equivalence test revealed that the statistical parameters, based on the coefficient of determination (R²) and intracluster correlation coefficient (ρ), were significantly improved when raters used SADs (R² = 0.880 and 0.717, and ρ = 0.926 and 0.798 , with and without the SADs, respectively).

Discussion

The main objective of this study was to develop and validate a standard area diagram set to assess bacterial blight severity on yellow passion fruit leaves. The diagrams reflected the range of severity found in the field and demonstrated an improvement in accuracy, precision and reliability of estimates.

Visual ratings are the most often used methods to estimate disease severity. Other technologies have been proposed, for example, image analysis and hyperspectral imaging, which can make great contributions to assessments, but visual quantification will continue to be the main approach for assessing plant disease (³). The use of SADs as an aid to visually estimate disease severity can considerably enhance the quality of assessment data, which is helpful to choose the most appropriate disease control (²⁴).

Disease quantification is an important procedure in epidemiological study and management of bacterial blight on yellow passion fruit. The SADs proposed in this report demonstrated to be an effective tool to estimate disease severity, as they improved accuracy, precision and reliability of the data, reducing the chances of drawing incorrect conclusions, i.e. failing to reject the null hypothesis when the null hypothesis was false (type II error) (³, ²²). Standard area diagram sets have recently been proved to be useful for several diseases of important crops, such as coffee leaf rust (⁵), potato early blight (¹⁰), wheat spot blotch (⁹), loquat scab (¹²) and soybean frogeye leaf spot (⁸).

The nine diagrams used in this SADs similar to the number used in SADs of other pathosystems, which is of practical use and sufficient to guide estimates of disease severity (²⁹). An increase in the number of diagrams has not resulted in more accurate or reliable disease estimates, as demonstrated by Bock et al. (²), when three-, five-, seven- or 10-diagram SADs were compared. The maximum disease severity and the disease severity range likely have a greater effect on accuracy and reliability. It is understood that the use of a high number of diagrams can be time-consuming and therefore can affect the quality of assessments, especially under field conditions. Severity evaluations using an interval scale instead of the nearest percentage might be negatively affected by a low number of diagrams (less than six) (², ²⁹).

Similar to what was found in other studies, the nine diagrams, representing the range 1 – 62% disease severity, were linearly arranged, to ensure more precise and accurate assessments (⁷, ⁸). Some SADs have used a logarithmic arrangement based on the Weber-Fechner law, which assumes a logarithmic relationship between estimated and actual disease. But this methodology has been questioned (³, ²³) because the Weber-Fechner law has been found to be partially invalid (¹⁶, ¹⁷) and may lead to false conclusions (³, ²³).

Accuracy, precision and reliability of the SADs to estimate bacterial blight severity were determined using an equivalence test (based on 95% CIs by bootstrapping of the difference between means), which is an appropriate statistical method to be used for agreement studies (³⁰). In the present study, the equivalence test demonstrated that most of the statistics variables (ρ_c, C_b and r) were improved when raters used the diagrams to estimate severity, compared to when SADs were not used. Agreement and reliability values were also improved in other SADs that were statistically analyzed by the equivalence test (², ⁷, ⁸, ⁹, ¹⁰).

In most studies of SADs, disease severity has been overestimated by raters (⁵, ²⁹). However, some reports show the severity being underestimated by raters (⁷, ⁸, ¹²). The tendency to over or underestimate disease severity might be affected by lesion size, shape and number, as much as disease distribution might be (³). In the present study, the location bias (u) values were greater than zero for most of the raters’ estimates using the SADs, indicating a tendency for overestimation.

The use of the SADs to assess bacterial blight severity contributed to a reduction in the absolute errors. In order to reduce the absolute errors and standardize results, computerized disease assessment training programs and human guidance (¹), the SADs used in this report can be used to train evaluators to assess disease severity. Speed and standardization that result from the use of SADs can counterbalance the presence of some level of absolute error in the estimates (²).

Inter-reliability of estimates between raters was used to indicate the effectiveness of the SADs. Our findings show that the inter-rater reliability was higher with the use of the diagram to estimate severity, compared to when the SADs were not used, which is congruent with previous reports (12, 29). Reliability between evaluators was measured by the linear relationship between disease severity estimation of the same leaf image assessed by different individuals. Preferably, the coefficient of determination (R²) value should approach 1.0, indicating that the estimates made by different raters are similar or identical when R² is 1.0. For the evaluations made without using the SADs, the R² values were lower than 0.80 for 66.67% comparisons between raters. Using the SADs, R² values were higher than 0.80 for 97.14%, indicating high reliability of estimates when the SADs is used.

Liner regression remains a useful analysis for evaluating how well estimates for disease severity provided by one rater agree with the severity assessments of one or more other raters, when the same set of disease leaf samples (experimental units) are assessed (²²). The R² generated by this analysis provides a quantitative measure of the amount of variation in one rater’s severity assessments, which can be explained by another rater’s severity estimates (i.e., inter-rater agreement).

The SADs developed and validated in this study will be a valuable tool for raters as an aid to estimate disease severity in epidemiological investigations, comparative studies on disease control methods, pathotype characterization, germplasm screening, and other applications that require accurate, precise and reliable assessment of bacterial blight severity on yellow passion fruit leaves in the field. The proposed SADs proved to be useful for obtaining more accurate, precise and reliable estimates of bacterial blight on yellow passion fruit leaves and consequently minimizing errors in results.

ACKNOWLEDGEMENTS

The authors acknowledge the Coordination for the Improvement of Higher Education Personnel - CAPES and Federal Institute Catarinense for providing the financial support and the student grants.

REFERENCES

• ¹
  Bardsley, S.J.; Ngugi, H.K. Reliability and accuracy of visual methods used to quantify severity of foliar bacterial spot symptoms on peach and nectarine. Plant Pathology, Malden, v.62, n.2, p.460-474, 2013.
• ²
  Bock, C.H.; Hotchkiss, M.W.; Wood, B.W. Assessing disease severity: accuracy and reliability of rater estimates in relation to number of diagrams in a standard area diagram set. Plant Pathology, Malden, v.65, n.2, p.261-272, 2016.
• ³
  Bock, C.H.; Poole, G.; Parker, P.E.; Gottwald, T.R. Plant disease severity estimated visually, by digital photography and image analysis, and by hyperspectral imaging. Critical Review in Plant Sciences, Philadelphia, v.29, n. 1/3, p.59-107, 2010.
• ⁴
  Campbell, C.L.; Madden, L.V. Nonlinear disease progress curves. In: Kranz, J. (Ed.). Epidemics of plant disease: mathematical modelling and analysis. 2nd ed. Berlin: Springer, 1990. p.181-229.
• ⁵
  Capucho, A.S.; Zambolim, L.; Duarte, H.S.S.; Vaz, G.R.O. Development and validation of a standard area diagram set to estimate severity of leaf rust in Coffe arabica and C. canephora Plant Pathology, Malden, v.60, n.6, p.1144-1150, 2011.
• ⁶
  Castillo, S.Y.; Rivera, J.F.; Hoyos, L.M. Escala diagramática para evaluar la severidad de la bacteriosis de la gulupa (Passiflora edulis Sims). Fitopatología Colombiana, v.34, n.2, p.41-45, 2010.
• ⁷
  CostaLage, D.A.; Marouelli, W.A.; Duarte, H.S.S.; Café Filho, A.C. Standard area diagrams for assessment of powdery mildew severity on tomato leaves and leaflets. Crop Protection, Oxford, v.67, n.9, p.26-34, 2015.
• ⁸
  Debona, D.; Nascimento, K.J.T.; Rezende. D.; Rios, D.J.A.; Bernardeli, A.A.; Silva, L.; Rodrigues, F. A set of standard area diagrams to assess severity of frogeye leaf spot on soybean. European Journal Plant Pathology, Dordrecht, v.142, n.3, p.603-614, 2015.
• ⁹
  Domiciano, G.P.; Duarte, H.S.S.; Moreira, E.N.; Rodrigues, F.A. Development and validation of a set of standard area diagrams to aid in estimation of spot blotch severity on wheat leaves. Plant Pathology, Malden, v.63, n.4, p.922-928, 2014.
• ¹⁰
  Duarte, H.S.S.; Zambolim, L.; Capucho, A.S.; Junior, A.F.N.; Rosado, A.W.C.; Cardoso, C.R.; Paul, P.A.; Mizubuti, E.S.G. Development and validation of a set of standard area diagrams to estimate severity of potato early blight. European Journal of Plant Pathology, Dordrecht, v.137, n.2, p.249-257, 2013.
• ¹¹
  Gamer, M.; Lemon, J.; Fellows, I.; Singh, P. irr: Various Coefficients of Interrater Reliability and Agreement. R Package Version 0.84. Hamburg, 2012.
• ¹²
  González-Domínguez, E.; Martins, R.B.; Del Ponte, E.M.; Michereff, S.J.; García-Jimenez, J.; Armengol, J. Development and validation of a standard area diagram set to aid assessment of severity of loquat scab on fruit. European Journal of Plant Pathology, Dordrecht, v.139, n.2, p.1-10, 2014.
• ¹³
  Halfeld-Vieira, B.A.; Nechet, K.L. Ocorrência da mancha-bacteriana do maracujazeiro em Roraima. Fitopatologia Brasileira, Brasília, v.31, n.2, p.5113, 2006.
• ¹⁴
  IBGE – INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA. Maracujá Brasília, 2013. Available at <ftp://ftp.ibge.gov.br/Producao_Agricola/Producao_Agricola_Municipal_[anual]/2013/pam2013.pdf>. Acessed on: 10 nov. 2016.
  » ftp://ftp.ibge.gov.br/Producao_Agricola/Producao_Agricola_Municipal_[anual]/2013/pam2013.pdf
• ¹⁵
  Kranz, J. The methodology of comparative epidemiology. In: Kranz, J.; Rotem, J. (Ed.). Experimental techniques in plant disease epidemiology Heidelberg: Springer-Verlag, 1998. p.279-289.
• ¹⁶
  Lin, L.I. A concordance correlation coefficient to evaluate reproducibility. Biometrics, Washington DC, v.45, n.1, p.255-268, 1989.
• ¹⁷
  Madden, L.V.; Hughes, G.; Van den Bosch, F. The study of plant disease epidemics Saint Paul: APS Press, 2007. 432p.
• ¹⁸
  Meletti, L.M.M. Avanços na cultura do maracujá no Brasil. Revista Brasileira de Fruticultura, Jaboticabal, v.33, n.1, p.7783-91, 2010.
• ¹⁹
  Miranda, J.F. Reação de variedades de maracujazeiro amarelo (Passiflora edulis Sims f. flavicarpa Deg.) a bacteriose causada por Xanthomonas campestres pv. passiflorae 2004. 62f. Dissertação (Mestrado em Fitopatologia) - Escola Superior de Agricultura Luiz de Queiroz, Universidade de São Paulo, Piracicaba.
• ²⁰
  Nita, M.; Ellis, M.A.; Madden, L.V. Reliability and accuracy of visual estimation of Phomopsis leaf blight of strawberry. Phytopathology, Saint Paul, v.93, n.8, p.995-1005, 2003.
• ²¹
  Nutter, F.W.; Schultz, P.M. Improving the accuracy and precision of disease assessments: selection of methods and use of computer-aided training programs. Canadian Journal Plant Pathology, Montreal, v.17, n.1, p.174-184, 1995.
• ²²
  Nutter, F.W.; Gleason, M.L.; Jenco, J.H.; Christinas, N.C. Assessing the accuracy, intra-rater repeatability, and inter-rater reliability of disease assessment systems. Phytopathology, Saint Paul, v.83, n.8, p.806-812, 1993.
• ²³
  Nutter, F.W.; Esker, P.D. The role of psychophysics in phytopathology. European Journal of Plant Pathology, Dordrecht, v.114, n.2, p.199-213, 2006.
• ²⁴
  Nutter, F.W.; Esker, P.D.; Netto, R.A.C. Disease assessment concepts and the advancements made in improving the accuracy and precision of plant disease data. European Journal of Plant Pathology, Dordrecht, v.115, n.1, p.95-103, 2006.
• ²⁵
  Stevenson, M. Epir: An R Package for the Analysis of Epidemiological Data. R package version 0.9-43. Berlin, 2012. 70p.
• ²⁶
  Vale, F.X.R.; Fernandes Filho, E.I.; Liberato, J.R. QUANT: A software plant disease severity assessment. In: International Congress of Plant Pathology, 8., 2003, Christchurch. Proceedings Christchurch: New Zealand, 2003. p.105.
• ²⁷
  Viana, F.M.P.; Freire, F.C.O.; Cardoso, J.E.; Vidal, J.C. Principais doenças do maracujazeiro na Região Nordeste e seu controle Fortaleza: Embrapa Agroindústria Tropical, 2003. 12p. (Comunicado técnico, 86).
• ²⁸
  Wosch, L.; Imig, D.C.; Cervi, A.C.; Moura, B.B.; Budel, J.M.; Santos, C.A.M. Comparative study ofapple-converted-spaceapple-converted-spacePassifloraapple-converted-spaceapple-converted-spacetaxa leaves: I. A morpho-anatomic profile. Brazilian Journal of Pharmacognosy, Campina Grande, v.25, n.4, p.328-343, 2015.
• ²⁹
  Yadav, N.V.; Vos, S.M.; Bock, C.H.; Wood, B.W. Development and validation of standard area diagrams to aid assessment of pecan scab symptoms on fruit. Plant Pathology, Malden, v.62, n.2, p.325-335, 2013.
• ³⁰
  Yi, Q.; Wang, P.P.; He, Y. Reliability analysis for continuous measurements: equivalence test for agreement. Statistics in Medicine, Boston, v.27, n.15, p.2816–2825, 2008.

Publication Dates

• Publication in this collection
  Oct-Dec 2018

History

• Received
  09 Feb 2017
• Accepted
  15 Sept 2017