Accuracy , precision and robustness of different methods to obtain samples from silages in fermentation studies

The objective of this study was to evaluate accuracy, precision and robustness of two methods to obtain silage samples, in comparison with extraction of liquor by manual screw-press. Wet brewery residue alone or combined with soybean hulls and citrus pulp were ensiled in laboratory silos. Liquor was extracted by a manual screw-press and a 2-mL aliquot was fixed with 0.4 mL formic acid. Two 10-g silage samples from each silo were diluted in 20 mL deionized water or 17% formic acid solution (alternative methods). Aliquots obtained by the three methods were used to determine the silage contents of fermentation end-products. The accuracy of the alternative methods was evaluated by comparing mean bias of estimates obtained by manual screw-press and by alternative methods, whereas precision was assessed by the root mean square prediction error and the residual error. Robustness was determined by studying the interaction between bias and chemical components, pH, in vitro dry matter digestibility (IVDMD) and buffer capacity. The 17% formic acid method was more accurate for estimating acetic, butyric and lactic acids, although it resulted in low overestimates of propionic acid and underestimates of ethanol. The deionized water method overestimated acetic and propionic acids and slightly underestimated ethanol. The 17% formic acid method was more precise than deionized water for estimating all organic acids and ethanol. The robustness of each method with respect to variation in the silage chemical composition, IVDMD and pH is dependent on the fermentation end-product at evaluation. The robustness of the alternative methods seems to be critical at the determination of lactic acid and ethanol contents.


Introduction
The extraction of liquor from silages is crucial for evaluating fermentation profiles because it allows to know the mass pH and the concentrations of fermentation endproducts, such as organic acids, ethanol and ammonia nitrogen (N-NH 3 ).
However, in some cases, obtaining silage liquor by using the traditional method of manual screw may not be possible.The addition of high doses of adsorbent additives (eg.citrus pulp) to high moisture silages has been observed to compromise the extraction of liquor by manual screw (Rodrigues, unpublished data) and make the evaluation of fermentation parameters of silages unfeasible.Batista et al. (2006) tested silages composed of 100% elephantgrass and replaced by 33%, 67% and 100% mesquite pod and reported that they could not extract liquor from the silage that contained 100% mesquite pod by using the manual screw due to the high dry matter content of that plant.Consequently, in that silage, the authors could not measure pH or N-NH 3 .
Other methods to obtain samples for determining fermentation end-products are found in the literature.Among them, there is the dilution of fresh silage with saline Ringer solution (Kizilsimsek et al., 2007;Schmidt et al., 2009), the dilution with distilled cold water (Nishino et al., 2003), the dilution in 17% formic acid and homogenization using a blender (Cherney et al., 2004), and others.In Brazil, fixing the manual screw silage liquor in 17% formic acid seems to be the most usual method.However, if differences between the techniques exist in relation to the determination of silage fermentation end-products, they are still unknown.
Therefore, the objective of this study was to evaluate accuracy, precision and robustness of two alternative methods for determining the content of organic acids and ethanol in brewers wet grain silages, in comparison with using a manual screw to extract the liquor.

Material and Methods
The study was carried out at the Laboratory of Animal Nutrition of the College of Veterinary Medicine and Animal Science of Universidade de Sao Paulo, Pirassununga, SP, Brazil.Wet brewery residue was ensiled in laboratory silos without additives and with 15% soybean hulls, 30% soybean hulls and 15% citrus pulp (as is basis), with four replicates for each additive + wet brewery residue mixture (Table 1).Sixteen 6-liter polypropylen tubes were used as laboratory silos.Wet brewery residue and the additives were weighed, homogenized and each mixture was sampled to determine the chemical composition.The mixtures were put in tubes and compacted tightly, then tubes were sealed and weighed.Compaction was carried out to exceed a density of 600 kg of fresh wet brewery residue/m 3 .Silos were kept upright in a sheltered barn and were open after 60 to 120 days of storage.
After silos were opened, the ensiled mass was homogenized by hand and sampled for chemical analyses.In the remaining silage, three different methodologies were employed to obtain material for the silage characterization: one method traditionally used and two other methods herein referred to as alternative methods.
For the first method (manual screw-press), a 400 g sample (fresh weight) was pressed in a manual screw (MA-098 model, Marconi Equipamentos para Laboratório Ltda., Piracicaba, SP) and the resulting liquor was collected.After homogenization, a 2 mL aliquot was fixed in 0.4 mL formic acid in glass flasks, which were frozen (-18 °C) until analyses.During the analyses, the liquor was thawed in room temperature and centrifuged at 1,500 xg for 10 minutes.Supernatant was aliquoted for analyses of organic acids and ethanol.This method was considered as the reference method for the evaluation of the two other alternative methods.
In the second method (17% formic acid), a 10 g sample of silage was diluted in 20 mL 17% formic acid and frozen subsequently with no pressing.The solution was kept in 50 mL capped glass flasks that were frozen (-18 °C) until analyses.In the analyses, the material was thawed at room temperature and centrifuged at 1,500 x g for 10 minutes.Supernatant was aliquoted for analyses of organic acids and ethanol.
In the last method (deionized water), a 10 g silage sample was diluted in water, without pressing and without the addition of 17% formic acid.The dilution was carried out in 20 mL deionized water and the solution was also kept frozen in glass flasks at -18 °C, until analyses.As in the two other methods, the material was thawed at room temperature and centrifuged as previously described for the determination of organic acids and ethanol.
In the material that was obtained using the three different methods, the concentrations of ethanol, acetic, propionic and butyric acids were determined by gas chromatography (Erwin et al.,1961), in a Thermo Scientific ® chromatograph (Focus GC, Thermo Fusher Scientific Inc., Waltham, MA, USA) with automatic sample injection (Thermo Electron Corporation ® , model AS-3000).The lactic acid concentration was analyzed according to Wilson (1971), by high performance liquid chromatography (HPLC), using a LC-10ADVP Shimadzu HPLC system (Shimadzu Inc., Kyoto, Japan).The content of these organic acids were presented in percentage of dry matter.
The concentrations of fermentation end-products of the material that was obtained using the two alternative methods were compared with those that were found using the reference method manual screw-press in order to evaluate the accuracy, precision and robustness of estimates.Accuracy was evaluated through the mean bias that was calculated as the difference between the values obtained using the alternative method and the values obtained using manual screw-press.Therefore, the most accurate alternative method is the one whose mean bias was closest to zero (Kohn et al., 1998).
Precision is a measure of the dispersion of the residuals, i.e., the mean variability of the distance between the predicted  However, whenever the mean bias is high (lack of accuracy), there will be an overestimation of the lack of precision, i.e., an overestimation of RMSPE, since the mean distance between the predicted and the observed values lead to an increase of the variability between predicted and observed values.Thus, precision can be better evaluated when RMSPE is adjusted for the lack of accuracy by using the residual error.According to Kohn et al. (1998), the residual error is the remaining error in model prediction after accounting for the mean bias or the prediction error excluding mean bias.Residual error can be calculated as follows: The software SAS (Statistical Analysis System, version 9.1) was used for data analyses.The data was tested for normality of residues using the Shapiro-Wilk test (Proc Univariate) and homogeneity of variance (Hartley test).In order to compare the mean concentrations of silage fermentation end-products across the three methodologies as well as the mean biases of the two alternative methods, a restricted maximum likelihood analysis of variance was carried out for a completely randomized block design to fit a model that contained the method as fixed effect and the silos (block) as random, in Proc Mixed of SAS.Mean concentrations of fermentation end-products were compared by the Tukey-Kramer adjusted test.Treatment difference of P≤0.05 was considered significant and 0.05<P≤0.10 was considered a tendency.The t-test for means equal zero (Proc Univariate of SAS) was used to evaluate the significance of mean biases.
Comparison of precision across the alternative methods used the Hartley test to perform pairwise comparisons of the residual errors, in Proc Ttest of SAS.Residuals were regressed on the values that were observed using the reference method to calculate the linear bias (slope), the model coefficient of determination (R 2 ) and the bias significance for each method, using Proc Reg of SAS.This procedure was carried out to evaluate the behavior of residuals in relation to the variation of the concentrations of fermentation end-products.
The evaluation of robustness of each alternative method was accomplished by regressing the residuals on observed contents of NDF, ADF, SC, N-NH 3 , CP and DM, IVDMD, pH and BC of silage in each silo, using Proc Reg of SAS.The slopes were compared across methods by testing the interaction between the selected independent variable and the alternative method (class variable), using Proc GLM of SAS (Meyer, 2003).

Results and Discussion
In the present study, wet brewery residue presented higher humidity and NDF (16.7% DM and 65.4% NDF) but similar CP content to that found in others studies (Table 1).In the Brazilian literature, wet brewery residue dry matter content varied from 20.7 to 27.5%; the CP varied from 20.3 to 29.9% and NDF content varied from 58 to 62% (Geron et al., 2008;Silva et al., 2010;Gilavert et al., 2011).These values demonstrated that there may be significant variability of the chemical composition of wet brewery residue according to its origin.
The addition of different levels of soybean hulls and citrus pulp to the ensiling of wet brewery residue allowed the chemical composition of silages to vary (Table 2).The greatest coefficient of variation occurred for SC (67.5%),N-NH 3 (43.8%),IVDMD (25.5%),ADF (25.4%),CP (23.8%) and DM (23.5%).This variability was favorable in this  study because it allowed evaluating the robustness of the alternative methods in variable conditions of fermentation.Although a great variability in the fermentation profile was observed, the overall ensiling process can be considered adequate regardless of the inclusion or lack of adsorbent additives, because the pH varied within a normal range (3.79 to 4.35).Furthermore, low levels of butyric acid were observed and the most abundant organic acid found was the lactic acid (Table 3), indicating the presence of Lactobacillus and low growth of Clostridium.The fermentation profile of wet brewery residue silage was similar to that reported by Wang & Nishino (2008) and Nishino et al. (2003) with respect to the pH, SC, ethanol and organic acids contents.
Although adding citrus pulp in the ensiling may increase ethanol contents (Rodrigues et al., 2005), in the present study the ethanol contents were lower than that found in sugarcane (Freitas et al., 2006;Castro Neto et al., 2008), alfalfa (Rodrigues et al., 2004) and sorghum silages (Rodrigues et al., 2002) and similar to that observed in the elephant-grass silage (Rodrigues et al., 2005).However, although the pH values were adequate, the acetic acid content was greater than the recommended values (0.8%), as it reached mean value of 2.23%.The lactic acid levels were satisfactory (5.5% on average) because they were above 3.0%, which characterizes good quality silages.The ammonia nitrogen (in percentage of the total N) was adequately below the recommended limits (8% to 12%), with minimum, maximum and mean values of 0.46%, 2.12% and 1.26%.On the other hand, the butyric acid was above the acceptable level on average (0.41%)There were no differences for the butyric and lactic acid contents across the three different methods.However, 17% formic acid presented acetic acid values that were similar to that found by manual screw-press, whereas deionized water provided the highest concentrations among the three methods (Table 3).The highest concentration of propionic acid was observed for deionized water, whereas manual screw-press resulted in the lowest estimate.Manual screwpress and deionized water provided similar concentrations of ethanol but higher than that obtained by 17% formic acid.Although the results suggested the alternative methods may differ from the reference method, it is important to investigate the accuracy of the methods through the evaluation of the mean biases.
The accuracy of the alternative methods varied according to the type of fermentation end-product analyzed (Table 4).For the acetic acid, deionized water presented the greatest mean bias when compared with 17% formic acid (P<0.05).The mean bias for 17% formic acid was not different from zero (P>0.05), which means that acetic acid concentration of silage samples obtained by the method is similar to that found in the sample obtained using the manual screw.Conversely, diluting the silage sample in deionized water had a mean bias of 0.62 percentage units on average, resulting in an overestimation of 44%.In turn, the estimates of butyric and lactic acid by the alternative methods were very accurate, since the mean biases were small and not different from zero (P>0.05).
The estimates of propionic acid were biased for both alternative methods (P<0.05);however, deionized water had the greatest mean bias (P<0.05) and consequently 17% formic acid was the most accurate among the alternative methods.In turn, ethanol estimates were more accurate for deionized water (P<0.05)although this method tended to underestimate ethanol concentrations.Dilution in 17%  formic acid underestimated ethanol concentrations by 0.07 percentage units.
Regarding the precision of the alternative methods as investigated by the residual error (Table 4), 17% formic acid resulted in more precise estimates of concentrations of all organic acids when compared with deionized water (P<0.05).However, the alternative methods did not differ with respect to the precision of ethanol estimates (P>0.05).
The linear biases of the alternative methods (Table 4) did not differ for determining the concentration of acetic acid (P>0.05).There was a trend (P<0.10) for 17% formic acid linear bias to differ from zero, which may indicate a weak relation between the resulting bias and the variation in acetic acid concentrations.This weak relation was also confirmed by a small coefficient of determination for the fitted linear regression (R 2 = 0.20).For the butyric acid, there was an important relationship between the prediction error of 17% formic acid and the acid concentration that was determined when using the reference method (P<0.01,R 2 = 0.44).The error of 17% formic acid for estimating butyric concentration was negative and the error magnitude increased as butyric acid concentration increased.
The linear bias for the dilution with water was significant and tended to be greater than the linear bias of the dilution with 17% formic acid when the concentrations of lactic acid were evaluated.The biases were positive and their magnitudes were greater when the acid concentration was close to 2%.In acid concentration of 7%, the biases decreased to zero and at greater concentrations the biases became negative and their magnitude increased.For the propionic acid, the biases were positive and their magnitude increased as the acid concentration became greater.
There were no differences between the linear biases when the ethanol concentration was evaluated.In both alternative methods, the linear biases were significant and negative.The prediction error of the alternative methods was close to zero when ethanol concentrations were 0.2%; however, their magnitude rose as the ethanol concentration increased.A strong relation (R 2 = 0.94) was observed between the residuals of 17% formic acid and the silage ethanol concentration, whereas this relation was not as strong for the deionized water residuals (R 2 = 0.24).
With respect the robustness of the methods in relation to variation of silage DM content (Table 5), no differences were observed between the slopes of the alternative methods obtained by the bias × DM content linear regression for butyric, lactic and propionic acids and ethanol.There was a trend of the slope of deionized water to be greater than the slope of 17% formic acid for the acetic acid.Both slopes were negative; however, only the slope for deionized water was significant.
The bias × CP content interaction was more pronounced for the acetic acid concentrations.There was a trend of deionized water to present a greater slope when compared with 17% formic acid.Only the slope of deionized water was significant, whereas the slope of 17% formic acid tended to be different from zero.Likewise, the CP content of the silage explained a greater portion of the error in deionized water (R 2 = 0.35) when compared with 17% formic acid (R 2 = 0.21).These results indicate that, unlike deionized water, the error in acetic acid concentration estimated by 17% formic acid was little, in comparison with the variation in the silage CP content.When the butyric acid was investigated, deionized water presented a positive slope, whereas 17% formic acid was negative.The slopes were different from each other; however, neither of them was significant.Slopes of both methods did not differ and were not significant when the lactic and propionic acids and ethanol were investigated.The slope of 17% formic acid tended to be significant in all end-products, except for the lactic acid.When evaluating the robustness with respect to the variation in silage IVDMD, there were no differences between the slopes of the alternative methods regardless of the type of end-product.Slopes were not significant for either methods when investigating the acetic, butyric and propionic acids and for deionized water in ethanol.In these cases, low coefficients of determination were observed for the linear regression between bias and IVDMD, indicating a weak relationship between the two variables.When evaluating the lactic acid, slopes of both methods were negative and significant.
The prediction error of estimates of acetic, butyric and propionic acid concentrations for both alternative methods was independent of the variation on the NDF content of silage, as indicated by the non-significant and similar slopes (Table 6).Furthermore, the coefficients of determination of the linear regressions showed weak relation between the bias and NDF content.For lactic acid, the slopes of both methods were significant (P<0.01),but that related to deionized water presented a greater magnitude (P<0.05).In addition, according to the coefficients of determination, the variation in NDF content corresponded to 76% of the error of deionized water and 39% of the error related to 17% formic acid.For ethanol, slopes were significant in this case and similar across methods.Notable difference was observed between the coefficients of determination of the linear regression, so the NDF content of the silage explained a greater portion of the prediction error in 17% formic acid.
The slopes of bias × ADF content regression did not differ across alternative methods for none of the fermentation end-products.The slopes were not significant for butyric and acetic acid and tended to differ from zero for ethanol.For lactic acid, the slopes were significant in both methods and tended to differ from zero for propionic acid in deionized water.Slopes were positive for lactic acid and negative for propionic acid.Moderate relations between bias and ADF content were found when lactic and propionic acids were evaluated.
When evaluating the error as a function of the variation in SC, the linear regression slopes were different across methods for the lactic acid.The slopes were significant for  β -slope estimate of linear regression of residuals and the independent variable of interest.Values in the same row followed by different letters differ at 5% or 10% ( ‡ ) significance level; R 2 -coefficient of determination of the linear regression model of residuals and the independent variable of interest; Pr>|t| -probability of acceptance of the null hypothesis: slope = 0; PRESS -manual screw press; DWATER -dilution in deionized water; DFORM -diluted in formic acid.
Table 5 -Robustness of alternative methods with respect to variation in dry matter (DM) and crude protein (CP) contents and in in vitro dry matter digestibility (IVDMD) of wet brewery residue silage the lactic acid in both methods, for ethanol in 17% formic acid and tended to be significant for ethanol in deionized water.Low coefficients of determination were observed in both methods for acetic, butyric and propionic acids, showing a weak relation between bias and silage SC content.
In both alternative methods, the prediction error was not related to variation in silage buffer capacity for all fermentation end-products, as seen by non-significant slopes of the bias × BC regression (Table 7).Likewise, slopes were not significant for either methods in the study of acetic, butyric and propionic acids estimates as influenced by variation in N-NH 3 (% total N).For lactic acid, the slope related to deionized water was significant and tended to be greater than the slope of 17% formic acid, which was not different from zero (P>0.05).When studying the ethanol concentration, slopes were similar across methods and the slope related to 17% formic acid differed from zero.In general, the variation in N-NH 3 explained a moderate to small fraction of the prediction error.
The slopes of the bias × silage pH regression were not significant in either methods for acetic, butyric and propionic acids, in deionized water for ethanol and in 17% formic acid for the lactic acid.The slopes differed across methods for the lactic acid and deionized water presented the greater slope.The bias × pH relation explained a great portion of the prediction error for deionized water (R 2 = 76%).
The significant linear relations between bias and the independent variables in the robustness study can be better detailed.With respect to the silage DM content, the deionized water method was not robust when determining acetic acid concentrations and both alternative methods were not robust when measuring propionic acid contents.In these cases, positive biases of great magnitudes occurred at DM contents close to 20% but biases decreased until zero at DM contents close to 40%.The deionized water method was not robust to variation in CP when acetic acid concentrations were determined.The biases were positive and lower when CP was around 15%; however, biases rose until 25% CP contents.Dilution with17% formic acid was not robust to variation in IVDMD when predicting ethanol.Biases around zero were observed when IVDMD was close to 40%; however, they had greater magnitude and were higher and negative in IVDMD between 70% to 80%.In the study of the lactic acid, the alternative methods were not robust to variation in IVDMD.When IVDMD was close to 40%, biases were positive and ranged from 1 to 3%.They remained close to zero at IVDMD within 55% to 60% and increased negatively at IVDMD from 70% to 80%.
The greatest underestimates of ethanol concentrations were found when using both methods at NDF lower than 50%.The biases approached zero as silage NDF increased until around 65%.For the lactic acid, underestimates between 1% to 3% occurred at 45% to 50% NDF, biases close to zero were found between 55% to 60% NDF and overestimates up to 4% occurred around 65% NDF.
Lower errors when determining the concentrations of propionic acid were observed at 35% to 40% ADF.In this case, biases were positive and greater at 20% to 25% ADF.For the lactic acid, errors approaching zero were shown in silages with 25% to 30% ADF; however, biases were positive and greater at higher ADF contents (40%).The errors in ethanol estimates approached zero at SC contents around 1%. Nevertheless, the underestimates magnitude increased as SC increased until 4%.Biases of lactic acid estimates were close to zero at 3% SC, positive between 1 to 3% SC and negative between 4 to 6% SC.
The deionized water method was not robust with respect to variation in N-NH 3 when determining lactic acid concentrations, and 17% formic acid presented lack of robustness concerning the variation in silage ethanol  contents.For the last method, biases were null at 2% N-NH 3 and underestimates were observed at lower N-NH 3 concentrations.Deionized water resulted in N-NH 3 underestimates of about 0.5 percentage units, null biases at 1% N-NH 3 and increasing overestimates at concentrations above this level.Similar pattern was observed with respect to the robustness to pH variation.In this case, null biases occurred in a pH range between 4.0 and 4.4 for ethanol estimates in a pH of 4.0 for lactic acid estimates.Greater underestimates for ethanol and lactic acid were shown at pH close to 3.8, whereas lactic acid overestimates occurred at pH higher than 4.2 when using the deionized water method.The present study proposed and evaluated alternative methodologies for extracting silage liquor for measuring concentrations of organic acids and ethanol in silages.It is noteworthy that there is a lack of results in the literature, so the comparison of the present findings was not possible.Overall, the present study showed that the dilution of silage samples with 17% formic acid allowed accurate estimates of acetic, butyric and lactic acid concentrations, but it resulted in low overestimates of propionic acid and underestimates of ethanol.In turn, the dilution of silage samples in water resulted in overestimates of acetic and propionic acids and small underestimates of ethanol.All estimates of organic acids and ethanol were more precise when silage samples were diluted in 17% formic acid than in water.One must bear in mind that, in studies aiming to compare different treatments (e.g., different microbial inoculants in silage), more precise methodologies are desirable.
The robustness of the proposed alternative methods with respect to variation in the chemical composition, digestibility and pH of the ensiled mass is dependent on the fermentation end-product at evaluation.Both methods were robust to variation in buffer capacity, across the observed range.However, the robustness of both methods was critical when evaluating the concentrations of ethanol and lactic acid for almost all sources of variation that were investigated.
The lack of robustness in these cases means that the accuracy of the methods is significantly dependent on the chemical profile of the ensiled mass and, in a broad manner, probably dependent on the type of material ensiled (eg.co-product, roughage or grass), species and degree of physiological maturity of the forage, presence of microbial or adsorbent additive, and other factors.Therefore, caution is needed when interpreting results of ethanol and lactic acid concentrations in other types of silages, when the alternative methods proposed here were employed.New investigations are necessary to evaluate accuracy, precision and robustness of the two alternative methods in investigations of other silage types.

Conclusions
In wet brewery residue silages, the dilution of a silage sample in 17% formic acid results in more precise estimates of fermentation end-products concentrations when compared with dilution in water.However, the accuracy of both alternative methods, when compared with the extraction of liquor by manual screw as the reference method, is dependent on variables related to the ensiled material, such as its chemical composition and in vitro dry matter digestibility.Furthermore, the robustness of the alternative methods seems to be critical for the determination of lactic acid and ethanol contents.

Table 2 -
Number of observations (n), minimum (min), maximum (max) and mean values, standard deviation (SD) and coefficient of variation (CV) for chemical composition, in vitro dry matter digestibility, buffer capacity and pH of wet brewery residue silages without additives and with citrus pulp and soybean hulls

Table 3 -
Number of observations (n), minimum (min), maximum (max) and mean values, standard deviation (SD), coefficient of variation (CV) for fermentation end-products concentrations (% of DM) in brewers wet grain silage extracts obtained by pressing in manual screw (PRESS) and by diluting in deionized water (DWATER) or in formic acid (DFORM) Values in the same column, within each item, followed by different letters differ by the Hartley test at 5% significance.4Slope,coefficient of determination and probability of acceptance of null hypothesis (linear bias = 0) of the linear regression between residuals and observed values (PRESS).DM -dry matter; RMSPE -root mean square prediction error; PRESS -manual screw press; DWATER -dilution in deionized water; DFORM -diluted in formic acid; ns -non significant.† P<0.10; * P<0.05; ** P<0.0001. 3

Table 4 -
Parameters of accuracy and precision of estimates of organic acids and ethanol concentrations in silages

Table 6 -
Robustness of alternative methods with respect to variation in neutral detergent fiber (NDF), acid detergent fiber (ADF) and soluble carbohydrates (SC) contents of wet brewery residue silage

Table 7 -
Robustness of alternative methods with respect to variation in buffer capacity, ammonia nitrogen (N-NH 3 , % total N) and pH of wet brewery residue silages