Predicting the metabolizable energy content of corn for ducks: a comparison of support vector regression with other methods

Support vector regression (SVR) is used in this study to develop models to estimate apparent metabolizable energy (AME), AME corrected for nitrogen (AMEn), true metabolizable energy (TME), and TME corrected for nitrogen (TMEn) contents of corn fed to ducks based on its chemical composition. Performance of the SVR models was assessed by comparing their results with those of artificial neural network (ANN) and multiple linear regression (MLR) models. The input variables to estimate metabolizable energy content (MJ kg) of corn were crude protein, ether extract, crude fibre, and ash (g kg). Goodness of fit of the models was examined using R, mean square error, and bias. Based on these indices, the predictive performance of the SVR, ANN, and MLR models was acceptable. Comparison of models indicated that performance of SVR (in terms of R) on the full data set (0.937 for AME, 0.954 for AMEn, 0.860 for TME, and 0.937 for TMEn) was better than that of ANN (0.907 for AME, 0.922 for AMEn, 0.744 for TME, and 0.920 for TMEn) and MLR (0.887 for AME, 0.903 for AMEn, 0.704 for TME, and 0.902 for TMEn). Similar findings were observed with the calibration and testing data sets. These results suggest SVR models are a promising tool for modelling the relationship between chemical composition and metabolizable energy of feedstuffs for poultry. Although from the present results the application of SVR models seems encouraging, the use of such models in other areas of animal nutrition needs to be evaluated. Additional key words: maize; poultry; nutritive value; chemical composition; artificial neural network; multiple linear regression. * Corresponding author: ako.faridi@stu-mail.um.ac.ir Received: 20-03-13. Accepted: 12-11-13 Abbreviations used: AME (apparent metabolizable energy); AMEn (apparent metabolizable energy corrected for nitrogen); ANN (artificial neural network); CF (crude fibre); CP (crude protein); EE (ether extract); ME (metabolizable energy); MLR (multiple linear regression); SVM (support vector machine); SVR (support vector regression); TME (true metabolizable energy); TMEn (true metabolizable energy corrected for nitrogen); VIF (variance inflation factor). Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria (INIA) Spanish Journal of Agricultural Research 2013 11(4): 1036-1043 Available online at www.inia.es/sjar ISSN: 1695-971-X http://dx.doi.org/10.5424/sjar/2013114-4220 eISSN: 2171-9292 ME prediction models for ducks 1037 composition. However, despite the ability of ANN models to handle complex nonlinear problems (Faridi et al., 2012a), this approach is not necessarily simple and may provide an apparently good fit to the data-set from which predictive equations are derived, but a poor predictive performance on newly introduced data. Support vector machines (SVM), i.e. supervised learning models with associated learning algorithms, can be used for classification, regression or other tasks (Cortes & Vapnik, 1995; Vapnik et al., 1997). In recent years, they have been introduced as a new technique for solving a variety of learning, classif ication and prediction problems (Cristianini & Shawe-Taylor, 2000). Support vector regression (SVR), the regression version of SVM, was developed to estimate regression functions (Drucker et al., 1997) and similar to SVM, it is capable of solving non-linear problems (Nandi et al., 2004). SVR models have been successfully applied across a broad range of areas in engineering, science and economics (e.g. Kara et al., 2011) but, to our knowledge, application to animal nutrition studies has not been investigated. Therefore, the objectives of this study were 1) to test the ability of SVR models to estimate apparent ME (AME), apparent ME corrected for nitrogen (AMEn), true ME (TME), and true ME corrected for nitrogen (TMEn) of corn for ducks based on its chemical composition, and 2) to compare the predictive performance of SVR to that of ANN and multiple linear regression (MLR) models. Material and methods

composition. However, despite the ability of ANN models to handle complex nonlinear problems (Faridi et al., 2012a), this approach is not necessarily simple and may provide an apparently good fit to the data-set from which predictive equations are derived, but a poor predictive performance on newly introduced data. Support vector machines (SVM), i.e. supervised learning models with associated learning algorithms, can be used for classification, regression or other tasks (Cortes & Vapnik, 1995;Vapnik et al., 1997). In recent years, they have been introduced as a new technique for solving a variety of learning, classification and prediction problems (Cristianini & Shawe-Taylor, 2000). Support vector regression (SVR), the regression version of SVM, was developed to estimate regression functions (Drucker et al., 1997) and similar to SVM, it is capable of solving non-linear problems (Nandi et al., 2004). SVR models have been successfully applied across a broad range of areas in engineering, science and economics (e.g. Kara et al., 2011) but, to our knowledge, application to animal nutrition studies has not been investigated. Therefore, the objectives of this study were 1) to test the ability of SVR models to estimate apparent ME (AME), apparent ME corrected for nitrogen (AME n ), true ME (TME), and true ME corrected for nitrogen (TME n ) of corn for ducks based on its chemical composition, and 2) to compare the predictive performance of SVR to that of ANN and multiple linear regression (MLR) models.

Data sources
Data used to develop the SVR and ANN models for AME, AME n , and TME n were taken from Zhao et al. (2008), and information reported by Zhao et al. (2008) and Zhou et al. (2010) was used to develop the TME prediction models. There were 36 records of observations for AME, AME n and TME n and 42 for TME. For AME, AME n , and TME n , the models investigated (SVR, ANN, and MLR) used 27 randomly selected observations for calibration, and the remainder (n = 9) as the testing data set, whereas for TME, 29 and 13 randomly selected records were used for calibration and testing, respectively. Ranges on the data used to develop the SVR and ANN models for AME, AME n , TME, and TME n are presented in Table 1. Quantitative examination of the predictions produced was made using R 2 (amount of variance of the dependent variable explained or accounted for by the model), mean square error, and bias.

Support vector regression model development
As SVM is a non-parametric statistical learning technique, no assumptions about the underlying data Table 1. Ranges of the data used to develop support vector regression, artificial neural network, and multiple linear regression models to estimate metabolizable energy (AME: apparent metabolizable energy, AME n : apparent ME corrected for nitrogen, TME: true ME, TME n : true ME corrected for nitrogen) of corn for ducks distribution are required (Cristianini & Shawe-Taylor, 2000). The SVM formulation, which follows the principle of structural risk minimization, has been demonstrated to be superior to the empirical risk minimization principle employed by conventional ANN models (Yang & Shieh, 2010). Structural risk minimization minimizes the upper bound on the expected error whereas ANN minimizes the error on the calibration data set. This difference gives SVM greater ability to generalize, which is the goal of statistical learning (Vapnik, 1995). The basic idea in SVR is to map the input data onto a higher dimensional plane via nonlinear mapping. A linear regression problem is then obtained and solved in this space (Scholkopf & Smola, 2002). A kernel function is introduced to make the support vector algorithm nonlinear. The algorithm performs the regression estimation by risk minimization where risk is measured by a loss function (for details see . Before calibrating the SVR models, the type of kernel function needs to be chosen and three parameters (υ, C, and ε) need to be determined. The υ is the kernel parameter, C is the regularization parameter, and ε is the radius of a tube within which the regression function must lie after the successful learning. Parameter ε is the accepted deviation (error) between observed and predicted values in the εinsensitive loss function. If the predicted value is within the ε-tube, the loss is zero. If the predicted value is outside the tube, the loss equals the magnitude of the difference between the predicted value and the radius ε of the tube. Therefore, ε is a precision parameter representing the radius of the tube located around the regression function, and the region enclosed by the tube is known as ε-intensive zone (Lahiri & Ghanta, 2008). These three parameters may be set empirically by the user. A detailed explanation of kernel functions and parameters can be found in Cheng et al. (2011). To use all the data for calibration and testing, a subset-swapping method is commonly applied (Cristianini & Shawe-Taylor, 2000). This technique is known as cross-validation, the statistical practice of partitioning a data sample into subsets such that the analysis is initially performed on a single subset, while the other subset(s) are retained for subsequent use in conf irming and validating the initial analysis. The technique estimates the generalization error of a given model and uses all the data to construct and test the model (Witten et al., 2011). In this study, we used a 10-fold cross-validation algorithm to f ind the best value of the SVR free parameters. The 10-fold cross validation procedure splits the calibration data set equally into 10 smaller subsets. During each fold of the calibration stage, every subset is used as the testing data set once and the remaining sets are used for calibration. The total number of misclassified samples is accumulated to compute f inal accuracy. This algorithm is provided in the software Statistica (StatSoft, 2009). There are several kernels that can be used for SVM model construction including linear, polynomial, radial basis function and sigmoid. Radial basis function is by far the most popular choice of kernel type (StatSoft, 2009). Since SVR only deals with one output at a time, one prediction model must be constructed for each objective. In this study, therefore, four SVR models were constructed to estimate the AME, AME n , TME, and TME n (MJ kg -1 ) of corn fed to ducks. The variables of interest for constructing the models were crude protein (CP), ether extract (EE), crude fibre (CF), and ash (g kg -1 ). Two different random data groups (calibration and testing sets) were used to develop the models. Statistica Machine Learning version 8.0 was used to construct, calibrate and validate the SVR models (StatSoft, 2009). The configuration of each SVR model developed is summarized in Table 2.

Artificial neural network model development
In this study, feed-forward multilayer perceptron models (the most common type of ANN model) were constructed to estimate AME, AME n , TME, and TME n (a separate model for each variable). The configuration of all the models developed consisted of one hidden layer, and the hyperbolic tangent and identity were employed as activation functions in hidden and output units, respectively. The quasi-Newton method with 400 cycles was used as the calibration algorithm while the hidden and output weight decays were set at default values of 0.001 and 0.0001, respectively. The number of hidden neurons was determined using a trial and error method to achieve best predictive performance in both the calibration and testing sets. The network randomization was set to normal with mean and variance of 0 and 0.1, respectively. This option specifies how the weights should be initialized at the beginning of the calibration process (Faridi et al., 2012b). Statistica Neural Networks version 8.0 software was used to construct and calibrate the ANN models (StatSoft, 2009). The calibration and testing data sets used for the ANN models were the same as those used to develop the SVR models.

Multiple linear regression model development
MLR models were constructed to estimate AME, AME n , TME, and TME n and compared with the SVR and ANN models. The data investigated were subjected to MLR analysis using the REG procedure of SAS (2003). All the input variables (CP, EE, CF, and ash) were considered in developing the MLR models. The models were developed on the same calibration data set used for the SVR and ANN models, and the testing data were used to evaluate their performance.

Results and discussion
Predictive ability of the SVR models and their configuration (kernel function and parameters) are summarized in Table 2. Results for the ANN and MLR models are also shown in Table 2. R 2 values for the SVR models ranged from 0.856 to 0.982 compared with 0.708 to 0.943 and 0.666 to 0.930 for the ANN and MLR models, respectively. The MLR equations to estimate metabolizable energy from chemical composition (CF, CP, EE and ash) are shown in Table 3. The parameter estimates and variance inflation factor (VIF) for the MLR models are summarized in Table 3. VIF is a common measure of multi-collinearity. Observed, predicted, and residual values for TME of the models investigated are shown in Table 4. Only results for the TME models are reported owing to limitations of space. Table 2. Statistics and information on support vector regression, artificial neural network, and multiple linear regression models developed to estimate metabolizable energy (AME: apparent metabolizable energy, AME n : apparent ME corrected for nitrogen, TME: true ME, TME n : true ME corrected for nitrogen) of corn for ducks Item AME I AME n I TME I TME n In this paper, the machine learning approach SVR is introduced and used to develop models for estimating the ME of corn for ducks based on its chemical composition. The motivation for this study is threefold. First, SVR is not so well-known as other alternatives (e.g. conventional statistical methods of MLR and ANN) in the field of animal nutrition. Second, its performance seems well-suited to problems in poultry nutrition yet no reference information is available in the literature. Third, determination of ME of poultry feedstuffs can be an expensive and timeconsuming task. Therefore, developing flexible efficient models to estimate the ME of feedstuffs is of primary interest to nutritionists.

Calibration Test All Calibration Test All Calibration Test All Calibration Test All
Although SVR models have shown excellent generalization performance, a problem that faces the user of the technique is how to choose a kernel and specify its parameters. In our study, radial basis function was used as the kernel function as it tends to give better performance. In order to conduct the present study, four SVR, four ANN, and four MLR models were developed to estimate the AME, AME n , TME, and TME n of corn for ducks. Model performance was assessed after generating output by the models. Previous studies have shown the ability of ANN to estimate the TME n of poultry feedstuffs based on their chemical composition (Ahmadi et al., 2008;Sedghi et al., 2011). The results of our study reveal that the SVR approach offers a competitive alternative to existing powerful ANN models. Comparison of the predictive ability of SVR, ANN and MLR showed that the performance of SVR (in terms of R 2 ) on the whole data set was greater than that of both ANN and MLR. The same f indings were obtained for the calibration data set, where the goodness-of-fit attained with SVR was greater than that with ANN and MLR. However, the results were different for the testing set. Predictive ability of SVR was less than (AME n , AME, Table 3. Variable estimates and variance inflation factor of multiple linear regression models developed to estimate metabolizable energy (AME: apparent metabolizable energy, AME n : apparent ME corrected for nitrogen, TME: true ME, TME n : true ME corrected for nitrogen) of corn for ducks and TME n ), or greater than (TME) that of ANN. Accuracy of SVR for the testing set was less than (AME n ), close to (TME n ) or greater than (AME and TME) that of MLR.
These results reveal that SVR models are a promising tool for estimating the ME of corn. In this study VIF was calculated to determine the level of correlation among the variables. Usually, values larger than 10 suggest that multi-collinearity might be causing estimation problems (Chatterjee et al., 2000).
Variables with VIF > 10 should be eliminated or utilized in separate models or both. However, in this study, severe multi-collinearity was not observed among the variables (Table 3). Our results indicated that, in the MLR models, crude fibre was negatively correlated with AME, AME n , TME, and TME n , while regression coefficients for CP, EE, and ash were not statistically significant (Table 3). The same findings were reported by Zhao et al. (2008). SVR models are known as universal approximations of any function to a desired degree of accuracy (Kecman, 2005). SVM and SVR models are particularly appealing due to their ability to handle small data sets successfully, often producing better classification or predictive accuracy than traditional methods (Mantero et al., 2005). However, it is worth pointing out that SVR like ANN are highly data-based models, and therefore the use of different data sets is required to prove the effectiveness of SVR in poultry and animal nutrition generally. In summary, metabolizable energy content of corn for ducks can be predicted from chemical composition with a high degree of accuracy using SVR models, with a statistical performance comparable to or better than that attained with other approaches (ANN or MLR). The SVR approach offers a promising alternative to ANN and MLR in poultry nutrition to predict the energy value of feedstuffs from chemical composition. Although the application of SVR in this study was promising, further evaluation of this methodology in other areas of animal nutrition is suggested.