Classification of hazelnut varieties by using artificial neural network and discriminant analysis

Aim of study: This study was conducted to classify hazelnut ( Corylus avellana L.) varieties by using artificial neural network and dis criminant analysis. Material and methods: The physical, mechanical and optical properties of 11 hazelnut varieties were determined for three major axes. The parameters of physical, mechanical and optical properties were included as independent variables, while hazelnut varieties were included as dependent variables. Models were created for each of the three axes to classify hazelnut varieties. Main results: Classification success rates with Artificial Neural Networks (ANN) and Discriminant Analysis (DA) were found as 89.1% and 92.7% for X axis, as 92.7% and 92.7% for Y axis and as 86.8% and 88.7% for Z axis, respectively. The classification results of ANN and DA models were found to be very close to each other. Both models can be used in the classification of hazelnut varieties. Research highlights: The results obtained for the identification and classification of hazelnut varieties show the feasibility and effecti veness of the proposed models. contributions: the data the paper. Both authors designed and conducted the experiments, read and approved the final article.


Introduction
Hazelnut (Corylus avellana L.) is a very healthy nutrient for humans and animals since it contains fatty acids, vitamins, proteins and minerals. It has a considerable economic contribution to the agriculture sector in Turkey (Alasavar et al., 2003). Hazelnut is one of the most important raw materials for the pastry and chocolate industry (Fallico et al., 2003). It is widely used to add distinct and favorable flavour and texture to pastries, confectioneries, cornflakes, dairy products and sauces (Parcerisa et al., 1998).
The production area and amount of hazelnut were 966,196 ha and 863,888 tons in the world, respectively. In the same year, 515,000 tons of hazelnut were produced in Turkey in the total area of 728,381 ha (FAOSTAT, 2018). Turkey is ranked as the first in the world with approximately 70% of the hazelnut production.
The physical properties of agricultural materials are important parameters used to design agricultural and food processing machines such as crop processing, handling, sieving, storing and drying machines and equipment. In addition, their physical properties, mechanical properties and optical properties are used for classification of varieties (Mohsenin, 1970;Tabatabaeefar, 2003). Therefore, the physical properties of agricultural materials must be determined accurately and the correlations among them must be mathematically modelled.
Classification of hazelnut varieties is very important for the development of agricultural industry and profitable farming of hazelnut. Varieties are extremely important factors in terms of productivity and processing hazelnut. Each variety has its specific physical and mechanical properties. The varieties can be distinguished from each other based on their physical properties. In this way, variety standardization can also be achieved.
It is very important for producers, industrialists, traders and consumers to know the varieties of that agricultural products they are interested in. Producers want to know the variety of the agricultural product they grow to make a correct breeding; industrialists and dealers want to know the variety of the agricultural product they trade and process to set standards; and consumers want to know the variety of the agricultural product they buy. For these reasons, reliable methods are needed to identify varieties (Chen et al., 2010;Pourreza et al., 2012).
The recent increase of commercial interest in hazelnut has caused the development of new analytical methods for the standardization and traceability of its varieties. The insufficiency of univariate statistical methods for examining possible associations among more than two features creates a problem. This problem occurs because all features may affect each other. This situation causes complex associations among those features. Therefore, the application of multivariate statistical methods has gained value (Oddone et al., 2009). Multivariate statistical methods have been developed to explain multiple associations (Özdamar, 2004). Artificial neural networks and discriminant analysis are also used in classification studies (Yang et al., 2003).
Artificial neural networks (ANN) is one of the most important artificial intelligence methods. ANN is frequently used in biological practices for classification. ANN is extremely efficient and successful in studies with non-linear data. For this reason, ANN has a very important potential in the classification of agricultural products (Visen et al., 2002;Dubey et al., 2006;Guiné et al., 2015).
Discriminant analysis is a multivariate statistical method that determines the relationships between categorical dependent variables and metric independent variables aiming to group objects into two or more categories. Discriminant analysis finds the linear combinations of independent variables with the largest difference between group means. In the analysis, the groups are determined in advance as special research groups. A set of distinguishing variables related to the characteristics that measure the expected difference in the groups are chosen to distinguish among the groups. Linear combinations are made from these variables. Then, new observations are classified into the groups by using linear combinations (Alpar, 2017).
Image processing technique is one of the technologies used for classification. Image processing technique consists of the analysis of images taken with a camera or a scanner and transferred to computers by using image processing programs (Demirbaş & Dursun, 2007). Image processing is frequently used in classification, sorting, quality control and automation processes of agricultural products such as fruit, vegetable and cereals. Thus, productivity increases and production costs decrease. In addition, higher quality and more healthy products are provided to consumers (Chen et al., 2010).
The study area of our work is defined as the identification of fruit classification tasks to determine class according to the specific type. In addition, the study is an example of practical application to be used in different samples by presenting approaches to researchers. This study aims to classify hazelnut varieties with ANNs and discriminant analysis using physical, mechanical, and optical properties. 'İncekara','Kalınkara','Kan','Kuş','Okay28','Palaz','Sivri','Tombul','Uzunmusa','Yassı Badem',and 'Yuvarlak Badem' hazelnut varieties were used in the study.

Material and methods
Biological material test device (Lloyd Instrument LRX Plus, Lloyd Instruments Ltd, An AMATEK Company and NEXYGEN Plus software) was used to find out hazelnut shells' mechanical properties of under compression load. In order to measure the mechanical properties of hazelnut shells, the device was equipped with a load cell of 500 N and the load cell had a measurement accuracy of 0.5% (Fig. 1).
Colour parameters (L, a, b) were measured by using colorimeter (Minolta, model CR-400). A camera with a resolution of 12 mega pixels was used to take the pictures.
Adobe Photoshop and MATLAB program were used for image processing. MATLAB was used for modelling with artificial neural networks and SPSS 21.0 program was used for the statistical evaluation of discriminant analysis and the data.
In the study, physical, mechanical and optical properties of hazelnut varieties were determined for each of three major axes (X, Y, Z) (Fig. 2). Physical properties include geometric mean diameter, sphericity, grain volume, surface area, shell thickness and grain weight; mechanical properties include rupture force, rupture energy and deformation, optical properties include colour parameters (L, a, b) and image processing parameters.
In order to determine the physical, mechanical and optical properties, 50 hazelnuts randomly were selected for each variety to determine their physical properties. The length (L), width (W) and thickness (T) and shell thickness values of each hazelnut were measured. Grain weights were measured according to Aydın (2002). Measurements were taken at 8% humidity.
The physical properties of hazelnut cultivars used in the study are presented ( Table 1). The following equations were used to calculate physical properties include geometric mean diameter (D g ), sphericity (Ø), surface area (S) and grain volume (V) of hazelnut (Mohsenin, 1970;Jain & Bal, 1997;Taner et al., 2018).
Mechanical properties were found by performing rupture tests on three major axes with biological material test device. Rupture force and deformation were determined from the obtained force-deformation graph, while rupture energy was determined from the area under this graph. The mechanical properties of hazelnut cultivars used in the study are presented (Table 2).
Colour parameters (L, a, b) were measured with a colorimeter. "L" value means white when it is 100 and black when it is 0; "a" value means red when it is positive and green when it is negative; and "b" value means yellow when it is positive and blue when it is negative (Francisco et al., 2021). The optical properties of hazelnut cultivars used in the study are presented (Table 2).
Twelve independent and 11 dependent variables were used in the study. Independent variables were geometric mean diameter, sphericity, grain volume, shell thickness, rupture force, rupture energy, deformation, L, a, b and image processing parameters. Dependent variables were the 11 hazelnut varieties.
A total of six models were developed with independent variables: ANN X , ANN Y and ANN Z for ANN, and DA X , DA Y and DA Z models for discriminant analysis.
Four hundred and eighty-one data were used for the X axis, while 476 data were used for the Y axis and 472 data were used for the Z axis. The data used in the study were normalized between 0 and 1 (Purushothaman & Srinivasa, 1994); 80% of the data were used for training, while 10% were used for test and 10% were used for validation to develop ANN and discriminant models.
Feed Forward Back Propagation network structure was used for ANN models (Jacobs, 1988;Minai & Williams, 1990). Levenberg-Marquardt and Scaled Conjugate Gradient algorithms were used as training algorithms (Levenberg, 1944;Marquardt, 1963). As a transfer function, Hyperbolic Sigmoid Tangent function was used in the hidden layer, while softmax function was used in the output layer (Bishop, 1995). Termination cycle limit of the training algorithm was chosen as 1000. Verification was conducted in all trainings in the study. Training was terminated successfully when six verification errors stayed sequentially below the chosen error value or one verification error was equal to 0.
In order to find out the optimum number of neurons, the number of neurons in the hidden layer was increased sequentially in selected amounts and their performance was evaluated (Taner et al., 2010). As a result of these tests, the network with the lowest error was chosen. For the single layer network, firstly, the number of neurons was increased by 10 from 10 to 100 and the networks were compared. Secondly, ten more trials were conducted by increasing or decreasing the number of neurons by one ranging from five more or five less neurons for the network that passed the test with the least error during the initial training.
In the discriminant analysis, univariate normal distribution analysis and then multivariate normal distribution analysis were conducted for each variable by using Mahalanobis distance (Sharma, 1996;Alpar, 2017). Equality of variance-covariance matrices was tested with Box-M test. In order to test if there was a multi linear connection problem, correlation matrix consisting of correlation coefficients between variables was calculated (Tabachnick & Fidell, 2007). Discriminant function is a Here, y is the discriminant index, a is the constant, b is the discriminant coefficient and x is the independent variables (Tabachnick & Fidell, 2007). Eigenvalues, canonical correlation and Wilks' Lambda values were calculated to evaluate discriminant functions (Kalaycı, 2018). Image processing was carried out by capturing the image, filte-ring, converting the image to grey image, thresholding and determining the dimensions (Gonzalez & Woods, 2008;Ağın & Taner, 2015) (Fig. 3).

Classification with discriminant analysis
In discriminant analyses which were conducted separately for the three major axes, the existences of the variables fitting into multiple normal distribution, the variance-covariance matrices being equal and not having multiple connection problems were tested. The data used in the study showed multivariate normal distribution. The variance-covariance matrices were tested with Box-M test. Quadratic discriminant function was used since the variance-covariance matrices were not equal (Tabachnick & Fidell, 2007). In order to test whether there were multi linear connection problems, correlation matrix consisting of the correlation coefficients between variables were calculated (Alpar, 2017). The corre-lation coefficient of hazelnut surface area was found to be well above 0.90; therefore, this variable was excluded from the independent variables (Alpar, 2017). In discriminant analysis, the most effective variables for separating the varieties were found to be sphericity, grain volume, geometric mean diameter and colour parameters, while the least effective ones were found to be deformation, rupture energy, rupture force and grain weight variables. For all three main axes, the significance level of independent variable groups was checked with equality of

Classification with discriminant analysis for X axis (DA X )
Eigenvalues of discriminant functions, canonical correlation and Wilks' Lambda values were calculated. Eigenvalues of the functions were found to range between 0.014 and 11.474. Eigenvalues greater than 0.40 can be considered to be good (Kalaycı, 2018). Greater eigenvalue expresses that more of the variance in the dependent variable can be explained by the function. The first function explained 57.5% of the total variance. Canonical correlation values of the functions were between 0.118 and 0.959 and the square of these values ranged between 0.014 and 0.92. The square of canonical correlation value shows the variance that the function can explain in the dependent variable. Wilks' Lambda values, which show the part of the functions that cannot be explained by the differences between groups of the total variance, were between 0.001 and 0.986. These values indicate the significance of the eigenvalue statistics for each discriminant function (Kalaycı, 2018).
In order to evaluate the accurate classification of discriminant analysis, that is, the success of the analysis, the classification table of training and test sets was obtained (Table  3). According to the classification results obtained with DA X model, the discriminant function that was chosen for the training set classified 397 of the 426 samples accurately into their varieties, while 29 samples were classified inaccurately. The general accurate classification rate of DA X model was found as 93.2%. On the other hand, DA X model classified 51 of the 55 samples in test set to their varieties accurately, while 4 samples were classified inaccurately. Its general accurate classification rate was found to be 92.7%.

Classification with discriminant analysis on Y axis (DA Y )
Eigenvalues of the discriminant functions were found to range between 0.005 and 12.397. The first function ex-plained 54.6% of the total variance. Canonical correlation values of the functions were between 0.071 and 0.962 and the square of these values ranged between 0.005 and 0.93. Wilks' Lambda values were between 0.0003 and 0.995 (Kalaycı, 2018).
According to the classification results obtained with DA Y model, the discriminant function that was chosen for the training set classified 403 of the 421 samples accurately into their varieties, while 18 samples were classified inaccurately. The general accurate classification rate of DA Y model was found as 95.7%. On the other hand, DA Y model classified 51 of the 55 samples in test set to their varieties accurately, while 4 samples were classified inaccurately. Its general accurate classification rate was found to be 92.7%. (Table 3).

Classification with discriminant analysis on Z axis (DA Z )
Eigenvalues of the discriminant functions were found to range between 0.007 and 11.910. The first function explained 57.7% of the total variance. Canonical correlation values of the functions were between 0.081 and 0.960 and the square of these values ranged between 0.006 and 0.92. Wilks' Lambda values, which show the part of the functions that cannot be explained by the differences between groups of the total variance, were between 0.001 and 0.993 (Kalaycı, 2018).
According to the classification results obtained with DA Z model, the discriminant function that was chosen for the training set classified 399 of the 419 samples accurately into their varieties, while 20 samples were classified inaccurately. The general accurate classification rate of DA Z model was found as 95.2%. On the other hand, DA Z model classified 47 of the 53 samples in test set to their varieties accurately, while 6 samples were classified inaccurately. Its general accurate classification rate was found to be 88.7% (Table 3).

Classification with artificial neural networks
Since the network structure used in ANN applications is similar for three main axes, they were evaluated together. In the study, the best results were obtained by using feed forward back propagation network structure and Levenberg-Marquardt training algorithm. In addition, hyperbolic sigmoid tangent function was chosen for the hidden layer and softmax function was chosen for the output layer as transfer functions. While the networks were trained, it was ensured that the training was terminated before overfitting occurred by controlling it with the verification set. The success of the network was controlled with the test set. The best training results were obtained  Varieties  y1  y2  y3  y4  y5  y6  y7  y8  y9  y10  y11  Success (%)   Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test   y1  35  5  3  1 Varieties  y1  y2  y3  y4  y5  y6  y7  y8  y9  y10  y11  Success (%)   Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test   y1  39  5  100 Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test  Training  Test   y1  39  5   Classification of hazelnut varieties by using ANN and discriminant analysis from the networks having 60, 60 and 40 neuron numbers, respectively for X, Y and Z axes.

Classification with artificial neural networks on X axis (ANN X )
According to the classification results obtained with ANN X model, 419 of the 426 hazelnut samples were classified accurately into their varieties and 7 samples were classified inaccurately. Its general accurate classification rate was found to be 98.4%. By using the test set, 49 of 55 hazelnut samples were classified accurately into their varieties, while 6 samples were classified inaccurately. Its general accurate classification rate was found to be 89.1%. The samples belonging to 'İncekara', 'Kuş', 'Okay28', 'Sivri', 'Tombul' and 'Yuvarlak Badem' hazelnut varieties were classified accurately (Table 4).

Classification with artificial neural networks on Y axis (ANN Y )
According to the classification results obtained with ANN Y model, 418 of the 421 hazelnut samples were classified accurately into their varieties and 3 samples were classified inaccurately. Its general accurate classification rate was found to be 99.3%. By using the test set, 51 of 55 hazelnut samples were classified accurately into their varieties, while 4 samples were classified inaccurately. Its general accurate classification rate was found to be 92.7%. While one sample belonging to 'Kuş' variety was classified inaccurately, three samples belonging to 'Kalınkara' variety was classified inaccurately. All the other samples were classified accurately into their varieties (Table 4).

Classification with artificial neural networks on Z axis (ANN Z )
According to the classification results obtained with ANN Z model, 413 of the 419 hazelnut samples were classified accurately into their varieties and 6 samples were classified inaccurately. Its general accurate classification rate was found to be 98.6%. By using the test set, 46 of 53 hazelnut samples were classified accurately into their varieties, while 7 samples were classified inaccurately. Its general accurate classification rate was found to be 86.8%. While all samples belonging to 'İncekara', 'Kalınkara', 'Okay28', 'Yassı Badem' and 'Yuvarlak Badem' varieties were classified accurately, one sample belonging to 'Kan', 'Kuş', 'Palaz', 'Tombul' and 'Uzunmusa' was classified inaccurately and two samples belonging to 'Sivri' were classified inaccurately (Table 4).
Variety classification is required for many different purposes such as machine design, evaluation of consumer preference, cultivar identification for cultivar registrations, investigation of heritability of fruit shape traits, analysis of shape abnormalities (Kays, 1999;Cannon & Manos, 2001;Beyer et al., 2002;Brewer et al., 2007;Taner et al., 2021).
Similar results were obtained with both ANN and discriminant analysis models for the variety classification of hazelnuts. The reason for obtaining these similar results with both models can be due to processing data used in this study with multivariate normal distribution analyses before fitting the models. In fact, ANN can be more successful in nonlinear data according to the literature (Ford et al., 2004).
In previous studies, hazelnut cultivars have not been classified by using both ANN and discriminant analysis based on their physical, mechanical, and optical properties. However, one study using discriminant analysis were available in the literature (Menesatti et al., 2008). In that study, four traditional Italian hazelnut cultivars were classified by discriminant analysis. They found the classification success of 77.5%-98.8%. ANN and discriminant analysis methods were used separately or together to classify different agricultural products (Chen et al., 2010;Azizi et al., 2016;Taner et al., 2018).
In conclusion, the most effective variables for the classification of hazelnut varieties were sphericity, grain volume, geometric mean diameter and image processing variables, while the least effective ones were deformation, rupture energy, rupture force and grain weight variables. ANN and discriminant analysis gave close results to each other probably because of that nonlinear data used in this study were normally distributed. ANN and discriminant analysis models can be used successfully for the classification of hazelnut varieties. The classification performance of ANN and discriminant analysis models for the same hazelnut varieties grown in different parts of Turkey and in different climatic conditions can be determined in future studies. In addition, smartphone applications using the models proposed in this study can be developed.