A machine vision system for classification of wheat and barley grain kernels

This study presents in detail a machine vision system that classifies objects into two classes. The procedure for the classification comprises two stages: a training stage and a testing stage. A feature vector, which is a sorted list of features that maximize the classification power, is computed in the training stage. Object classification was accomplished in the testing stage by means of discriminant analysis (DA) and K-nearest neighbors (K-NN) algorithms. The system was applied to the classification of wheat and barley grain kernels. Results obtained allow the researchers to conclude that in the classification of wheat and grain kernels with the presented system: (i) a high classification accuracy can be obtained; (ii) the employment of morphologic, color, and texture feature types together offers better accuracy than the employment of only one feature type; (iii) the extraction of the maximum radius, the green mean, and the y mean of the gray level co-occurrence matrix (GLCM) for 90° allows the highest classification accuracy; and (iv) the employment of more than three features increases the computational cost and may also reduce the classification accuracy. Additional key words: cereal grains classification; color; digital image processing; features; morphology; texture; seed identification.


Resumen
Sistema de visión artificial para la clasificación de granos de trigo y cebada Este estudio presenta en detalle un sistema de visión artificial que clasifica objetos en dos clases. El procedimiento por el que se realiza la clasificación se compone de dos etapas, una de entrenamiento y otra de prueba. En la etapa de entrenamiento se obtiene un vector de características, que es una lista ordenada de características que maximiza el poder de clasificación. En la etapa de prueba se lleva a cabo una clasificación basada en el análisis discriminante (AD) y en el algoritmo de los K vecinos más próximos. El sistema fue aplicado a la clasificación de granos de trigo y cebada. Los resultados obtenidos permiten concluir que en la clasificación de granos de trigo y cebada mediante el sistema presentado: (i) se puede obtener una precisión alta en la clasificación; (ii) el empleo de características morfológicas, de textura y de color de forma conjunta ofrece mejores resultados que el empleo de características de un sólo tipo; (iii) la extracción del radio máximo, el valor medio del componente verde y el valor y medio del nivel de gris de la matriz de co-ocurrencia GLCM orientada 90°ofrece los mejores resultados de clasificación; y (iv) el empleo de un número de características superior a tres incrementa el coste computacional y puede reducir la precisión en la clasificación.
Palabras clave adicionales:características; clasificación de granos de cereal; color; identificación de semillas; morfología; procesado digital de imágenes; textura. field tasks, such as guiding systems in sewing (Leemans and Destain, 2007) and agriculture-related tasks performed in laboratories, as in the case of the product inspection of vegetables (Marchant et al., 1990). This last kind of study includes works relating to cereal grain identification (Visen et al., 2002), and, in fact, machine vision is usually employed to detect weed seeds in cereal grains (Granitto et al., 2002).
Accurate identif ication of weed seeds in cereal grains should serve to prevent weeds developing in certain types of crops, thereby improving crop quality. For this reason, most of the studies focus on systems capable of identifying different grain types. These systems allow for the detection of weed seeds in commercial cereal grain lots (Churchill et al., 1993;Chtioui et al., 1996;Paliwal et al., 2003). There are several studies specific to the identification of wheat kernels, the most important ones being those by Zayas et al. (1989), Luo et al. (1999 and Majumdar and Jayas (2000a,b,c,d).
This article has two objectives: (i) to propose a machine vision system to classify objects between two possible types and (ii) to configure the proposed system for an optimal wheat and barley grain classification, and to obtain the classifying system accuracy for these two cereal grain types.

Material
A BenQ D E520 digital camera was used for image acquisition. The camera was stabilized using a Velbon DV-45 tripod, and a 20-watt desk lamp was employed for lighting the area to be photographed. The pictures were taken in total darkness. The kernels were placed on a white background, and a direct overhead beam from a desk lamp was cast on them, so as to avoid the formation of shadows. The camera was set on the tripod and angled to the kernels in the same fashion as the desk lamp. The distance between the camera lens and the seeds was 25 cm.
The LabVIEW development environment with the artificial image package IMAQ Vision 8.2.1 was employed to develop the software applications of this study. The software development and the performed tests were carried out on a Dell Inspiron 6400 laptop. This computer had a 1.83 GHz Intel Core 2 Duo processor, 2 GB of RAM memory and the Windows XP Media Center 2002 service pack 2 operating system.

Methods
Ten images of wheat grains and 10 images of barley grains were taken for the system training. Each one of these training images had around 60 grain samples. Ten images of wheat and barley grains were taken for the system testing. Each one of these testing images had around 60 grains, with approximately the same quantity of wheat and barley grains. The total number of wheat and barley kernels used in the testing system was 545. All the images were acquired in JPEG format, over the RGB space and with a 5-megapixel resolution (2,560 × 1,920 pixels). Figure 1 shows examples of training and testing images.
The methodology of this work was comprised of two stages: (i) the system training, where some parameters of the system were obtained; and (ii) the system testing, where the system performance was measured. The system training was accomplished according to the flow diagram of Figure 2. First, the image was converted to 256 gray levels. Next, an edge enhancement was accomplished by means of the mask presented in Table 1, where the x parameter was chosen as 9.8 after visual tests. Then, an automatic thresholding segmentation to separate the kernels from the background was performed. The necessary threshold value was obtained from the B histogram of the images by means of the next iterative process (Parker, 1993): 1. The number of pixels for each gray level was multiplied with the gray level value and summed.
2. The sum was divided by the total number of pixels to obtain the first threshold value, X 1 .
3. The same procedure was performed for the two parts, i.e., 0 to X 1 and X 1 + 1 to 255 (the number of gray level values in the image), to generate two more numbers, P 1 and P 2 . The mean of P 1 and P 2 gave the second threshold value, X 2 .
4. Now the procedure was performed for values from zero to X 2 and X 2 + 1 to 255, to generate P 3 and P 4 . Taking the mean of P 3 and P 4 gave the new threshold, X 3 .
5. This process was repeated until X n was equal to X n + 1 . This stabilized value of X was taken as the threshold for the image.
The images obtained after this thresholding were binary; the value of the pixels of kernels was 1 and the value of the pixels of the background was 0 (Fig. 3a). Then, various morphological operations were carried out to obtain the desired images. Initially, a dilation operation aimed at closing the contours of every kernel in the images was carried out. Next, a hole-filling algo-rithm was applied, followed by an erosion operation with the same structural element. This last operation was done in order to recover the original shape of the kernels (Fig. 3b).
Each region of the processed images was labeled. The value of the center of mass and the original orien-tation values for each kernel were computed. Then, each region was rotated over the center of mass in order to make an appropriate measurement of dimensions during the feature extraction process. Since more computation time is needed when processing color images, prior to its rotation, each grain sample color image was 674 F. Guevara-Hernandez and J. Gomez-Gil / Span J Agric Res (2011) 9(3), 672-680 transformed into a grayscale-based image with 256 gray levels. In later processes, either the grayscale rotated image or the original color unrotated image was used depending upon the later needs. Figure 4 shows two samples of individual cereal grain kernels before and after their rotations.
The next step was to extract 99 features from each cereal grain kernel, of which: ⎯ 21 were morphological features: area, perimeter, thinness ratio, length, width, equivalent rectangle long side, equivalent rectangle short side, ratio equivalent rectangle sides, the seven Hu moments, rectangular aspect ratio, maximum radius, minimum radius, radius ratio, radius standard deviation and Haralick radius. Majumdar and Jayas (2000a) provided complete information about the meaning and computation of these morphological features.
⎯ 6 were color features: the mean values and standard deviations for R, G and B components.
-72 were texture features. Initially, the number of gray levels in the images was reduced from 256 to 32. Gray level co-occurrence matrix (GLCM) and gray level run length matrix (GLRLM) were calculated for different angles. Texture features were extracted using statistical methodology (Gonzalez and Woods, 2006). Majumdar and Jayas (2000c) provided complete information about the meaning and computation of these morphological features. In the end, 72 features were extracted. These were: i) mean of X, mean of Y, variance of X, variance of Y, uniformity or entropy, maximum probability, correlation homogeneity, cluster shade and cluster prominence of the GLCM for angles 0°, 90°, 45°and 135°; ii) short run emphasis (SRE), long run emphasis (LRE), gray level non-uniformity (GLN), run length non-uniformity (RLN), low gray level run emphasis (LGRE), run percentage (RP) and high gray level run emphasis (HGRE) of GLRLM for angles 0°, 90°, 45°and 135°.
The number of features extracted was too high for fast computation. Furthermore, some of these features were correlated and provided similar information, thus making the system inefficient. Therefore, it was necessary to make a classification of the features based on their contribution to the model, taking these correlations into account. For this purpose, A machine vision system for classification of wheat and barley grain kernels 675  initially, the feature values were normalized by means of Eq. [1]: where x tk is a feature value normalized, x k is a feature value, E[x k ] is the mean value of a feature for all kernels, σ k is the standard deviation of the values of a feature in all kernels, k is the feature index, and i is the kernel index. After applying Eq. [1], the features achieved a mean equal to zero and a mean error equal to one. Then, the features were arranged in a features vector, choosing as the first component of this vector the feature that had a larger Fisher discriminant ratio (FDR), which represents a measure of the contribution to the model of a feature and is computed by Eq.
where i is the feature index, M is the number of features, μ is the mean value of a feature, and σ is the standard deviation of the values of a feature. The next components of the features vector were selected by means of a sequential forward selection (SFS) algorithm, which employed a criterion function based on the scatter matrix (Theodoridis and Koutroumbas, 2006). This algorithm adds a new feature to the vector features, choosing the feature that offers a larger value in the criterion function. The criterion function chosen was based on the scatter matrix, which is defined by: where trace is the sum of eigenvalues, S m is the mixture scatter matrix, and S w is the within-class scatter matrix.
where p i is the a priori probability of class ω i , which is p i ≅ n i /N being the number of samples in class ω i , out of a total of N samples, and μ i the vector of the mean of each class. The within-class scatter matrix S w is defined by: where S i is the covariance matrix for class ω i , given by: being μ i the vector of the mean of each class. This SFS procedure was repeated until the features vector contained all extracted features.
The system performance was evaluated in the system testing stage, which was accomplished according to the flow diagram of Figure 5. In this diagram, a new task appears, the classification task. It was accomplished in two ways: (i) by means of a discriminant analysis (DA) parametric classifier based on the normal distribution and (ii) by means of the non-parametric classifier K-nearest neighbor (K-NN).
The DA parametric classifier chosen was based on the normal distribution given by: where ω i is the i possible class, S W is the common group covariance matrix, m i is a vector with the mean of the features of the class i, and x¯i is a sample of the features vector to be assigned to a class. The measured features x¯ are assigned to the class i in which is larger (Webb, 2002).
The non-parametric classif ier K-NN assigns the sample of the features vector to the class represented more often by the K nearest samples of the features vector in the training stage (Duda et al., 2001). Unlike parametric classifiers, this non-parametric classifier needs to maintain all training data.
Finally, the hold-out method was used to assess system performance (Theodoridis and Koutroumbas, 2006). According to this method, the samples were divided into two different data sets: the training data set that was used in the system training stage and the test data set that was used in the system test stage.

Results and discussion
The system training was performed with (i) only morphologic features, (ii) only color features, and (iii) morphologic, color, and texture features. Then, the results of the system training were the three vector features that optimize the wheat and barley classification with these feature types ( Table 2).
The system testing was performed using (i) a DA parametric classif ier based on normal distribution function and (ii) the nonparametric classifier based on K-NN. Both classif iers were tested with the three different feature vectors obtained in the system training. The system accuracy in the classif ication of wheat and barley grain kernels with both classifiers  was obtained (Table 3 and Fig. 6). Table 3 and Figure 6 show that (i) the combination of the morphologic, color, and texture features offers better classification accuracy than the extraction of features of only one type, (ii) the maximum radius, the green mean, and the y mean of the GLCM for 90°features offer the highest classification accuracy obtained, and (iii) a higher number of features sometimes does not improve the classification accuracy and sometimes worsens it.

678
F. Guevara-Hernandez and J. Gomez-Gil / Span J Agric Res (2011) 9(3), 672-680 Table 3. Classification accuracy results obtained regarding the number of features employed, for each feature type selected and for both classifiers employed. The first number in each cell represents the number of misclassified kernels from a total of 545 kernels, and the second number represents the classification correct percentage