Segmentation of foreground apple targets by fusing visual attention mechanism and growth rules of seed points

Weifeng Qu, Wenjing Shang, Yanhua Shao, Dandan Wang, Xiuli Yu, Huaibo Song


Accurate segmentation of apple targets is one of the most important problems to be solved in the vision system of apple picking robots. This work aimed to solve the difficulties that background targets often bring to foreground targets segmentation, by fusing the visual attention mechanism and the growth rule of seed points. Background targets could be eliminated by extracting the ROI (region of interest) of apple targets; the ROI was roughly segmented on the HSV color space, and then each of the pixels was used as a seed growing point. The growth rule of the seed points was adopted to obtain the whole area of apple targets from seed growing points. The proposed method was tested with 20 images captured in a natural scene, including 54 foreground apple targets and approximately 84 background apple targets. Experimental results showed that the proposed method can remove background targets and focus on foreground targets, while the k-means algorithm and the chromatic aberration algorithm cannot. Additionally, its average segmentation error rate was 13.23%, which is 2.71% higher than that of the k-means algorithm and 2.95% lower than that of the chromatic aberration algorithm. In conclusion, the proposed method contributes to the vision system of apple-picking robots to locate foreground apple targets quickly and accurately under a natural scene


picking robots; Malus domestica; ROI; Itti model

Full Text:



Cheng X, Shi X, 2013. Target extraction study on the vision system of apple picking robot. Proc Chinese Intelligent Automation Conference. Springer Berlin Heidelberg, pp: 45-52.

Dandapat S, Chutatape O, Krishnan SM, 2004. Perceptual model based data embedding in medical images. Image Processing, Int Conf IEEE ICIP'04, Vol 4, pp: 2315-2318.

Deng Y, Manjunath BS, 2001. Unsupervised segmentation of color-texture regions in images and video. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(8): 800-810.

Harel J, Koch C, Perona P, 2006. Graph-based visual saliency. Advances in Neural Information Processing Systems, Vancouver, BC, Canada. pp: 545-552. Available in:

Itti L, Koch C, Niebur E, 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis & Machine Intelligence (11): 1254-1259.

Ji W, Zhao D, Cheng F, Xu B, Zhang Y, Wang J, 2012. Automatic recognition vision system guided for apple harvesting robot. Comput Electr Eng 38(5): 1186-1195.

Jiang H, Peng Y, Chen C, Ying Y, 2008. Recognizing and locating ripe tomatoes based on binocular stereovision technology. T CSAE 24(8): 279-283.

Jung C, Kim C, 2012. A unified spectral-domain approach for saliency detection and its application to automatic object segmentation. IEEE Transactions on Image Processing 21(3): 1272-1283.

Li B, Wang N, Wang M, Li L, 2010. In-field pineapple recognition based on monocular vision. T CSAE 26(10): 345-349.

Linker R, Cohen O, Naor A, 2012. Determination of the number of green apples in RGB images recorded in orchards. Comput Electron Agr 81: 45-57.

Lv J, Ji W, Chen F, Zhao D, Xu B, 2012. Research on the recognition method for obscured apple in natural environment. IEEE Control Conference (CCC), 31st Chinese. pp: 3932-3937. Available in:

Peng H, Wen Y, Wu L, 2011. Citrus edge detection based on self-adaptive canny operator. Comput Eng Appl 47(9): 163-166.

Rumelhart DE, Zipser D, 1985. Feature discovery by competitive learning. Cognitive Science 9(1): 75-112.

Si Y, Qiao J, Liu G, Liu Z, Gao R, 2009. Recognition and shape features extraction of apples based on machine vision. T CSAE 40: 161-165, 73.

Si Y, Liu G, Feng J, 2015. Location of apples in trees using stereoscopic vision. Comput Electron Agr 112: 68-74.

Song H, He D, Pan J, 2012. Recognition and localization methods of occluded apples based on convex hull theory. T CSAE 28(22): 174-180.

Song H, Zhang C, Pan J, Yin, X, Zhuang Y, 2013. Segmentation and reconstruction of overlapped apple images based on convex hull. T CSAE 29(3): 163-168.

Tu J, Liu C, Li Y, Zhou J, Yuan J, 2010. Apple recognition method based on illumination invariant graph. T CSAE 26 (Suppl. 2): 26-31.

Wang J, Zhao D, Ji W, Zhang C, 2009. Apple fruit recognition based on support vector machine using in harvesting robot. T CSAM 40(1): 148-151.

Wang X, Wang B, Zhang L, 2011. Airport detection in remote sensing images based on visual attention. Neural Information Processing. Springer Berlin Heidelberg, pp: 475-484.

Wang D, Song H, Tie Z, Zhang W, He D, 2015. Recognition and localization of occluded apples using K-means clustering algorithm and convex hull theory: a comparison. Multimedia Tools and Applications, 1-22.

Xu H, Ye Z, Ying Y, 2005. Identification of citrus fruit in a tree canopy using color information. T CSAE 5: 023.

Zhang P, Wang R, 2005. A survey of detecting regions of interest in a static image. J Image Graph 10(2): 142-148. Available in:

Zhang Y, Li M, Qiao J, Liu G, 2008. Segmentation algorithm for apple recognition using image features and artificial neural network. Acta Optica Sinica 28(11): 2104-2108.

Zhang J, Shen L, Gao J, 2009. Region of interest detection based on visual attention model and evolutionary programming. J Electron Inf Technol 31(7): 1646-1652.

Zhao D, Lv J, Ji W, Zhang Y, Chen Y, 2011. Design and control of an apple harvesting robot. Biosyst Eng 110(2): 112-122.

Zhou T, Zhang T, Yang L, Zhao J, 2007. Comparison of two algorithms based on mathematical morphology for segmentation of touching strawberry fruits. T CSAE 23(9): 164-168.

DOI: 10.5424/sjar/2015133-7047