Scientia Agricultura Sinica ›› 2020, Vol. 53 ›› Issue (12): 2360-2370.doi: 10.3864/j.issn.0578-1752.2020.12.005

• TILLAGE & CULTIVATION·PHYSIOLOGY & BIOCHEMISTRY·AGRICULTURE INFORMATION TECHNOLOGY • Previous Articles     Next Articles

Surface Defect Detection of Navel Orange Based on Region Adaptive Brightness Correction Algorithm

ZHANG Ming1,2,WANG Teng2,LI Peng2,DENG Lie2,ZHENG YongQiang2,YI ShiLai2,LÜ Qiang2(),SUN RongRong2()   

  1. 1 College of Engineering and Technology, Southwest University, Chongqing 400715;
    2 Citrus Research Institute, Southwest University/Citrus Research Institute, Chinese Academy of Agricultural Sciences, Chongqing 400712
  • Received:2019-09-23 Online:2020-06-16 Published:2020-06-25
  • Contact: Qiang Lü,RongRong SUN E-mail:qlu@swu.edu.cn;sunrongrong@cric.cn

Abstract:

【Objective】 Because of uneven lightness distribution on navel orange surface due to its geometric shape of sphere to oval, it is difficult to segment directly the defect area from the orange image by single threshold method. In order to solve this problem, a novel detection method of the surface defects of navel orange based on the region brightness adaptive correction algorithm was proposed in this study. 【Method】 The Newhall navel orange with high economic value was selected as the research object, and its original image was collected using self-developed image acquisition system. During this experiment, it was found that the gray level of R-B fused component image had obvious bimodal distribution, and the color contrast between the background and the foreground of the original image was relatively large. Therefore, the background was removed using the mask generated by R-B fused component image and the single threshold method. The R-B target image was extracted with the background removed, which could effectively remove the background without damaging the navel orange surface. Then, the brightness information of the target image was calculated based on the region adaptive brightness correction algorithm proposed in this work, and the brightness information was estimated according to the set of brighter pixels in the neighborhood of the points on the target image. In this study, the neighborhood window size was set to w×w, and the value of w was set to 13 by experiment comparison. The pixels in the neighborhood were moved on the target image pixel by pixel, and a new matrix was generated. The new matrix was sorted in ascending order in the column, then the last row was removed to exclude the influence of white noise, and the last five rows were selected and the matrix was obtained by means of column operation. The matrix was inversely transformed and stored. Finally, the matrix image with the same size of the target image, that was, the surface brightness information of the target image was obtained. Based on the obtained luminance information, the R-B fused component image after background removal was corrected by luminance homogenization. After brightness correction, the gray contrast between the defect area and the normal tissue area of the image was relatively large. The single threshold method (segmentation threshold T=194) could be used to extract the surface defect directly from the brightness corrected navel orange image. Finally, the area filter was applied to the binary image after surface defect segmentation to remove the spurious points and noise. 【Result】 The bimodal method could effectively remove the background of the image and preserve the surface information of the target navel orange. All the sample images were also corrected and inspected by using other brightness correction algorithms, including histogram equalization algorithm, Retinex theory algorithm and illumination-reflection theory algorithm. The experimental results showed that the algorithm proposed in this study could correct both the middle and the edge of the normal region of the image, which were to be highlighted and the fluctuation range of the gray scale difference was small, while the defect regions still exhibited low gray level. The average processing time of each image was 0.29 seconds in proposed algorithm. Compared with the other three algorithms, the average processing time of each image could be reduced 0.27 s, 0.14 s, and 1.45 s by proposed algorithm. Finally, the algorithm proposed in this study combined with single threshold method, and it was used to detect the defects of 356 Newhall navel orange samples. Based on this algorithm, 8 kinds of common navel orange surface defects (e.g., canker, thrips, scale infestation, insect injury, blackspot, wind damage, anthracnose, dehiscent) were detected, and the defect recognition rate reached 95.8%. The correct defect recognition rate was improved by 2.6%-8.2% which compared with other three algorithms. 【Conclusion】 The proposed algorithm effectively solved the problem of defect segmentation caused by uneven brightness distribution on the surface of spherical fruits, provided technical support for online accurate classification of navel orange fruits, and provided a new idea for rapid detection of surface defects for other fruits.

Key words: spherical fruit, navel orange, brightness correction, threshold segmentation, defect detection

Fig. 1

Image background removal"

Fig. 2

Brightness correction and defect extraction"

Fig. 3

Processing results of different navel oranges surface defects Original RGB images (1st and 5th rows), R-B difference component image (2nd and 6th), brightness corrected images (3rd and 7th rows), and defect region images (4th and 8th rows)"

Table 1

Detection result of navel orange surface defect based on region brightness adaptive correction"

表皮类型
Epidermis type
样本数
Number of samples
正确识别样本数
Number of correctly recognized samples
误判样本数
Number of misjudged samples
准确率
Detection rate (%)
误判率
False rate (%)
正常果Normal fruit 64 59 5 92.2 7.8
溃疡病果Canker spot 29 29 0 100.0 0.0
蓟马虫果Thrips scarring 44 42 2 95.5 4.5
介壳虫果Scale infestation 37 36 1 97.3 2.7
虫伤果Insect injury 38 36 2 94.7 5.3
黑星病果Blackspot fruit 35 34 1 97.1 2.9
风伤果Wind scarring fruit 61 57 4 93.4 6.6
炭疽病果Anthracnose fruit 26 26 0 100.0 0.0
裂伤果Dehiscent fruit 22 22 0 100.0 0.0
合计Total 356 341 15 95.8 4.2

Table 2

Effect of neighbor window size on processing speed and defect recognition rate"

邻域窗口大小
w
平均耗时
Average time consuming (s)
识别率
Recognition rate (%)
7 0.11 92.1
9 0.16 92.9
11 0.21 93.5
13 0.29 95.8
15 0.45 95.9
17 0.72 96.1

Fig. 4

Results of brightness correction based on different algorithms"

Fig. 5

Gray-scale curve of 235 columns of image corrected by various algorithms"

Fig. 6

Intensity energy distribution of navel orange surface"

Table 3

Processing time of each algorithm and defect recognition rate"

算法
Algorithm
样本数
Number of samples
识别样本数
Number of recognized samples
缺陷识别率
Defect recognition rate (%)
平均耗时
Average time consuming (s)
HE算法HE algorithm 356 312 87.6 0.43
Retinex算法Retinex algorithm 323 90.7 0.56
照度-反射理论Illumination-reflection theory 332 93.3 1.74
本文算法Proposed algorithm 341 95.8 0.29
[1] 郭峰. 基于机器视觉的水果品质无损检测与分析方法研究[D]. 上海: 上海交通大学, 2007.
GUO F. Study of machine vision based no-contact inspect and analysis method for fruit quality[D]. Shanghai: Shanghai Jiao Tong University, 2007. (in Chinese)
[2] ZHANG B, HUANG W, GONG L, LI J B, ZHAO C J, LIU C L. Computer vision detection of defective apples using automatic lightness correction and weighted RVM classifier. Journal of Food Engineering, 2015,146:143-151.
doi: 10.1016/j.jfoodeng.2014.08.024
[3] IQBAL S M, GOPAL A, SANKARANARAYANAN P E, NAIR A B. Classification of selected citrus fruits based on color using machine vision system. International Journal of Food Properties, 2016,19(2):272-288.
doi: 10.1080/10942912.2015.1020439
[4] LÓPEZ-GARCÍA F, ANDREU-GARCÍA G, BLASCO J, ALEIXOS N, VALIENTE J M. Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach. Computers and Electronics in Agriculture, 2010,71(2):189-197.
doi: 10.1016/j.compag.2010.02.001
[5] 白雪冰, 宋恩来, 李润佳, 许景涛. 柑橘表面缺陷图像快速准确分割方法. 沈阳农业大学学报, 2018,49(2):242-249.
BAI X B, SONG E L, LI R J, XU J T. Fast and accurate segmentation method for surface defects of citrus. Journal of Shenyang Agricultural University, 2018,49(2):242-249. (in Chinese)
[6] GÓMEZ-SANCHIS J, MOLTÓ E, CAMPS-VALLS G, GÓMEZ- CHOVA L, ALEIXOS N, BLASCO J. Automatic correction of the effects of the light source on spherical objects. An application to the analysis of hyperspectral images of citrus fruits. Journal of Food Engineering, 2008,85(2):191-200.
doi: 10.1016/j.jfoodeng.2007.06.036
[7] NIPHADKAR N P, BURKS T F, JIAN W Q, RITENOUR M. Edge effect compensation for citrus canker lesion detection due to light source variation - a hyperspectral imaging application. Agricultural Engineering International Cigr Journal, 2013,15(4):314-327.
[8] LI J B, RAO X Q, WANG F, WU W, YING Y B. Automatic detection of common surface defects on oranges using combined lighting transform and image ratio methods. Postharvest Biology and Technology, 2013,82:59-69.
doi: 10.1016/j.postharvbio.2013.02.016
[9] RONG D, RAO X Q, YING Y B. Computer vision detection of surface defect on oranges by means of a sliding comparison window local segmentation algorithm. Computers & Electronics in Agriculture, 2017,137:59-68.
[10] RONG D, YING Y B, RAO X Q. Embedded vision detection of defective orange by fast adaptive lightness correction algorithm. Computers & Electronics in Agriculture, 2017,138:48-59.
[11] TAO Y, WEN Z. An adaptive spherical image transform for high- speed fruit defect detection. Transactions of the ASME, 1999,42(1):241-246.
[12] 冯斌, 汪懋华. 计算机视觉技术识别水果缺陷的一种新方法. 中国农业大学学报, 2002,7(4):73-76.
FENG B, WANG M H. Study on identifying measurement about default of fruit in computer vision. Journal of China Agricultural University, 2002,7(4):73-76. (in Chinese)
[13] RAJAVEl P. Image dependent brightness preserving histogram equalization. IEEE Transactions on Consumer Electronics, 2010,56(2):756-763.
doi: 10.1109/TCE.2010.5505998
[14] TAN T L, SIM K S, TSO C P. Image enhancement using background brightness preserving histogram equalisation. Electronics Letters, 2012,48(3):155-157.
doi: 10.1049/el.2011.3740
[15] JIA L. Application of image enhancement method for digital images based on Retinex theory. Optik - International Journal for Light and Electron Optics, 2013,124(23):5986-5988.
doi: 10.1016/j.ijleo.2013.04.115
[16] SHUKRI D S, ASMUNI H, OTHMAN R M, HASSAN R. An improved multiscale retinex algorithm for motion-blurred iris images to minimize the intra-individual variations. Pattern Recognition Letters, 2013,34(9):1071-1077.
doi: 10.1016/j.patrec.2013.02.017
[17] 李江波, 饶秀勤, 应义斌. 基于照度-反射模型的脐橙表面缺陷检测. 农业工程学报, 2011,27(7):338-342.
LI J B, RAO X Q, YING Y B. Detection of navel surface defects based on illumination-reflectance model. Transactions of the Chinese Society of Agricultural Engineering, 2011,27(7):338-342. (in Chinese)
[18] LEE P H, WU S W, HUNG Y P. Illumination compensation using oriented local histogram equalization and its application to face recognition. IEEE Transactions on Image Processing, 2012,21(9):4280-4289.
doi: 10.1109/TIP.2012.2202670
[19] CHEOLKON J, TIAN S, LICHENG J. Eye detection under varying illumination using the retinex theory. Neurocomputing, 2013,113(596):130-137.
doi: 10.1016/j.neucom.2013.01.038
[20] WANG S H, ZHANG J, HU H M, LI B. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Image Processing, 2013,22(9):3538-3548.
doi: 10.1109/TIP.2013.2261309
[21] 王殿伟, 王晶, 许志杰, 刘颖. 一种光照不均匀图像的自适应校正算法. 系统工程与电子技术, 2017,39(6):1383-1390.
WANG D W, WANG J, XU Z J, LIU Y. Adaptive correction algorithm for non-uniform illumination images. Systems Engineering and Electronics, 2017,39(6):1383-1390. (in Chinese)
[22] 刘志成, 王殿伟, 刘颖, 刘学杰. 基于二维伽马函数的光照不均匀图像自适应校正算法. 北京理工大学学报, 2016,36(2):191-196.
LIU Z C, WANG D W, LIU Y, LIU X J. Adaptive adjustment algorithm for non-uniform illumination images based on 2D gamma function. Transaction of Beijing Institute of Technology, 2016,36(2):191-196. (in Chinese)
[23] 李江波. 脐橙表面缺陷的快速检测方法研究[D]. 杭州: 浙江大学, 2012.
LI J B. Study on rapid detection methods of defects on navel orange surface[D]. Hangzhou: Zhejiang University, 2012. (in Chinese)
[24] 王干, 孙力, 李雪梅, 张明, 吕强, 蔡建荣. 基于机器视觉的脐橙采后田间分级系统设计. 江苏大学学报(自然科学版)>, 2017(6):57-61.
WANG G, SUN L, LI X L, ZHANG M, LÜ Q, CAI J R. Design of postharvest in-field grading system for navel orange based on machine vision. Journal of Jiangsu University (Natural Science Edition), 2017(6):57-61. (in Chinese)
[25] 庞江伟. 基于计算机视觉的脐橙表面常见缺陷种类识别的研究[D]. 杭州: 浙江大学, 2006.
PANG J W. Study on external defects classification of navel orange based on machine vision[D]. Hangzhou: Zhejiang University, 2006. (in Chinese)
[26] 饶秀勤. 基于机器视觉的水果品质实时检测与分级生产线的关键技术研究[D]. 杭州: 浙江大学, 2007.
RAO X Q. Real-time inspection technology of fruit quality using machine vision[D]. Hangzhou: Zhejiang University, 2007. (in Chinese)
[27] CHACON-MURGUIA M I, RAMIREZ-QUINTANA J, URIAS-ZAVALA D. Segmentation of video background regions based on a DTCNN- clustering approach. Signal, Image and Video Processing, 2015,9(1):135-144.
doi: 10.1007/s11760-014-0718-4
[28] 应义斌. 水果图像的背景分割和边缘检测技术研究. 浙江大学学报, 2000,26(1):35-38.
YING Y B. Study on background segment and edge detection of fruit image using machine vision. Journal of Zhejiang University, 2000,26(1):35-38. (in Chinese)
[29] 李江波, 黄文倩, 张保华, 彭彦昆, 赵春江. 类球形水果表皮颜色变化校正方法研究. 农业机械学报, 2014,45(4):226-230.
LI J B, HUANG W Q, ZHANG B H, PENG Y K, ZHAO C J. Correction algorithm of lighting non-uniformity on spherical fruit. Transactions of the Chinese Society for Agricultural Machinery, 2014,45(4):226-230. (in Chinese)
[30] 张明, 李鹏, 邓烈, 何绍兰, 易时来, 郑永强, 谢让金, 马岩岩, 吕强. 基于掩模及亮度校正算法的脐橙表面缺陷分割. 中国农业科学, 2019,52(2):327-338.
doi: 10.3864/j.issn.0578-1752.2019.02.011
ZHANG M, LI P, DENG L, HE S L, YI S L, ZHENG Y Q, XIE R J, MA Y Y, LÜ Q. Segmentation of navel orange surface defects based on mask and brightness correction algorithm. Scientia Agricultura Sinica, 2019,52(2):327-338. (in Chinese)
doi: 10.3864/j.issn.0578-1752.2019.02.011
[31] 李江波, 饶秀勤, 应义斌, 马本学, 郭俊先. 基于掩模及边缘灰度补偿算法的脐橙背景及表面缺陷分割. 农业工程学报, 2009,25(12):133-137.
LI J B, RAO X Q, YING Y B, MA B X, GUO J X. Background and external defects segmentation of navel orange based on mask and edge gray value compensation algorithm. Transactions of the Chinese Society of Agricultural Engineering, 2009,25(12):133-137. (in Chinese)
[1] CHEN XueSen, YIN HuaLin, WANG Nan, ZHANG Min, JIANG ShengHui, XU Juan, MAO ZhiQuan, ZHANG ZongYing, WANG ZhiGang, JIANG ZhaoTao, XU YueHua, LI JianMing. Interpretation of the Case of Bud Sports Selection to Promote the High-Quality and Efficient Development of the World’s Apple and Citrus Industry [J]. Scientia Agricultura Sinica, 2022, 55(4): 755-768.
[2] ZHANG Ji,LI JunJie,WAN LianJie,YANG JiangBo,ZHENG YongQiang,LÜ Qiang,XIE RangJin,MA YanYan,DENG Lie,YI ShiLai. Effects of Potassium Application Levels on Nutrient, Yield and Quality of Newhall Navel Orange [J]. Scientia Agricultura Sinica, 2020, 53(20): 4271-4286.
[3] ZHANG Ming,LI Peng,DENG Lie,HE ShaoLan,YI ShiLai,ZHENG YongQiang,XIE RangJin,MA YanYan,LÜ Qiang. Segmentation of Navel Orange Surface Defects Based on Mask and Brightness Correction Algorithm [J]. Scientia Agricultura Sinica, 2019, 52(2): 327-338.
[4] LI WenTao, YANG JiangBo, ZHANG Ji, WANG KeJian, DENG Lie, Lü Qiang, HE ShaoLan, XIE RangJin, ZHENG YongQiang, MA YanYan, YI ShiLai. The Evaluation of Chlorophyll Content Detection in the Leaves of Newhall Navel Orange Based on Different Sensors [J]. Scientia Agricultura Sinica, 2018, 51(6): 1057-1066.
[5] ZHENG YongQiang, WANG Ya, YANG Qiong, JIA XueMei, HE ShaoLan, DENG Lie, XIE RangJin, YI ShiLai, Lü Qiang, MA YanYan . Leaf Nutritional Diagnosis of Powell Navel Orange at Flowering Stage in Chongqing Three Gorges Reservoir Area [J]. Scientia Agricultura Sinica, 2018, 51(12): 2378-2390.
[6] LIAO Qiu-hong, HE Shao-lan, XIE Rang-jin, QIAN Chun, HU De-yu, Lü Qiang1,YI Shi-lai, ZHENG Yong-qiang, DENG Lie. Study on Producing Area Classification of Newhall Navel Orange Based on the Near Infrared Spectroscopy [J]. Scientia Agricultura Sinica, 2015, 48(20): 4111-4119.
[7] FAN Wei-guo, GE Hui-min. Effects of Nitrogen Fertilizer of Different Forms and Ratios on the Growth, Nitrogen Absorption and Utilization of Young Navel Orange Trees Grafted on Poncirus trifoliata [J]. Scientia Agricultura Sinica, 2015, 48(13): 2666-2675.
[8] HE Yi-Zhong-1, CHEN Zhao-Xing-1, 2 , LIU Run-Sheng-1, FANG Yi-Wen-2, GU Zu-Liang-3, YAN Xiang-2, CHEN Hong-3, ZHANG Hong-Ming-2, TANG Huan-Qing-3, CHENG Yun-Jiang-1. Effects of Different Storage Methods on Fruit Quality of ‘Newhall’ Navel Orange (Citrus sinensis Osbeck‘Newhall’) in Southern Jiangxi Province [J]. Scientia Agricultura Sinica, 2014, 47(4): 736-748.
[9] LIU Shi-Yao. Construction and Application of Flavonoids Characteristic Constituents HPLC Fingerprints of Fengjie Navel Orange (Citrus sinensis Osbeck) [J]. Scientia Agricultura Sinica, 2013, 46(19): 4131-4148.
[10] . Anatomical Structures of Oleocellosis of Navel Orange Peel
[J]. Scientia Agricultura Sinica, 2011, 44(6): 1301-1306 .
[11] LIU Gui-dong,JIANG Cun-cang,WANG Yun-hua,PENG Shu-ang,ZENG Qing-luan
. Effect of Boron on Leaf Boron Forms of Newhall Navel Orange Grafted on Two Rootstocks
[J]. Scientia Agricultura Sinica, 2011, 44(5): 982-989 .
[12] DENG Yu-yan,MING Jian,ZHANG Zhao-qi,ZENG Kai-fang
. Effect of Chitosan on Salicylic Acid and Active Oxygen Metabolism of Navel Orange Fruit
[J]. Scientia Agricultura Sinica, 2010, 43(4): 812-820 .
[13]

LING Li-li,PENG Liang-zhi,CHUN Chang-pin,CAO Li,JIANG Cai-lun,LEI Ting

. Relationship Between Leaf Yellowing Degree and Nutrimental Elements Levels in Navel Orange (Citrus sinensis Osbeck) Leaves in Southern Jiangxi Province of China
[J]. Scientia Agricultura Sinica, 2010, 43(17): 3602-3607 .
[14] ,. Analysis of DNA Methylation in Navel Oranges Based on MSAP Marker [J]. Scientia Agricultura Sinica, 2005, 38(11): 2301-2307 .
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!