Scientia Agricultura Sinica ›› 2020, Vol. 53 ›› Issue (16): 3257-3268.doi: 10.3864/j.issn.0578-1752.2020.16.005

• TILLAGE & CULTIVATION·PHYSIOLOGY & BIOCHEMISTRY·AGRICULTURE INFORMATION TECHNOLOGY • Previous Articles     Next Articles

Research and Development of the Intelligent Identification System of Agricultural Pests for Mobile Terminals

SHAO ZeZhong1(),YAO Qing1(),TANG Jian2(),LI HanQiong3,YANG BaoJun2,LÜ Jun1,CHEN Yi4   

  1. 1School of Information and Technology, Zhejiang Sci-Tech University, Hangzhou 310018
    2State Key Laboratory of Rice Biology, China National Rice Research Institute, Hangzhou 310006
    3Jia Shan County Agricultural and Rural Bureau, Jiaxing 314100, Zhejiang
    4Tongxiang Agricultural Technology Extension Service Center, Tongxiang 314500, Zhejiang
  • Received:2019-10-16 Accepted:2019-12-02 Online:2020-08-16 Published:2020-08-27
  • Contact: Qing YAO,Jian TANG E-mail:812193586@qq.com;q-yao@126.com;tangjian@caas.cn

RichHTML

51

PDF

416

Abstract:

【Objective】There are various species of pests in crop fields, and interspecific similarity and intraspecific difference are common in agricultural pests, which are easy to be confused. In this study, an intelligent system based on mobile terminals was developed to identify agricultural field pests. This system is an easy and intelligent tool of pest identification for peasants and pest forecasting technicians. 【Method】The intelligent identification system of agricultural field pests consists of a mobile client with a system APP, a server and a pest identification model based on deep learning. The application (APP) can be installed in mobile devices with Android system and includes user registration, pest information inquiry, pest automatic identification, pest location information and remote expert identification. The UI interface in this APP uses the style of bottom navigation bar, the information exchange between mobile client and server adopts HTTP protocol and the SDK of Baidu Android map is used to mark the geographic information of pests. The information of users and pests is saved in MySQL database. In the same training and testing sets, different convolutional neural network models were developed to identify agricultural pests. The results showed that the DenseNet121 model achieved the highest precision and lowest false alarm rate. The pest identification model based on DenseNet121 was installed in Alibaba Cloud remote server. When the server received the images from the mobile clients, the identification model was performed. The identification results were feed back to clients from the server. All images and results were saved in database for being traced back in future.【Result】When the users met unidentified pests in crop fields, the users could collect pest images and upload them to the server by the APP installed in mobile clients, such as mobile phone or PAD. The identification results and pest control information would be fed back to the mobile clients in 1-2 seconds. If the results were unsatisfied, the user could ask the expert to remotely identify pests. This system could identify 66 species of pests, and the average precision was 93.9% and false alarm rate was 8.2%. 【Conclusion】The intelligent identification system of agricultural pests could automatically identify the 66 species of agricultural pests. The system could inquire pest information, show the pest geographic information, and ask expert to remotely identify pests. This system is a tool for peasants and pest forecasting technicians to easily and accurately identify agricultural pests in crop fields. It can provide users the one-to-one pest control information and experts needn't go to crop fields for guiding peasants to manually identify pests. This system can save money and time.

Key words: agricultural pests, mobile clients, cloud server, convolutional neural networks, image intelligent identification

Fig. 1

Frame map of the intelligent identification system of agricultural pests for mobile terminals"

Fig. 2

Images of 66 agricultural pests"

Table 1

66 species of common agricultural pests and image number"

序号
ID		 害虫种类
Pest name		 样本总量
Sample size		 序号
ID		 害虫种类
Pest name		 样本总量
Sample size
1 瘤缘蝽 Acanthocoris scaber 49 34 褐边绿刺蛾 Latoia consocia 122
2 鬼脸天蛾 Acherontia lachesis 200 35 大稻缘蝽 Leptocorisa acuta 36
3 苎麻珍蝶 Acraea issoria 278 36 劳氏粘虫 Leucanialoryi 70
4 绿尾大蚕蛾 Actias ningpoana 190 37 东亚飞蝗 Locusta migratoria manilensis 202
5 小地老虎 Agrotis ypsilon 46 38 草地螟 Loxostege stieticatis 65
6 黄地老虎 Agrotis segetum 77 39 甘蓝夜蛾 Mamestra brassicae Linnaeus 115
7 铜绿丽金龟 Anomala corpulenta 115 40 樟叶蜂 Mesonura rufonota 64
8 苎麻夜蛾 Arcte coerula 112 41 草蝉 Mogannia hebes 65
9 绿灰蝶 Artipe 220 42 稻眼蝶 Mycalesis gotama 155
10 二点委夜蛾 Athetis lepigone 129 43 鹰粘夜蛾 Mythimna impura 77
11 枯球箩纹蛾 Brahmaea wallichii 140 44 黑尾叶蝉 Nephotettix bipunctatus 215
12 稻赤斑沫蝉 Callitetix versicolor 60 45 稻绿蝽 Nezara viridula 119
13 桃小食心虫 Carposina sasakii 37 46 褐飞虱 Nilaparvata lugens 236
14 红褐斑腿蝗 Catantops pinguis 150 47 黑翅土白蚁 Odontotermes formosanus 94
15 咖啡透翅天蛾 Cephonodes hylas 223 48 中华稻蝗 Oxya chinensis 112
16 曲纹紫灰蝶 Chilades pandava 260 49 玉带凤蝶 Papilio polytes 212
17 二化螟 Chilo suppressalis 310 50 直纹稻弄蝶 Parnara guttata 170
18 稻蓟马 Chloethrips oryzae 76 51 隐纹谷弄蝶 Pelopidas mathias 120
19 豆天蛾 Clanis bilineata 193 52 苹掌舟蛾 Phalera flavescens 112
20 稻棘缘蝽 Cletus punctiger 60 53 扶桑绵粉蚧 Phenacoccus solenopsis 81
21 长肩棘缘蝽 Cletus trigonus 92 54 菜粉蝶 Pieris rapae 470
22 稻纵卷叶螟 Cnaphalocrocis medinalis 312 55 小菜蛾 Plutella xylostella 80
23 中华草螽 Conocephalus chinensis 99 56 白星花金龟 Protaetia brevitarsis 190
24 稻象甲 Echinocnemus squameus 76 57 紫茎甲 Sagra femorata purpurea 91
25 豆芫菁 Epicauta gorhami 62 58 大螟 Sesamia inferens 144
26 麻皮蝽 Erthesina fullo 74 59 黑额光叶甲 Smaragdina nigrifrons 65
27 宽边黄粉蝶 Eurema hecabe 280 60 白背飞虱 Sogatella furcifera 150
28 硕蝽 Eurostus validus 86 61 甜菜夜蛾 Spodoptera exigua 150
29 菜蝽 Eurydema dominulus 99 62 草地贪夜蛾 Spodoptera frugiperda 66
30 梨小食心虫 Grapholita molesta 56 63 斜纹夜蛾 Spodoptera litura 150
31 蟋蟀 Gryllidae 130 64 红脊长蝽 Tropidothorax elegans 62
32 蝼蛄 Gryllotalpa spps 132 65 大红蛱蝶 Vanessa indica 130
33 棉铃虫 Helicoverpa armigera 108 66 短角外斑腿蝗 Xenocatantops brachycerus 112

Fig. 3

Network architecture of DenseNet"

Fig. 4

Nonlinear transformation structure in Dense Block"

Fig. 5

Loss curves of five CNN models"

Fig. 6

Visualization of pest feature map (a) raw image; (b) Image features after the first convolution; (c) Image features of the first pooling"

Table 2

Identification results of 66 species of agricultural pests by different CNN models"

评价方法Evaluation measures CaffeNet GoogleNet VGG19 ResNet101 DenseNet121
平均识别率Average identification rate (%) 86.2 87.4 88.3 91.7 93.9
平均虚警率Average false alarm rate (%) 16.5 17.1 16.4 11.9 8.2

Fig. 7

Densenet121 accuracy curve"

Fig. 8

Some images from misidentified pests"

Fig. 9

APP interface of agricultural pest identification based on Android mobile phone"

[1] 刘德营. 稻飞虱自动识别关键技术的研究[D]. 南京: 南京农业大学, 2011.
LIU D Y. Study on automatic identification technology for rice plant hoppers[D]. Nanjing: Nanjing Agricultural University, 2011. (in Chinese)
[2] 吴翔. 基于机器视觉的害虫识别方法研究[D]. 杭州: 浙江大学, 2016.
WU X. Study on identification of pests based on machine vision[D]. Hangzhou: Zhejiang University, 2016. (in Chinese)
[3] 杨慧勇, 高灵旺, 牛国飞, 乔建, 刘伟, 邢鲲, 王鑫, 李捷. 农业害虫远程自动识别诊断系统应用技术研究. 山西农业科学, 2010,38(6):40-42.
YANG H Y, GAO L W, NIU G F, QIAO J, LIU W, XING K, WANG X, LI J. Technology research of agricultural pests remote automatic diagnosis system. Journal of Shanxi Agricultural Sciences, 2010,38(6):40-42. (in Chinese)
[4] HAN Y Q, SHAN X, ZHU D. Design and implementation of locust data collecting system based on android. Advances in Computer Science and Education Applications, 2011,2:328-337.
[5] 张谷丰, 罗岗, 孙雪梅. 基于Android的水稻害虫诊断系统. 应用昆虫学报, 2015,4:837-843.
ZHANG G F, LUO G, SUN X M. An android, mobile phone-based identification system for rice insects. Chinese Journal of Applied Entomology, 2015,4:837-843. (in Chinese)
[6] 朱莉, 罗靖, 徐胜勇, 杨勇, 赵海涛, 李卫豪. 基于颜色特征的油菜害虫机器视觉诊断研究. 农机化研究, 2016,38(6):55-58.
ZHU L, LUO J, XU S Y, YANG Y, ZHAO H T, LI W H. Machine vision recognition of rapeseed pests based on color feature. Journal of Agricultural Mechanization Research, 2016,38(6):55-58. (in Chinese)
[7] 王德发. 基于图像识别的储粮害虫检测[D]. 北京: 北京邮电大学, 2017.
WANG D F. Detection of stored grain pests based on image recognition. Beijing: Beijing University of Posts and Telecommunications, 2017. (in Chinese)
[8] WEN C, GUYER D. Image-based orchard insect automated identification and classification method. Computers and Electronics in Agriculture, 2012,89:110-115.
[9] JUAN Z, XIAO P C. Field pest identification by an improved Gabor texture segmentation scheme. New Zealand Journal of Agricultural Research, 2007,50(5):719-723.
doi: 10.1080/00288230709510343
[10] 程鲁玉, 孟小艳, 达新民. 关于果林中果害虫图像特征高效分类识别仿真. 计算机仿真, 2018,35(2):425-428.
CHENG L Y, MENG X Y, DA X M. Simulation of efficient classification for image characteristics of fruit in fruit pest. Computer Simulation, 2018,35(2):425-428. (in Chinese)
[11] WEN C, GUYER D E, LI W. Local feature-based identification and classification for orchard insects. Biosystems Engineering, 2009,104(3):299-307.
[12] LIU T, CHEN W, WU W, SUN C, GUO W, ZHU X. Detection of aphids in wheat fields using a computer vision technique. Biosystems Engineering, 2016,141:82-93.
[13] QING Y, XIAN D, LIU Q, YANG B, DIAO G, TANG J. Automated counting of rice planthoppers in paddy fields based on image processing. Journal of Integrative Agriculture, 2014,13(8):1736-1745.
[14] LARIOS N, DENG H, ZHANG W, SARPOLA M, YUEN J, PAASCH R, MOLDENKE A, LYTLE D, CORREA S, MORTENSEN E, SHAPIRO L, DIETTERICH T. Automated insect identification through concatenated histograms of local appearance features: Feature vector generation and region detection fordeformable objects. Machine Vision and Applications, 2008,19(2):105-123.
[15] 胡永强, 宋良图, 张洁, 谢成军, 李瑞. 基于稀疏表示的多特征融合害虫图像识别. 模式识别与人工智能, 2014,27(11):985-992.
HU Y Q, SONG L T, ZHANG J, XIE C J, LI R. Pest image recognition of multi-feature fusion based on sparse representation. Pattern Recognition and Artificial Intelligence, 2014,27(11):985-992. (in Chinese)
[16] WEN C, WU D, HU H, PAN W. Pose estimation-dependent identification method for field moth images using deep learning architecture. Biosystems Engineering, 2015,136:117-128.
[17] ESPINOZA K, VALERA D L, TORRES J A, LÓPEZ A, MOLINA-AIZ F. Combination of image processing and artificial neural networks as a novel approach for the identification of Bemisia tabaci and Frankliniellaoccidentalis on sticky traps in greenhouse agriculture. Computers and Electronics in Agriculture, 2016,127:495-505.
doi: 10.1016/j.compag.2016.07.008
[18] 周曼, 周明全. 基于BP神经网络的水稻害虫自动识别. 北京师范大学学报(自然科学版), 2008,44(2):165-167.
ZHOU M, ZHOU M Q. Automatic rice pest insects recognition based on bp neural network. Journal of Beijing Normal University(Natural Science), 2008,44(2):165-167. (in Chinese)
[19] XIE C, ZHANG J, LI R, LI J, HONG P, XIA J, CHEN P. Automatic classification for field crop insects via multiple-task sparse representation and multiple-kernel learning. Computers and Electronics in Agriculture, 2015,119:123-132.
doi: 10.1016/j.compag.2015.10.015
[20] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe,America, 2012: 1097-1105.
[21] LIU Z, GAO J, YANG G, ZHANG H, HE Y. Localization and classification of paddy field pests using a saliency map and deep convolutional neural network. Scientific Reports, 2016,6:1-12.
pmid: 28442746
[22] DING W, TAYLOR G. Automatic moth detection from trap images for pest management. Computers & Electronics in Agriculture, 2016,123:17-28.
[23] 陈娟, 陈良勇, 王生生, 赵慧颖, 温长吉. 基于改进残差网络的园林害虫图像识别. 农业机械学报, 2019,50(5):187-195.
CHEN J, CHEN L Y, WANG S S, ZHAO H Y, WEN C J. Pest image recognition of garden based on improved residual network. Transactions of the Chinese Society for Agricultural Machinery, 2019,50(5):187-195. (in Chinese)
[24] CHENG X, ZHANG Y, CHEN Y, WU Y, YUE Y. Pest identification via deep residual learning in complex background. Computers and Electronics in Agriculture, 2017,141:351-356.
doi: 10.1016/j.compag.2017.08.005
[25] XIA D, CHEN P, WANG B, ZHANG J, XIE C. Insect detection and classification based on an improved convolutional neural network. Sensors, 2018,18(12):1-12.
[26] LIU L, WANG R, XIE C, WANG F. PestNet: An end-to-end deep learning approach for large-scale multi-class pest detection and classification. IEEE Access, 2019,7:45301-45312.
[27] ALFARISY A A, CHEN Q, GUO M. Deep learning based classification for paddy pests & diseases recognition. Proceedings of 2018 International Conference on Mathematics and Artificial Intelligence. Chengdu,China, 2018: 21-25.
[28] 沙小梅, 张路正. 识花软件在花卉栽培与室内装饰课程教学中的应用——以形色(赏花识花APP)为例. 课程教育研究, 2018,37:227.
SHAO X M, ZHANG L Z. Application of flower recognition software in flower cultivation and interior decoration course teaching--take xingse(identifying flowers app) for example. Course Education Research, 2018,37:227. (in Chinese)
[29] 王桢, 何建勇. 花伴侣:把“植物大百科”装进小口袋. 绿化与生活, 2017,10:30-31.
WANG Z, HE J Y. Flower companion: Put "encyclopedia of plants" in your pocket. Greening and Life, 2017,10:30-31. (in Chinese)
[30] 樊景超. 基于MobileNets的果园害虫分类识别模型研究. 天津农业科学, 2018,24(9):11-13.
FAN J C. Study on classification and recognition of orchard pests based on mobile nets. Tianjin Agricultural Sciences, 2018,24(9):11-13. (in Chinese)
[31] 赵彬宇, 周慧玲, 李江涛, 严晓平, 韩康榕. 储粮害虫智能图鉴及图像识别APP软件设计. 粮食储藏, 2019,48(3):42-46.
ZHAO B Y, ZHOU H L, LI J T, YAN X P, HAN K R. Mobile app design for stored grain insect illustration guide and image recognition. Grain Storage, 2019,48(3):42-46. (in Chinese)
[32] SHORTEN C, KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019,6(1):1-48.
doi: 10.1186/s40537-018-0162-3
[33] HOU J, SU H Y, YAN B, ZHENG H, SUN Z, CAI X. Classification of tongue color based on CNN. Proceedings of IEEE 2nd International Conference on Big Data Analysis.Beijing,China, 2017: 725-729 .
[34] DENG L M, WANG Y J, HAN Z Z, YU R R. Research on insect pest image detection and recognition based on bio-inspired methods. Biosystems Engineering, 2018,169:139-148.
doi: 10.1016/j.biosystemseng.2018.02.008
[35] SZEGEDY C, LIU W, JIA Y, SERMANET P, REED S, ANGUELOV D, ERHAN D, VANHOUCKE V, RABINOVICH A. Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition, Boston. USA, 2015: 1-9.
[36] WU Z, SHEN C, VAN DEN HENGEL A. Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 2019,90:119-133.
doi: 10.1016/j.patcog.2019.01.006
[37] HUANG G, LIU Z, VAN DER MAATEN L, KILIAN Q. Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Ameriaca, 2017: 4700-4708.
[38] 杨婧, 耿辰, 王海林, 纪建松, 戴亚康. 基于DenseNet的低分辨CT影像肺腺癌组织学亚型分类. 浙江大学学报(工学版), 2019,53(6):1164-1170.
YANG J, GENG C, WANG H L, JI J S, DAI Y K. Classification on histological subtypes of lung adenocarcinoma from low-resolution CT images based on DenseNet. Journal of Zhejiang University (Engineering Science), 2019,53(6):1164-1170. (in Chinese)
[39] ZHU Y, NEWSAM S. Densenet for dense flow. 2017 IEEE International Conference on Image Processing. Beijing,China, 2017: 790-794.
[40] 唐明轩. 基于DenseNet的医学图像分割研究与应用[D]. 成都: 成都信息工程大学, 2018.
TANG M X. Research and application of medical image segmentation based on densenet[D]. Chengdu: ChengDu University of Information Engineering, 2018. (in Chinese)
[41] RUSSAKOVSKY O, DENG J, SU H, KRAUSE J, SATHEESH S, MA S, HUANG Z, KARPATHY S, KHOSLA A, BERNSTEIN M, BERG A, LI F F. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015,115(3):211-252.
doi: 10.1007/s11263-015-0816-y
[42] RAVISHANKAR H, SUDHAKAR P, VENKATARAMANI R, THIRUVENKADAM S, ANNANGI P, BABU N, VAIDYA V. Understanding the mechanisms of deep transfer learning for medical images. Deep Learning and Data Labeling for Medical Applications. Springer, 2016: 188-196.
[43] KE H, CHEN D, LI X, TANG Y, SHAH T, RANJAN R. Towards brain big data classification: Epileptic EEG identification with a lightweight VGGNet on global MIC. IEEE Access, 2018,6:14722-14733.
doi: 10.1109/ACCESS.2018.2810882
[44] SZEGEDY C, VANHOUCKE V, IOFFE S, SHLENS J, WOJNA Z. Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,USA, 2016: 2818-2826.
[45] HE K, ZHANG X, REN S, SUN J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,USA, 2016: 770-778.
[1] LIU QingFei, ZHANG HongLi, WANG YanLing. Real-Time Pixel-Wise Classification of Agricultural Images Based on Depth-Wise Separable Convolution [J]. Scientia Agricultura Sinica, 2018, 51(19): 3673-3682.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备09089781号-30
Copyright 2018 © Editorial office of Scientia Agricultura Sinica
Tel: 86-010-82106279    E-mail: zgnykx@mail.caas.net.cn
Supported by Beijing Magtech Co.Ltd