面向移动终端的农业害虫图像智能识别系统的研究与开发

doi:10.3864/j.issn.0578-1752.2020.16.005

中国农业科学 ›› 2020, Vol. 53 ›› Issue (16): 3257-3268.doi: 10.3864/j.issn.0578-1752.2020.16.005

• 耕作栽培·生理生化·农业信息技术 • 上一篇下一篇

面向移动终端的农业害虫图像智能识别系统的研究与开发

邵泽中¹(),姚青¹(),唐健²(),李罕琼³,杨保军²,吕军¹,陈轶⁴

¹浙江理工大学信息学院,杭州 310018
²中国水稻研究所/水稻生物学国家重点实验室,杭州 310006
³嘉善县农业农村局,浙江嘉兴 314100
⁴桐乡市农业技术推广服务中心,浙江桐乡 314500

收稿日期:2019-10-16 接受日期:2019-12-02 出版日期:2020-08-16 发布日期:2020-08-27
通讯作者: 姚青,唐健
作者简介:邵泽中,E-mail：812193586@qq.com
基金资助:
浙江省公益性项目(LGN18C140007);国家“863”计划(2013AA102402)

Research and Development of the Intelligent Identification System of Agricultural Pests for Mobile Terminals

SHAO ZeZhong¹(),YAO Qing¹(),TANG Jian²(),LI HanQiong³,YANG BaoJun²,LÜ Jun¹,CHEN Yi⁴

¹School of Information and Technology, Zhejiang Sci-Tech University, Hangzhou 310018
²State Key Laboratory of Rice Biology, China National Rice Research Institute, Hangzhou 310006
³Jia Shan County Agricultural and Rural Bureau, Jiaxing 314100, Zhejiang
⁴Tongxiang Agricultural Technology Extension Service Center, Tongxiang 314500, Zhejiang

Received:2019-10-16 Accepted:2019-12-02 Online:2020-08-16 Published:2020-08-27
Contact: Qing YAO,Jian TANG

摘要/Abstract

摘要：

【目的】农作物田间害虫种类繁多,存在种间相似和种内差异的现象,容易混淆。本研究开发一个面向移动终端的农业害虫图像智能识别系统,为广大农户和基层测报人员提供一个便捷准确的农业害虫智能识别工具。【方法】农业害虫图像智能识别系统包括装有系统APP的移动客户端、服务器和基于深度学习的农业害虫识别模型。APP是在Android环境下开发的,可安装于Android系统的移动设备中。APP包括登录模块、害虫信息查询模块、害虫智能识别模块、害虫地图标记模块和害虫专家远程鉴定模块,UI界面采用底部导航栏形式。移动终端与服务器间的信息交互采用HTTP协议,害虫采集地信息显示使用百度的Android地图SDK来实现,用户和害虫信息使用MySQL数据库进行保存。在相同训练集和测试集条件下,比较了不同深度卷积神经网络模型,筛选出基于DenseNet121的农业害虫识别模型具有最高的精准度和最低的虚警率。农业害虫识别模型的程序部署在阿里云远程服务器上,当服务器端接收到移动客户端上传的害虫图像时,运行害虫识别模型,识别结果通过服务器反馈给客户端,同时将上传的图像和识别结果保存在数据库中,便于害虫图像的追溯。【结果】当用户在农田遇到不认识的害虫时,可通过装有该系统APP的移动设备（如手机或平板）拍摄害虫图像,并上传到服务器,识别结果和害虫防治信息在1—2 s内反馈至用户移动终端的屏幕上,对识别结果不满意还可远程请求专家鉴定。该系统对66种常见农业害虫图像平均识别率为93.9%,平均虚警率为8.2%。【结论】面向移动终端的农业害虫图像智能识别系统实现了66种常见农业害虫信息查询、自动识别,害虫采集地的地图显示和专家远程鉴定等功能。为农民和基层测报人员提供了一个农业害虫便捷准确的自动识别工具,无需专家到田间即可实现了用户“一对一”的防治指导,大大节省了经济和时间成本。

关键词: 农业害虫, 移动终端, 云服务器, 卷积神经网络, 图像智能识别

Abstract:

【Objective】There are various species of pests in crop fields, and interspecific similarity and intraspecific difference are common in agricultural pests, which are easy to be confused. In this study, an intelligent system based on mobile terminals was developed to identify agricultural field pests. This system is an easy and intelligent tool of pest identification for peasants and pest forecasting technicians. 【Method】The intelligent identification system of agricultural field pests consists of a mobile client with a system APP, a server and a pest identification model based on deep learning. The application (APP) can be installed in mobile devices with Android system and includes user registration, pest information inquiry, pest automatic identification, pest location information and remote expert identification. The UI interface in this APP uses the style of bottom navigation bar, the information exchange between mobile client and server adopts HTTP protocol and the SDK of Baidu Android map is used to mark the geographic information of pests. The information of users and pests is saved in MySQL database. In the same training and testing sets, different convolutional neural network models were developed to identify agricultural pests. The results showed that the DenseNet121 model achieved the highest precision and lowest false alarm rate. The pest identification model based on DenseNet121 was installed in Alibaba Cloud remote server. When the server received the images from the mobile clients, the identification model was performed. The identification results were feed back to clients from the server. All images and results were saved in database for being traced back in future.【Result】When the users met unidentified pests in crop fields, the users could collect pest images and upload them to the server by the APP installed in mobile clients, such as mobile phone or PAD. The identification results and pest control information would be fed back to the mobile clients in 1-2 seconds. If the results were unsatisfied, the user could ask the expert to remotely identify pests. This system could identify 66 species of pests, and the average precision was 93.9% and false alarm rate was 8.2%. 【Conclusion】The intelligent identification system of agricultural pests could automatically identify the 66 species of agricultural pests. The system could inquire pest information, show the pest geographic information, and ask expert to remotely identify pests. This system is a tool for peasants and pest forecasting technicians to easily and accurately identify agricultural pests in crop fields. It can provide users the one-to-one pest control information and experts needn't go to crop fields for guiding peasants to manually identify pests. This system can save money and time.

Key words: agricultural pests, mobile clients, cloud server, convolutional neural networks, image intelligent identification

邵泽中,姚青,唐健,李罕琼,杨保军,吕军,陈轶. 面向移动终端的农业害虫图像智能识别系统的研究与开发[J]. 中国农业科学, 2020, 53(16): 3257-3268.

SHAO ZeZhong,YAO Qing,TANG Jian,LI HanQiong,YANG BaoJun,LÜ Jun,CHEN Yi. Research and Development of the Intelligent Identification System of Agricultural Pests for Mobile Terminals[J]. Scientia Agricultura Sinica, 2020, 53(16): 3257-3268.

0
/ / 推荐

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: https://www.chinaagrisci.com/CN/10.3864/j.issn.0578-1752.2020.16.005

https://www.chinaagrisci.com/CN/Y2020/V53/I16/3257

图/表 11

图1

图2

表1

66种常见农业害虫种类与图像样本量"

序号 ID	害虫种类 Pest name	样本总量 Sample size	序号 ID	害虫种类 Pest name	样本总量 Sample size
1	瘤缘蝽 Acanthocoris scaber	49	34	褐边绿刺蛾 Latoia consocia	122
2	鬼脸天蛾 Acherontia lachesis	200	35	大稻缘蝽 Leptocorisa acuta	36
3	苎麻珍蝶 Acraea issoria	278	36	劳氏粘虫 Leucanialoryi	70
4	绿尾大蚕蛾 Actias ningpoana	190	37	东亚飞蝗 Locusta migratoria manilensis	202
5	小地老虎 Agrotis ypsilon	46	38	草地螟 Loxostege stieticatis	65
6	黄地老虎 Agrotis segetum	77	39	甘蓝夜蛾 Mamestra brassicae Linnaeus	115
7	铜绿丽金龟 Anomala corpulenta	115	40	樟叶蜂 Mesonura rufonota	64
8	苎麻夜蛾 Arcte coerula	112	41	草蝉 Mogannia hebes	65
9	绿灰蝶 Artipe	220	42	稻眼蝶 Mycalesis gotama	155
10	二点委夜蛾 Athetis lepigone	129	43	鹰粘夜蛾 Mythimna impura	77
11	枯球箩纹蛾 Brahmaea wallichii	140	44	黑尾叶蝉 Nephotettix bipunctatus	215
12	稻赤斑沫蝉 Callitetix versicolor	60	45	稻绿蝽 Nezara viridula	119
13	桃小食心虫 Carposina sasakii	37	46	褐飞虱 Nilaparvata lugens	236
14	红褐斑腿蝗 Catantops pinguis	150	47	黑翅土白蚁 Odontotermes formosanus	94
15	咖啡透翅天蛾 Cephonodes hylas	223	48	中华稻蝗 Oxya chinensis	112
16	曲纹紫灰蝶 Chilades pandava	260	49	玉带凤蝶 Papilio polytes	212
17	二化螟 Chilo suppressalis	310	50	直纹稻弄蝶 Parnara guttata	170
18	稻蓟马 Chloethrips oryzae	76	51	隐纹谷弄蝶 Pelopidas mathias	120
19	豆天蛾 Clanis bilineata	193	52	苹掌舟蛾 Phalera flavescens	112
20	稻棘缘蝽 Cletus punctiger	60	53	扶桑绵粉蚧 Phenacoccus solenopsis	81
21	长肩棘缘蝽 Cletus trigonus	92	54	菜粉蝶 Pieris rapae	470
22	稻纵卷叶螟 Cnaphalocrocis medinalis	312	55	小菜蛾 Plutella xylostella	80
23	中华草螽 Conocephalus chinensis	99	56	白星花金龟 Protaetia brevitarsis	190
24	稻象甲 Echinocnemus squameus	76	57	紫茎甲 Sagra femorata purpurea	91
25	豆芫菁 Epicauta gorhami	62	58	大螟 Sesamia inferens	144
26	麻皮蝽 Erthesina fullo	74	59	黑额光叶甲 Smaragdina nigrifrons	65
27	宽边黄粉蝶 Eurema hecabe	280	60	白背飞虱 Sogatella furcifera	150
28	硕蝽 Eurostus validus	86	61	甜菜夜蛾 Spodoptera exigua	150
29	菜蝽 Eurydema dominulus	99	62	草地贪夜蛾 Spodoptera frugiperda	66
30	梨小食心虫 Grapholita molesta	56	63	斜纹夜蛾 Spodoptera litura	150
31	蟋蟀 Gryllidae	130	64	红脊长蝽 Tropidothorax elegans	62
32	蝼蛄 Gryllotalpa spps	132	65	大红蛱蝶 Vanessa indica	130
33	棉铃虫 Helicoverpa armigera	108	66	短角外斑腿蝗 Xenocatantops brachycerus	112

表1

图3

图4

图5

图6

表2

图7

图8

图9

参考文献 45

[1]	刘德营. 稻飞虱自动识别关键技术的研究[D]. 南京: 南京农业大学, 2011.
	LIU D Y. Study on automatic identification technology for rice plant hoppers[D]. Nanjing: Nanjing Agricultural University, 2011. (in Chinese)
[2]	吴翔. 基于机器视觉的害虫识别方法研究[D]. 杭州: 浙江大学, 2016.
	WU X. Study on identification of pests based on machine vision[D]. Hangzhou: Zhejiang University, 2016. (in Chinese)
[3]	杨慧勇, 高灵旺, 牛国飞, 乔建, 刘伟, 邢鲲, 王鑫, 李捷. 农业害虫远程自动识别诊断系统应用技术研究. 山西农业科学, 2010,38(6):40-42.
	YANG H Y, GAO L W, NIU G F, QIAO J, LIU W, XING K, WANG X, LI J. Technology research of agricultural pests remote automatic diagnosis system. Journal of Shanxi Agricultural Sciences, 2010,38(6):40-42. (in Chinese)
[4]	HAN Y Q, SHAN X, ZHU D. Design and implementation of locust data collecting system based on android. Advances in Computer Science and Education Applications, 2011,2:328-337.
[5]	张谷丰, 罗岗, 孙雪梅. 基于Android的水稻害虫诊断系统. 应用昆虫学报, 2015,4:837-843.
	ZHANG G F, LUO G, SUN X M. An android, mobile phone-based identification system for rice insects. Chinese Journal of Applied Entomology, 2015,4:837-843. (in Chinese)
[6]	朱莉, 罗靖, 徐胜勇, 杨勇, 赵海涛, 李卫豪. 基于颜色特征的油菜害虫机器视觉诊断研究. 农机化研究, 2016,38(6):55-58.
	ZHU L, LUO J, XU S Y, YANG Y, ZHAO H T, LI W H. Machine vision recognition of rapeseed pests based on color feature. Journal of Agricultural Mechanization Research, 2016,38(6):55-58. (in Chinese)
[7]	王德发. 基于图像识别的储粮害虫检测[D]. 北京: 北京邮电大学, 2017.
	WANG D F. Detection of stored grain pests based on image recognition. Beijing: Beijing University of Posts and Telecommunications, 2017. (in Chinese)
[8]	WEN C, GUYER D. Image-based orchard insect automated identification and classification method. Computers and Electronics in Agriculture, 2012,89:110-115.
[9]	JUAN Z, XIAO P C. Field pest identification by an improved Gabor texture segmentation scheme. New Zealand Journal of Agricultural Research, 2007,50(5):719-723. doi: 10.1080/00288230709510343
[10]	程鲁玉, 孟小艳, 达新民. 关于果林中果害虫图像特征高效分类识别仿真. 计算机仿真, 2018,35(2):425-428.
	CHENG L Y, MENG X Y, DA X M. Simulation of efficient classification for image characteristics of fruit in fruit pest. Computer Simulation, 2018,35(2):425-428. (in Chinese)
[11]	WEN C, GUYER D E, LI W. Local feature-based identification and classification for orchard insects. Biosystems Engineering, 2009,104(3):299-307.
[12]	LIU T, CHEN W, WU W, SUN C, GUO W, ZHU X. Detection of aphids in wheat fields using a computer vision technique. Biosystems Engineering, 2016,141:82-93.
[13]	QING Y, XIAN D, LIU Q, YANG B, DIAO G, TANG J. Automated counting of rice planthoppers in paddy fields based on image processing. Journal of Integrative Agriculture, 2014,13(8):1736-1745.
[14]	LARIOS N, DENG H, ZHANG W, SARPOLA M, YUEN J, PAASCH R, MOLDENKE A, LYTLE D, CORREA S, MORTENSEN E, SHAPIRO L, DIETTERICH T. Automated insect identification through concatenated histograms of local appearance features: Feature vector generation and region detection fordeformable objects. Machine Vision and Applications, 2008,19(2):105-123.
[15]	胡永强, 宋良图, 张洁, 谢成军, 李瑞. 基于稀疏表示的多特征融合害虫图像识别. 模式识别与人工智能, 2014,27(11):985-992.
	HU Y Q, SONG L T, ZHANG J, XIE C J, LI R. Pest image recognition of multi-feature fusion based on sparse representation. Pattern Recognition and Artificial Intelligence, 2014,27(11):985-992. (in Chinese)
[16]	WEN C, WU D, HU H, PAN W. Pose estimation-dependent identification method for field moth images using deep learning architecture. Biosystems Engineering, 2015,136:117-128.
[17]	ESPINOZA K, VALERA D L, TORRES J A, LÓPEZ A, MOLINA-AIZ F. Combination of image processing and artificial neural networks as a novel approach for the identification of Bemisia tabaci and Frankliniellaoccidentalis on sticky traps in greenhouse agriculture. Computers and Electronics in Agriculture, 2016,127:495-505. doi: 10.1016/j.compag.2016.07.008
[18]	周曼, 周明全. 基于BP神经网络的水稻害虫自动识别. 北京师范大学学报(自然科学版), 2008,44(2):165-167.
	ZHOU M, ZHOU M Q. Automatic rice pest insects recognition based on bp neural network. Journal of Beijing Normal University(Natural Science), 2008,44(2):165-167. (in Chinese)
[19]	XIE C, ZHANG J, LI R, LI J, HONG P, XIA J, CHEN P. Automatic classification for field crop insects via multiple-task sparse representation and multiple-kernel learning. Computers and Electronics in Agriculture, 2015,119:123-132. doi: 10.1016/j.compag.2015.10.015
[20]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks. Proceedings of the 25^th International Conference on Neural Information Processing Systems. Lake Tahoe,America, 2012: 1097-1105.
[21]	LIU Z, GAO J, YANG G, ZHANG H, HE Y. Localization and classification of paddy field pests using a saliency map and deep convolutional neural network. Scientific Reports, 2016,6:1-12. pmid: 28442746
[22]	DING W, TAYLOR G. Automatic moth detection from trap images for pest management. Computers & Electronics in Agriculture, 2016,123:17-28.
[23]	陈娟, 陈良勇, 王生生, 赵慧颖, 温长吉. 基于改进残差网络的园林害虫图像识别. 农业机械学报, 2019,50(5):187-195.
	CHEN J, CHEN L Y, WANG S S, ZHAO H Y, WEN C J. Pest image recognition of garden based on improved residual network. Transactions of the Chinese Society for Agricultural Machinery, 2019,50(5):187-195. (in Chinese)
[24]	CHENG X, ZHANG Y, CHEN Y, WU Y, YUE Y. Pest identification via deep residual learning in complex background. Computers and Electronics in Agriculture, 2017,141:351-356. doi: 10.1016/j.compag.2017.08.005
[25]	XIA D, CHEN P, WANG B, ZHANG J, XIE C. Insect detection and classification based on an improved convolutional neural network. Sensors, 2018,18(12):1-12.
[26]	LIU L, WANG R, XIE C, WANG F. PestNet: An end-to-end deep learning approach for large-scale multi-class pest detection and classification. IEEE Access, 2019,7:45301-45312.
[27]	ALFARISY A A, CHEN Q, GUO M. Deep learning based classification for paddy pests & diseases recognition. Proceedings of 2018 International Conference on Mathematics and Artificial Intelligence. Chengdu,China, 2018: 21-25.
[28]	沙小梅, 张路正. 识花软件在花卉栽培与室内装饰课程教学中的应用——以形色(赏花识花APP)为例. 课程教育研究, 2018,37:227.
	SHAO X M, ZHANG L Z. Application of flower recognition software in flower cultivation and interior decoration course teaching--take xingse(identifying flowers app) for example. Course Education Research, 2018,37:227. (in Chinese)
[29]	王桢, 何建勇. 花伴侣:把“植物大百科”装进小口袋. 绿化与生活, 2017,10:30-31.
	WANG Z, HE J Y. Flower companion: Put "encyclopedia of plants" in your pocket. Greening and Life, 2017,10:30-31. (in Chinese)
[30]	樊景超. 基于MobileNets的果园害虫分类识别模型研究. 天津农业科学, 2018,24(9):11-13.
	FAN J C. Study on classification and recognition of orchard pests based on mobile nets. Tianjin Agricultural Sciences, 2018,24(9):11-13. (in Chinese)
[31]	赵彬宇, 周慧玲, 李江涛, 严晓平, 韩康榕. 储粮害虫智能图鉴及图像识别APP软件设计. 粮食储藏, 2019,48(3):42-46.
	ZHAO B Y, ZHOU H L, LI J T, YAN X P, HAN K R. Mobile app design for stored grain insect illustration guide and image recognition. Grain Storage, 2019,48(3):42-46. (in Chinese)
[32]	SHORTEN C, KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019,6(1):1-48. doi: 10.1186/s40537-018-0162-3
[33]	HOU J, SU H Y, YAN B, ZHENG H, SUN Z, CAI X. Classification of tongue color based on CNN. Proceedings of IEEE 2nd International Conference on Big Data Analysis.Beijing,China, 2017: 725-729 .
[34]	DENG L M, WANG Y J, HAN Z Z, YU R R. Research on insect pest image detection and recognition based on bio-inspired methods. Biosystems Engineering, 2018,169:139-148. doi: 10.1016/j.biosystemseng.2018.02.008
[35]	SZEGEDY C, LIU W, JIA Y, SERMANET P, REED S, ANGUELOV D, ERHAN D, VANHOUCKE V, RABINOVICH A. Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition, Boston. USA, 2015: 1-9.
[36]	WU Z, SHEN C, VAN DEN HENGEL A. Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 2019,90:119-133. doi: 10.1016/j.patcog.2019.01.006
[37]	HUANG G, LIU Z, VAN DER MAATEN L, KILIAN Q. Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Ameriaca, 2017: 4700-4708.
[38]	杨婧, 耿辰, 王海林, 纪建松, 戴亚康. 基于DenseNet的低分辨CT影像肺腺癌组织学亚型分类. 浙江大学学报(工学版), 2019,53(6):1164-1170.
	YANG J, GENG C, WANG H L, JI J S, DAI Y K. Classification on histological subtypes of lung adenocarcinoma from low-resolution CT images based on DenseNet. Journal of Zhejiang University (Engineering Science), 2019,53(6):1164-1170. (in Chinese)
[39]	ZHU Y, NEWSAM S. Densenet for dense flow. 2017 IEEE International Conference on Image Processing. Beijing,China, 2017: 790-794.
[40]	唐明轩. 基于DenseNet的医学图像分割研究与应用[D]. 成都: 成都信息工程大学, 2018.
	TANG M X. Research and application of medical image segmentation based on densenet[D]. Chengdu: ChengDu University of Information Engineering, 2018. (in Chinese)
[41]	RUSSAKOVSKY O, DENG J, SU H, KRAUSE J, SATHEESH S, MA S, HUANG Z, KARPATHY S, KHOSLA A, BERNSTEIN M, BERG A, LI F F. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015,115(3):211-252. doi: 10.1007/s11263-015-0816-y
[42]	RAVISHANKAR H, SUDHAKAR P, VENKATARAMANI R, THIRUVENKADAM S, ANNANGI P, BABU N, VAIDYA V. Understanding the mechanisms of deep transfer learning for medical images. Deep Learning and Data Labeling for Medical Applications. Springer, 2016: 188-196.
[43]	KE H, CHEN D, LI X, TANG Y, SHAH T, RANJAN R. Towards brain big data classification: Epileptic EEG identification with a lightweight VGGNet on global MIC. IEEE Access, 2018,6:14722-14733. doi: 10.1109/ACCESS.2018.2810882
[44]	SZEGEDY C, VANHOUCKE V, IOFFE S, SHLENS J, WOJNA Z. Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,USA, 2016: 2818-2826.
[45]	HE K, ZHANG X, REN S, SUN J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,USA, 2016: 770-778.

评价方法Evaluation measures	CaffeNet	GoogleNet	VGG19	ResNet101	DenseNet121
平均识别率Average identification rate (%)	86.2	87.4	88.3	91.7	93.9
平均虚警率Average false alarm rate (%)	16.5	17.1	16.4	11.9	8.2

面向移动终端的农业害虫图像智能识别系统的研究与开发

Research and Development of the Intelligent Identification System of Agricultural Pests for Mobile Terminals

RichHTML

PDF

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 45

相关文章 1

Metrics

本文评价

推荐阅读 0