Scientia Agricultura Sinica ›› 2021, Vol. 54 ›› Issue (21): 4562-4572.doi: 10.3864/j.issn.0578-1752.2021.21.007

• TILLAGE & CULTIVATION·PHYSIOLOGY & BIOCHEMISTRY·AGRICULTURE INFORMATION TECHNOLOGY • Previous Articles     Next Articles

Research on Fine-Grained Image Recognition of Agricultural Light- Trap Pests Based on Bilinear Attention Network

YAO Qing1(),YAO Bo1,LÜ Jun1,TANG Jian2,*(),FENG Jin3,ZHU XuHua3   

  1. 1School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018
    2Rice Technology Research and Development Center, China National Rice Research Institute, Hangzhou 311401
    3Zhejiang Top Cloud-Agri Technology Co., Ltd., Hangzhou 310015
  • Received:2020-12-21 Accepted:2021-03-03 Online:2021-11-01 Published:2021-11-09
  • Contact: Jian TANG E-mail:q-yao@126.com;tangjian@caas.cn

RichHTML

43

PDF

269

Abstract:

【Objective】Some agricultural pests trapped by the intelligent light traps show the intraspecies difference and interspecies similarity due to a variety of pest species, different pest poses and scale missing. To improve the identification rate of agricultural light-trap pests, a fine-grained image identification model of agricultural light-trap pests based on bilinear attention network (BAPest-net) was proposed to identify 19 pest species which were easily misjudged by YOLOv4 model.【Method】Firstly, according to the appearance similarity and false detection results, 19 light-trap pest species were divided into 6 similar classes. All the pest images were processed to be equal in length and width through the edge-filling operation. Then, they were scaled to a uniform size of 224×224 pixels. In order to improve the robustness and generalization ability of model, the pest images were enhanced by mirror and flipping, rotation by 180 degrees, Gaussian noise, and mean filtering. The proportions of training set, validation set, and test set in samples are 80%, 10% and 10% respectively. An agricultural light-trap pest identification model based on bilinear attention network (bilinear-attention pest net, BAPest-net) was developed to identify 19 pest species belong to 6 similar pest classes. The BAPest-net model consisted of three modules, which were a feature extraction module, an attention mechanism module and an identification module. The down-sampling step in the feature extraction module was post-handled to extract more features. The attention mechanism model could make the model focus on the local features, which could increase the identification rate and learning efficiency. The model optimizer used the stochastic gradient descent method, and the global average pooling was used in the classification module to avoid overfitting from the structure of the entire network. Finally, the five models, including VGG19, Densenet, ResNet50, bilinear model and BAPest-net, were training on the same training set and were used to test 19 light-trap pests in the 6 similar pest classes. Precision, Precision-Recall curve and average identification rate were used to evaluate the identification effects of different models on similar light-trap pests.【Result】 In five models, the BAPest-net model had the highest average identification rate of 94.9% on 19 light-trap pests in 6 similar pest classes. The bilinear model gained the second high identification rate of 90.2% and the VGG19 model had only the lowest identification rate of 82.1%. The average identification rates of Lepidoptera pests in four pest categories were greater than 95.0%. 【Conclusion】The fine-grained image identification model of agricultural light-trap pests based on bilinear attention network could automatically identify 19 agricultural light-trap pests in 6 similar pest classes and improve the automatic identification accuracy of agricultural light-trap pests.

Key words: agricultural light-trap pest, pest identification, fine-grained image, bilinear, attention mechanism

Fig. 1

The pest images from the intelligent light traps"

"

类别
Class		 害虫种类
Pest name		 图像
Image		 类别
Class		 害虫种类
Pest name		 图像
Image		 类别
Class		 害虫种类
Pest name		 图像
Image
A 铜绿丽金龟
Anomala corpulenta		 C 榆黄足毒蛾
Ivela ochropoda		 F 二化螟
Chilo suppressalis
灰胸突鳃金龟
Hoplosternus incanus 		杨雪毒蛾
Stilprotia salicis		 草地螟
Loxostege stieticatis
大黑鳃金龟
Holotrichia Oblita 		D 槐尺蛾
Semiothisa cinerearia		 金黄镰翅野螟
Circobotys aurealis
B 蝼蛄
Gryllotalpa spps		 格庶尺蛾
Semiothisa hebesata		 稻纵卷叶螟
Cnaphalocrocis medinalis
中华蟋蟀
Gryllus chinensis		 E 棉铃虫
Helicoverpa armigera		 玉米螟
Pyrausta nubilalis
黄足猎蝽
Sirthenea flavipes		 烟青虫
Heliothis assulta		 大螟
Sesamia inferens
劳氏粘虫
Leucania loreyi duponchel

Fig. 2

Data enhancement of light-trap pest images (a)Original image; (b)Mirror flip; (c)Rotate 180 degrees; (d)Gaussian noise; (e)Mean filtering"

Fig. 3

Different individual images of same pest species with great differences (a)Gryllotalpa spps; (b) Loxostege stieticatis; (c) Holotrichia oblita; (d) Pyrausta nubilalis"

Fig. 4

The structure diagram of the fine-grained image identification network of agricultural light-trap pests"

Fig. 5

Post-down sampling structure diagram"

Fig. 6

Feature maps extracted by ResNet50 and improved ResNet50 (a) Original feature maps; (b) Feature maps after improved"

Fig. 7

The PR curves of five models on 19 pest species"

Table 2

Average recognition rates of 19 light-trap pest species by five models"

模型 Model
VGG19 Densenet Resnet50 BCNN BAPest-net
平均识别率 Average recognition rate (%) 82.1 89.2 88.4 90.2 94.9

Table 3

Identification results of 6 categories of agricultural lamp-trap pests"

类别 Class
A B C D E F
平均识别率 Average Identification rate (%) 91.0 89.5 98.3 95.5 95.6 97.9

Fig. 8

Precision of 19 species agricultural light-trap pests on 5 models"

Fig. 9

Recognition results of agricultural light-trap pest images based on BAPest-net"

Fig. 10

Some images of misjudged pests (a)Anomala corpulenta; (b)Holotrichia oblita; (c)Sirthenea flavipes; (d)Gryllotalpa spps; (e)Stilprotia salicis; (f)Ivela ochropoda"

[1] 张心雨. 低密度下评估稻飞虱智能识别系统的有效性[D]. 南京: 南京农业大学, 2016.
ZHANG X Y. Evaluate the effectiveness of automatic identification system at the low planthoppers density. Nanjing: Nanjing Agricultural University, 2016. (in Chinese)
[2] 林培炯, 张智, 王旭龙, 刘冬雪, 胡高, 张云慧. 2019年北京市延庆区草地贪夜蛾种群动态与虫源分析. 植物保护学报, 2020, 47(4):758-769.
LIN P J, ZHANG Z, WANG X L, LIU D X, HU G, ZHANG Y H. Population dynamic and trajectory simulation of migratory moths of fall armyworm Spodoptera frugiperda in Yangqing of Beijing in 2019. Journal of Plant Protection, 2020, 47(4):758-769. (in Chinese)
[3] 王祥会, 焦玉霞, 王圣楠, 孙玉涛, 王宏梅, 孔德生. 祥辰智能虫情测报灯在大田害虫监测预报中的应用研究. 基层农技推广, 2020, 8(8):43-45.
WANG X H, JIAO Y X, WANG S N, SUN Y T, WANG H M, KONG D S. Application of Xiangchen intelligent pest situation forecasting lamp in field pest monitoring and forecasting. Grassroots Agricultural Technology Extension, 2020, 8(8):43-45. (in Chinese)
[4] YAO Q, FENG J, TANG J, XU W G, ZHU X H, YANG B J, LV J, XIE Y Z, YAO B, WU S Z, KUAI N Y, WANG L J. Development of an automatic monitoring system for rice light-trap pests based on machine vision. Journal of Integrative Agriculture, 2020, 19(10):2500-2513.
doi: 10.1016/S2095-3119(20)63168-9
[5] EBRAHIMI M A, KHOSHTAGHAZA M H, MINAEI S, JAMSHIDI B. Vision-based pest detection based on SVM classification method. Computers and Electronics in Agriculture, 2017, 137:52-58.
doi: 10.1016/j.compag.2017.03.016
[6] XIAO D Q, FENG J Z, LIN T Y, PANG C H, YE Y W. Classification and recognition scheme for vegetable pests based on the BOF-SVM model. International Journal of Agricultural and Biological Engineering, 2018, 11(3):190-196.
[7] DENG L M, WANG Y J, HAN Z Z, YU R S. Research on insect pest image detection and recognition based on bio-inspired methods. Biosystems Engineering, 2018, 169:139-148.
doi: 10.1016/j.biosystemseng.2018.02.008
[8] ZHAO J, CHENG X P. Field pest identification by an improved Gabor texture segmentation scheme. New Zealand Journal of Agricultural Research, 2007, 50(5):719-723.
doi: 10.1080/00288230709510343
[9] 佘颢, 吴伶, 单鲁泉. 基于SSD网络模型改进的水稻害虫识别方法. 郑州大学学报(理学版), 2020, 52(3):49-54.
SHE H, WU L, SHAN L Q. Improved rice pest recognition based on SSD network model. Journal of Zhengzhou University (Natural Science Edition), 2020, 52(3):49-54. (in Chinese)
[10] 钱蓉, 孔娟娟, 朱静波, 张萌, 董伟. 基于VGG-16卷积神经网络的水稻害虫智能识别研究. 安徽农业科学, 2020, 48(5):235-238.
QIAN R, KONG J J, ZHU J B, ZHANG M, DONG W. Research on intelligent identification of rice pests based on VGG-16 convolutional neural network. Journal of Anhui Agricultural Sciences, 2020, 48(5):235-238. (in Chinese)
[11] 梁万杰, 曹宏鑫. 基于卷积神经网络的水稻虫害识别. 江苏农业科学, 2017, 45(20):241-243, 253.
LIANG W J, CAO H X. Rice pest identification based on convolutional neural network. Jiangsu Agricultural Sciences, 2017, 45(20):241-243, 253. (in Chinese)
[12] ALFARISY A A, CHEN Q, GUO M. Deep learning based classificationfor paddy pests & diseases recognition. Proceedings of 2018 International Conference on Mathematics and Artificial Intelligence. Chengdu, 2018: 21-25.
[13] 邵泽中, 姚青, 唐健, 李罕琼, 杨保军, 吕军, 陈轶. 面向移动终端的农业害虫图像智能识别系统的研究与开发. 中国农业科学, 2020, 53(16):3257-3268.
SHAO Z Z, YAO Q, TANG J, LI H Q, YANG B J, LÜ J, CHEN Y. Research and development of the intelligent identification system of agricultural pests for mobile terminals. Scientia Agricultura Sinica, 2020, 53(16):3257-3268. (in Chinese)
[14] 姚青, 吴叔珍, 蒯乃阳, 杨保军, 唐健, 冯晋, 朱旭华, 朱先敏. 基于改进CornerNet的水稻灯诱飞虱自动检测方法构建与验证. 农业工程学报, 2021, 37(7):183-189.
YAO Q, WU S Z, KUAI N Y, YANG B J, TANG J, FENG J, ZHU X H, ZHUX M. Construction and verification of automatic detection method for rice planthoppers on light-trap insect images based on improved ConerNet. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(7):183-189.
[15] CHENG X, ZHANG Y H, CHEN Y Q, WU Y Z, YUE Y. Pest identification via deep residual learning in complex background. Computers and Electronics in Agriculture, 2017, 141:351-356.
doi: 10.1016/j.compag.2017.08.005
[16] SUN Y, LIU X X, YUAN M S, REN L L, WANG J X, CHEN Z B. Automatic in-trap pest detection using deep learning for pheromone- based dendroctonus valens monitoring. Biosystems Engineering, 2018, 176:140-150.
doi: 10.1016/j.biosystemseng.2018.10.012
[17] SHEN Y F, ZHOU H L, LI J T, JIAN F J, JAYAS D S. Detection of stored-grain insects using deep learning. Computers and Electronics in Agriculture, 2018, 145:319-325.
doi: 10.1016/j.compag.2017.11.039
[18] NAQVI S D Y, HAILE A, RAO S, TEWElDEMEDHIN B, SHARMA V, NYENDE A B. Evaluation of husbandry, insect pests, diseases and management practices of vegetables cultivated in Zoba Anseba, Eritrea. Journal of Eco-friendly Agriculture, 2017, 12(1):47-50.
[19] FERENTINOS K P. Deep learning models for plant disease detection and diagnosis. Computers and Electronics in Agriculture, 2018, 145:311-318.
doi: 10.1016/j.compag.2018.01.009
[20] LIU L, WANG R, XIE C, YANG P, WANG F, SUDIRMAN S, LIU W. PestNet: An end-to-end deep learning approach for large-scale multi-class pest detection and classification. IEEE Access, 2019, 7:45301-45312.
doi: 10.1109/Access.6287639
[21] 孔建磊, 金学波, 陶治, 王小艺, 林森. 基于多流高斯概率融合网络的病虫害细粒度识别. 农业工程学报, 2020, 36(13):148-157.
KONG J L, JIN X B, TAO Z, WANG X Y, LIN S. Fine-grained recognition of pests and diseases based on multi-stream Gaussian probability fusion network. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(13):148-157. (in Chinese)
[22] LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear cnn models for fine-grained visual recognition. Proceedings of the IEEE International Conference on Computer Vision. CentroParque Convention Center in Santiago,Chile, 2015: 1449-1457.
[23] 蓝洁, 周欣, 何小海, 滕奇志, 卿粼波. 基于跨层精简双线性网络的细粒度鸟类识别. 科学技术与工程, 2019, 19(36):240-246.
LAN J, ZHOU X, HE X H, TENG Q Z, QING L B. Fine-grained bird recognition based on cross-layer compact bilinear network. Science Technology and Engineering, 2019, 19(36):240-246. (in Chinese)
[24] 冀中, 赵可心, 张锁平, 李明兵. 基于空间变换双线性网络的细粒度鱼类图像分类. 天津大学学报: 自然科学与工程技术版, 2019, 52(5):475-482.
JI Z, ZHAO K X, ZHANG S P, LI M B. Fine-grained fish image classification based on a bilinear network with spatial transformation. Journal of Tianjin University: Natural Science and Engineering Technology Edition, 2019, 52(5):475-482. (in Chinese)
[25] 刘虎, 周野, 袁家斌. 基于多尺度双线性卷积神经网络的多角度下车型精细识别. 计算机应用, 2019, 39(8):2402-2407.
LIU H, ZHOU Y, YUAN J B. Fine-grained vehicle recognition under multiple angles based on multi-scale bilinear convolutional neural network. Journal of Computer Applications, 2019, 39(8):2402-2407. (in Chinese)
[26] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 25:1097-1105.
[27] ZHU Q, ZHENG H F, WANG Y B, CAO Y G, GUO S X. Study on the evaluation method of sound phase cloud maps based on an improved YOLOv4 algorithm. Sensors, 2020, 20(15):4313-4330.
doi: 10.3390/s20154313
[28] HOU J, SU H Y, YAN B, ZAHNG H, SUN Z, CAI X. Classification of tongue color based on CNN. IEEE 2nd International Conference on Big Data Analysis. Beijing, 2017: 725-729.
[29] SZEGEDY C, LIU W, JIA Y, SERMANET P, REED S, ANGUELOV D, ERHAN D, VANHOUCKE V, RABINOVICH A. Going deeper with convolutions. Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition. Hynes Convention Center, Boston, 2015: 1-9.
[30] HE K M, ZHANG X, REN S Q, SUN J. Deep residual learning for image recognition. Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, 2016: 770-778.
[31] MMIH V, HEESS N, GRAVES A. Recurrent models of visual attention. Advances in Neural Information Processing Systems. Palais des Congres de Montreal, Montreal Canada, 2014: 2204-2212.
[32] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, SUTSKEVER I, SALAKHUTDINOV R. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014, 15(1):1929-1958.
No related articles found!
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备09089781号-30
Copyright 2018 © Editorial office of Scientia Agricultura Sinica
Tel: 86-010-82106279    E-mail: zgnykx@mail.caas.net.cn
Supported by Beijing Magtech Co.Ltd