Please wait a minute...
Journal of Integrative Agriculture  2023, Vol. 22 Issue (6): 1671-1683    DOI: 10.1016/j.jia.2022.09.021
Crop Science Advanced Online Publication | Current Issue | Archive | Adv Search |
Detection of maize tassels for UAV remote sensing image with an improved YOLOX Model
SONG Chao-yu1, ZHANG Fan1, LI Jian-sheng2, XIE Jin-yi1, YANG Chen1, ZHOU Hang1, ZHANG Jun-xiong1#
1 College of Engineering, China Agricultural University, Beijing 100083, P.R.China

2 College of Agronomy and Biotechnology, China Agricultural University, Beijing 100083, P.R.China

Download:  PDF in ScienceDirect  
Export:  BibTeX | EndNote (RIS)      
摘要  玉米雄穗检测是玉米种植和育种农艺管理中必不可少的技术,可应用于产量估算、生长监测、智能采摘、病害检测等方面。然而,田间的玉米雄穗普遍存在遮挡现象,不同生长阶段的雄穗大小和形态颜色也不尽相同。针对这些问题,本文提出了SEYOLOX-tiny模型,可以更精准、更鲁棒地识别田间的玉米雄穗。通过无人机构建了丰富的玉米雄穗图像数据集,在保证图像质量和图像采集效率的同时兼顾不同时期的玉米雄穗的图像多样性。另外,YOLOX嵌入注意力机制,在关键特征的提取时,能有效抑制不利因素(遮挡、重叠)的噪声,有助于应对农田多变复杂的环境。实验结果显示,改进的识别算法SEYOLOX-tiny平均检测精度达到95.0%;相较于原始模型的mAP@0.5、mAP@0.5-0.95、mAP@0.5-0.95(面积=小)和mAP@0.5-0.95(面积=中)提升1.5、1.8、5.3和1.7个百分点。因此,本文提出的方法可以满足玉米穗检测视觉系统中所需要的精度和鲁棒性。

Abstract  

Maize tassel detection is essential for future agronomic management in maize planting and breeding, with application in yield estimation, growth monitoring, intelligent picking, and disease detection.  However, detecting maize tassels in the field poses prominent challenges as they are often obscured by widespread occlusions and differ in size and morphological color at different growth stages.  This study proposes the SEYOLOX-tiny Model that more accurately and robustly detects maize tassels in the field.  Firstly, the data acquisition method ensures the balance between the image quality and image acquisition efficiency and obtains maize tassel images from different periods to enrich the dataset by unmanned aerial vehicle (UAV).  Moreover, the robust detection network extends YOLOX by embedding an attention mechanism to realize the extraction of critical features and suppressing the noise caused by adverse factors (e.g., occlusions and overlaps), which could be more suitable and robust for operation in complex natural environments.  Experimental results verify the research hypothesis and show a mean average precision (mAP@0.5) of 95.0%.  The mAP@0.5, mAP@0.5–0.95, mAP@0.5–0.95 (area=small), and mAP@0.5–0.95 (area=medium) average values increased by 1.5, 1.8, 5.3, and 1.7%, respectively, compared to the original model.  The proposed method can effectively meet the precision and robustness requirements of the vision system in maize tassel detection.

Keywords:  maize       tassel detection       remote sensing       deep learning       attention mechanism  
Received: 11 April 2022   Online: 27 September 2022   Accepted: 26 August 2022
Fund: 

This research was supported by the Chinese Universities Scientific Fund (2022TC169). 

About author:  SONG Chao-yu, E-mail: chaoyu@cau.edu.cn; #Correspondence ZHANG Jun-xiong, E-mail: cau2007@cau.edu.cn.

Cite this article: 

SONG Chao-yu, ZHANG Fan, LI Jian-sheng, XIE Jin-yi, YANG Chen, ZHOU Hang, ZHANG Jun-xiong. 2023. Detection of maize tassels for UAV remote sensing image with an improved YOLOX Model. Journal of Integrative Agriculture, 22(6): 1671-1683.

Bendig J, Yu K, Aasen H, Bolten A, Bennertz S, Broscheit J, Gnyp M L, Bareth G. 2015. Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring in barley. International Journal of Applied Earth Observation and Geoinformation39, 79–87.

Camargo A, Smith J S. 2009. An image-processing based algorithm to automatically identify plant disease visual symptoms. Biosystems Engineering102, 9–21.

Chang L, He S, Huang D, Liu Q, Xiang J. 2018. Quantifying muskmelon fruit attributes with A-TEP-based model and machine vision measurement. Journal of Integrative Agriculture17, 1369–1379.

Gage J L, Miller N D, Spalding E P, Kaeppler S M, de Leon N. 2017. TIPS: A system for automated image-based phenotyping of maize tassels. Plant Methods13, 21.

Ge Z, Liu S, Li Z, Yoshie O, Sun J. 2021. OTA: Optimal transport assignment for object detection. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Nashville, TN, USA. pp. 303–312.

Ge Z, Liu S, Wang F, Li Z, Sun J. 2021. YOLOX: Exceeding YOLO series in 2021. arXiv Preprint arXiv:2107.08430.

Hu J, Shen L, Albanie S, Sun G, Wu E. 2020. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence42, 2011–2023.

Jannoura R, Brinkmann K, Uteau D, Bruns C, Joergensen R G. 2015. Monitoring of crop biomass using true colour aerial photographs taken from a remote controlled hexacopter. Biosystems Engineering129, 341–351.

Ji M, Yang Y, Zheng Y, Zhu Q, Huang M, Guo Y. 2021. In-field automatic detection of maize tassels using computer vision. Information Processing in Agriculture8, 87–95.

Jin G H, Hyeonjoon M, Jin T K, Syed I H, Minh D, O N L, Han Y P. 2017. Deep convolutional neural network for classifying Fusarium wilt of radish from unmanned aerial vehicles. Journal of Applied Remote Sensing11, 1–14.

Karami A, Quijano K, Crawford M. 2021. Advancing tassel detection and counting: Annotation and algorithms. Remote Sensing13, 2881.

Koh J C O, Hayden M, Daetwyler H, Kant S. 2019. Estimation of crop plant density at early mixed growth stages using UAV imagery. Plant Methods15, 64.

Kurtulmuş F, Kavdir 0. 2014. Detecting corn tassels using computer vision and support vector machines. Expert Systems with Applications41, 7390–7397.

Li L, Sun Q, Tu K L, Wang J H, Yang L M. 2018. Selection for high quality pepper seeds by machine vision and classifiers. Journal of Integrative Agriculture17, 1999–2006.

Lin T, Goyal P, Girshick R, He K, Dollar P. 2020. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence42, 318–327.

Liu Y, Cen C, Che Y, Ke R, Ma Y, Ma Y. 2020. Detection of maize tassels from UAV RGB imagery with faster R-CNN. Remote Sensing12, 338.

Lu H, Cao Z, Xiao Y, Fang Z, Zhu Y, Xian K. 2015. Fine-grained maize tassel trait characterization with multi-view representations. Computers and Electronics in Agriculture118, 143–158.

Osco L P, Marcato J, Ramos A P M, Jorge L A D, Fatholahi S N, Silva J D, Matsubara E T, Pistori H, Gonçalves W N, Li J. 2021. A review on deep learning in UAV remote sensing. International Journal of Applied Earth Observation and Geoinformation102, 102456.

Pan S, Qiao J, Wang R, Yu H, Wang C, Taylor K, Pan H. 2022. Intelligent diagnosis of northern corn leaf blight with deep learning model. Journal of Integrative Agriculture21, 1094–1105.

Ranum P, Peña-Rosas J P, Garcia-Casal M N. 2014. Global maize production, utilization, and consumption. Annals of the New York Academy of Sciences1312, 105–112.

Ren S, He K, Girshick R, Sun A J. 2017. Faster R-CNN: Towards Real-Time object detection with region proposal networks. Ieee Transactions on Pattern Analysis and Machine Intelligence39, 1137–1149.

Su Y, Wu F, Ao Z, Jin S, Qin F, Liu B, Pang S, Liu L, Guo Q. 2019. Evaluating maize phenotype dynamics under drought stress using terrestrial lidar. Plant Methods15, 11.

Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q. 2020. ECA-Net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Long Beach, CA, USA. pp. 11531–11539.

Woo S, Park J, Lee J, Kweon I S. 2018. CBAM: Convolutional block attention module. In: The 15th European Conference on Computer Vision (ECCV). Springer, Cham, Munich, Germany. pp. 3–19.

Yang G, Yang Y, He Z, Zhang X, He Y. 2022. A rapid, low-cost deep learning system to classify strawberry disease based on cloud service. Journal of Integrative Agriculture21, 460–473.

Yang S, Liu J, Xu K, Sang X, Ning J, Zhang Z. 2021. Improved CenterNet based maize tassel recognition for UAV remote sensing image. Transactions of the Chinese Society for Agricultural Machinery52, 206–212.

Ye M, Cao Z, Yu Z. 2013. An image-based approach for automatic detecting tasseling stage of maize using spatio-temporal saliency. In: Remote Sensing Image Processing, Geographic Information Systems, and Other Applications. International Society for Optics and Photonics. SPIE, Washington, DC, USA. p. 89210Z.

Zan X, Zhang X, Xing Z, Liu W, Zhang X, Su W, Liu Z, Zhao Y, Li S. 2020. Automatic detection of maize tassels from UAV images by combining random forest classifier and VGG16. Remote Sensing12, 3049.

Zhang C, Kovacs J. 2012. The application of small unmanned aerial systems for precision agriculture: A review. Precision Agriculture13, 693–712.

Zhang H, Wang Y, Dayoub F, Sunderhauf N. 2021. VarifocalNet: An IoU-aware dense object detector. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Nashville, TN, USA. pp. 8510–8519.

[1] WANG Xing-long, ZHU Yu-peng, YAN Ye, HOU Jia-min, WANG Hai-jiang, LUO Ning, WEI Dan, MENG Qing-feng, WANG Pu. Irrigation mitigates the heat impacts on photosynthesis during grain filling in maize [J]. >Journal of Integrative Agriculture, 2023, 22(8): 2370-2383.
[2] Tiago SILVA, Ying NIU, Tyler TOWLES, Sebe BROWN, Graham P. HEAD, Wade WALKER, Fangneng HUANG. Selection, effective dominance, and completeness of Cry1A.105/Cry2Ab2 dual-protein resistance in Helicoverpa zea (Boddie) (Lepidoptera: Noctuidae)[J]. >Journal of Integrative Agriculture, 2023, 22(7): 2151-2161.
[3] FAN Ting-lu, LI Shang-zhong, ZHAO Gang, WANG Shu-ying, ZHANG Jian-jun, WANG Lei, DANG Yi, CHENG Wan-li. Response of dryland crops to climate change and drought-resistant and water-suitable planting technology: A case of spring maize[J]. >Journal of Integrative Agriculture, 2023, 22(7): 2067-2079.
[4] ZHANG Chong, WANG Dan-dan, ZHAO Yong-jian, XIAO Yu-lin, CHEN Huan-xuan, LIU He-pu, FENG Li-yuan, YU Chang-hao, JU Xiao-tang. Significant reduction of ammonia emissions while increasing crop yields using the 4R nutrient stewardship in an intensive cropping system[J]. >Journal of Integrative Agriculture, 2023, 22(6): 1883-1895.
[5] ZHANG Miao-miao, DANG Peng-fei, LI Yü-ze, QIN Xiao-liang, Kadambot-H. M. SIDDIQUE. Better tillage selection before ridge–furrow film mulching can facilitate root proliferation, increase nitrogen accumulation, translocation, grain yield of maize in a semiarid area[J]. >Journal of Integrative Agriculture, 2023, 22(6): 1658-1670.
[6] WANG Peng, WANG Cheng-dong, WANG Xiao-lin, WU Yuan-hua, ZHANG Yan, SUN Yan-guo, SHI Yi, MI Guo-hua. Increasing nitrogen absorption and assimilation ability under mixed NO3 and NH4+ supply is a driver to promote growth of maize seedlings[J]. >Journal of Integrative Agriculture, 2023, 22(6): 1896-1908.
[7] WANG Jin-bin, XIE Jun-hong, LI Ling-ling, ADINGO Samuel. Review on the fully mulched ridge–furrow system for sustainable maize production on the semi-arid Loess Plateau[J]. >Journal of Integrative Agriculture, 2023, 22(5): 1277-1290.
[8] ZHAO Hai-liang, QIN Yao, XIAO Zi-yi, SUN Qin, GONG Dian-ming, QIU Fa-zhan. Revealing the process of storage protein rebalancing in high quality protein maize by proteomic and transcriptomic[J]. >Journal of Integrative Agriculture, 2023, 22(5): 1308-1323.
[9] ZHANG Bing-chao, HU Han, GUO Zheng-yu, GONG Shuai, SHEN Si, LIAO Shu-hua, WANG Xin, ZHOU Shun-li, ZHANG Zhong-dong. Plastic-film-side seeding, as an alternative to traditional film mulching, improves yield stability and income in maize production in semi-arid regions[J]. >Journal of Integrative Agriculture, 2023, 22(4): 1021-1034.
[10] WANG Fu-kuan, HUANG Yi-qi, HUANG Zhao-cheng, SHEN Hao, HUANG Cong, QIAO Xi, QIAN Wan-qiang.

MRUNet: A two-stage segmentation model for small insect targets in complex environments  [J]. >Journal of Integrative Agriculture, 2023, 22(4): 1117-1130.

[11] SHI Wen-xuan, ZHANG Qian, LI Lan-tao, TAN Jin-fang, XIE Ruo-han, WANG Yi-lun. Hole fertilization in the root zone facilitates maize yield and nitrogen utilization by mitigating potential N loss and improving mineral N accumulation[J]. >Journal of Integrative Agriculture, 2023, 22(4): 1184-1198.
[12] GAO Xing, LI Yong-xiang, YANG Ming-tao, LI Chun-hui, SONG Yan-chun, WANG Tian-yu, LI Yu, SHI Yun-su. Changes in grain-filling characteristics of single-cross maize hybrids released in China from 1964 to 2014[J]. >Journal of Integrative Agriculture, 2023, 22(3): 691-700.
[13] Irshad AHMAD, Maksat BATYRBEK, Khushnuma IKRAM, Shakeel AHMAD, Muhammad KAMRAN, Misbah, Raham Sher KHAN, HOU Fu-jiang, HAN Qing-fang.

Nitrogen management improves lodging resistance and production in maize (Zea mays L.) at a high plant density [J]. >Journal of Integrative Agriculture, 2023, 22(2): 417-433.

[14] XU Xiao-hui, LI Wen-lan, YANG Shu-ke, ZHU Xiang-zhen, SUN Hong-wei, LI Fan, LU Xing-bo, CUI Jin-jie. Identification, evolution, expression and protein interaction analysis of genes encoding B-box zinc-finger proteins in maize[J]. >Journal of Integrative Agriculture, 2023, 22(2): 371-388.
[15] CHEN Zhe, REN Wei, YI Xia, LI Qiang, CAI Hong-guang, Farhan ALI, YUAN Li-xing, MI Guo-hua, PAN Qing-chun, CHEN Fan-jun. Local nitrogen application increases maize post-silking nitrogen uptake of responsive genotypes via enhanced deep root growth[J]. >Journal of Integrative Agriculture, 2023, 22(1): 235-250.
No Suggested Reading articles found!