Scientia Agricultura Sinica ›› 2024, Vol. 57 ›› Issue (20): 3974-3985.doi: 10.3864/j.issn.0578-1752.2024.20.003

• TILLAGE & CULTIVATION·PHYSIOLOGY & BIOCHEMISTRY·AGRICULTURE INFORMATION TECHNOLOGY • Previous Articles     Next Articles

Oat Plant Height Estimation Based on a Dual Output Regression Convolutional Neural Network

ZHANG JianLong1,2(), XING WenWen1, YE ShaoBo1,2, ZHANG Chao1,2, ZHENG DeCong1,2()   

  1. 1 College of Agricultural Engineering, Shanxi Agricultural University, Taigu 030801, Shanxi
    2 Dryland Farm Machinery Key Technology and Equipment Key Laboratory of Shanxi Province, Taigu 030801, Shanxi
  • Received:2024-03-13 Accepted:2024-09-19 Online:2024-10-16 Published:2024-10-24
  • Contact: ZHENG DeCong

RichHTML

10

PDF

100

Abstract:

【Objective】 Oat plant height affects the productivity per plant and the yield per unit area together with planting density. This study explores automatic, real-time, and precise methods for acquiring oat plant height in a field environment, aiming to provide technical references for the automated field management of oat. 【Method】 Firstly, an oat depth image acquisition system was built based on Intel RealSense D435 depth camera and LabVIEW software development platform. Taking Oat ‘Pinyan No. 4’ as the research object, 26 376 modeling data and 2 205 test data were obtained during the whole oat growth process. The average and highest plant height of oats in each depth image were measured with a scale. The quantity of modeling data and test data in each height range of oat plant was relatively balanced. The images were preprocessed by high restoration, grayscale and scaling. Each image was tagged with two labels, one for the average and one for the highest plant height of the oats in the image. Then, based on 8 classical convolutional neural network models, the last layer (classification layer) of each network model was removed, and two fully connected layers with single nodes and no activation function were added to construct the double output regression convolutional neural network estimation model. Mean square error (MSE) was used to evaluate the accuracy of each model in estimating oat plant height. Finally, based on the TensorFlow deep learning platform, Modified EfficientNet V2L was selected as the estimation model by 5-fold cross-validation using the modeling data. 【Result】 The generalization performance of Modified EfficientNet V2L model to estimate oat plant height was investigated using test data not involved in model training. The mean absolute error (MAE), root mean square error (RMSE) and mean relative error (MRE) to estimate oat average plant height were 2.30 cm, 2.90 cm and 4.4%, respectively. Meanwhile, the MAE, RMSE and MRE to estimate highest plant height was 2.24 cm, 2.82 cm and 4.1%, respectively. The average estimated time of the model was 52.14 ms. The accuracy of estimating crop plant height using this method was similar to that of existing methods. However, when estimating crop plant height used this method, once the estimation model was trained, the average and maximum crop plant height could be automatically estimated by inputting the pre-processed crop depth image, and the average estimation time could meet the real-time requirements of crop plant height acquisition. The relative errors in estimating average plant height and maximum plant height of oat showed a general decline trend with the increase of crop plant height. This might be because when crop plant height was low, the estimated results were more affected by soil fluctuation. The results of feature map visualization showed that the model could estimate plant height according to the height and contour of oat in depth image. Finally, an oat plant height estimation system was built based on 2023 Q1 version of LabVIEW software development platform. After depth camera acquiring oat depth images, the system could accurately estimate average and highest oat plant heights in real time without manual intervention, and the average estimation time was less than 0.1 seconds. The system could be used for crops irrigation and fertilization management. It could also be installed on the tractors to control the height of a sprinkler head during spraying, and to adjust the height of a cutting table during harvesting. 【Conclusion】 The depth image and double output regression convolutional neural network could be used to estimate oat plant height, and the accuracy could meet the production demand, so this method provided a basis for field management of oat crops.

Key words: depth image, deep learning, plant height, LabVIEW, oat

Fig. 1

Oat depth image acquisition system"

Fig. 2

Depth image acquisition program"

Fig. 3

Sample of depth image collected during the experiment"

Fig. 4

Depth image preprocessing method"

Table 1

The configuration parameters of the Adam optimization function"

初始学习率
Learning rate		 权重衰减系数
Weight decay		 一阶矩衰减系数
Beta-1		 二阶矩衰减系数
Beta-2		 分母调节参数
Epsilon
0.001 0.01 0.9 0.999 1×10-8

Fig. 5

The average oat plant height obtained by the 5 plants average method and the real average plant height"

Table 2

Information of the models"

改造后的模型
Modified model		 参数量
Parameter		 可训练参数量
Trainable parameter		 模型规模
Model size (MB)		 训练时长
Training time (h)
MobileNet V3 Small 940274 928162 15.2 1.40
NasNet Mobile 4271830 4235092 70.4 5.35
RegNet Y002 2815582 2794734 37.5 2.40
EfficientNet V2 B0 5921874 5861266 75.3 3.20
MobileNet V3 Large 2998274 2973874 39.6 1.50
NasNet Large 84924884 84728216 993.3 11.90
RegNet Y008 5525594 5495706 68.9 3.00
EfficientNet V2 L 117749410 117236834 1372.2 23.30

Table 3

Minimum MSE and MET of each fold during training process for each model"

估测模型
Estimation model		 每折训练中的最低均方差 Minimum MSE of each fold 均方差
MSE		 平均估测时间
MET
Fold 1 Fold 2 Fold 3 Fold 4 Fold 5
Modified MobileNet V3 Small 0.046 0.051 0.075 0.184 0.131 0.098 8.53 ms
Modified NasNet Mobile 0.054 0.038 0.035 0.262 0.063 0.090 46.02 ms
Modified RegNet Y002 0.206 0.262 0.182 0.232 0.109 0.198 13.78 ms
Modified EfficientNet V2 B0 0.036 0.021 0.052 0.064 0.035 0.042 28.59 ms
Modified MobileNet V3 Large 0.034 0.040 0.523 0.016 0.011 0.125 12.05 ms
Modified NasNet Large 0.065 0.007 0.008 0.423 0.231 0.147 50.12 ms
Modified RegNet Y008 0.018 0.013 0.016 0.080 0.009 0.027 14.41 ms
Modified EfficientNet V2 L 0.003 0.001 0.003 0.002 0.001 0.002 52.14 ms

Fig. 6

MSE change in the Modified EfficientNet V2 L during training process"

Fig. 7

The average plant height estimation results of the Modified EfficientNet V2 L on the test data"

Fig. 8

The highest plant height estimation results of the Modified EfficientNet V2 L on the test data"

Fig. 9

Relative error of the Modified EfficientNet V2 L to estimate average plant height on the test data"

Fig. 10

Relative error of the Modified EfficientNet V2 L to estimate highest plant height on the test data"

Fig. 11

Input image and feature maps output from the first convolutional layers"

Fig. 12

Oat plant height estimation system"

[1]
任长忠, 闫金婷, 董锐, 胡新中. 燕麦营养成分、功能特性及其产品的研究进展. 食品工业科技, 2022, 43(12): 438-446.
REN C Z, YAN J T, DONG R, HU X Z. Research progress on oat nutrients, functional properties and related products. Science and Technology of Food Industry, 2022, 43(12): 438-446. (in Chinese)
[2]
张波, 任长忠. 燕麦基因组学与分子育种研究进展. 植物学报, 2022, 57(6): 785-791.

doi: 10.11983/CBB22182
ZHANG B, REN C Z. Advances in oat genomic research and molecular breeding. Chinese Bulletin of Botany, 2022, 57(6): 785-791. (in Chinese)
[3]
王小山, 纪冰沁. 31份燕麦种质主要株型性状比较及遗传分析. 江苏农业科学, 2018, 46(22): 76-79.
WANG X S, JI B Q. Comparison and genetic analysis of main agronomic characters of 31 oat germplasms. Jiangsu Agricultural Sciences, 2018, 46(22): 76-79. (in Chinese)
[4]
肖雅雯. 农业劳动力转移与农业机械化对中国粮食生产的关联影响分析[D]. 南昌: 南昌大学, 2023.
XIAO Y W. Union impact of agricultural labor transfer and agricultural mechanization on China's grain production[D]. Nanchang: Nanchang University, 2023. (in Chinese)
[5]
李道亮, 李震. 无人农场系统分析与发展展望. 农业机械学报, 2020, 51(7): 1-12.
LI D L, LI Z. System analysis and development prospect of unmanned farming. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(7): 1-12. (in Chinese)
[6]
GUPTA C, TEWARI V K, MACHAVARAM R, SHRIVASTAVA P. An image processing approach for measurement of chili plant height and width under field conditions. Journal of the Saudi Society of Agricultural Sciences, 2022, 21(3): 171-179.
[7]
KIM W S, LEE D H, KIM Y J, KIM T, LEE W S, CHOI C H. Stereo-vision-based crop height estimation for agricultural robots. Computers and Electronics in Agriculture, 2021, 181: 105937.
[8]
JIANG Y, LI C Y, PATERSON A H. High throughput phenotyping of cotton plant height using depth images under field conditions. Computers and Electronics in Agriculture, 2016, 130: 57-68.
[9]
ANDÚJAR D, RIBEIRO A, FERNÁNDEZ-QUINTANILLA C, DORADO J. Using depth cameras to extract structural parameters to assess the growth state and yield of cauliflower crops. Computers and Electronics in Agriculture, 2016, 122: 67-73.
[10]
DHAMI H, YU K, XU T, ZHU Q, DHAKAL K, FRIEL J, LI S, TOKEKAR P. Crop height and plot estimation for phenotyping from unmanned aerial vehicles using 3D LiDAR// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.
[11]
YUAN W A, LI J T, BHATTA M, SHI Y Y, BAENZIGER P S, GE Y F. Wheat height estimation using LiDAR in comparison to ultrasonic sensor and UAS. Sensors, 2018, 18(11): 3731.
[12]
ZHOU L F, GU X H, CHENG S, YANG G J, SHU M Y, SUN Q. Analysis of plant height changes of lodged maize using UAV-LiDAR data. Agriculture, 2020, 10(5): 146.
[13]
JIMENEZ-BERNI J A, DEERY D M, ROZAS-LARRAONDO P, CONDON A T G, REBETZKE G J, JAMES R A, BOVILL W D, FURBANK R T, SIRAULT X R R. High throughput determination of plant height, ground cover, and above-ground biomass in wheat with LiDAR. Frontiers in Plant Science, 2018, 9: 237.
[14]
WALTER J D C, EDWARDS J, MCDONALD G, KUCHEL H. Estimating biomass and canopy height with LiDAR for field crop breeding. Frontiers in Plant Science, 2019, 10: 1145.

doi: 10.3389/fpls.2019.01145 pmid: 31611889
[15]
CHANG Y K, ZAMAN Q U, REHMAN T U, FAROOQUE A A, ESAU T, JAMEEL M W. A real-time ultrasonic system to measure wild blueberry plant height during harvesting. Biosystems Engineering, 2017, 157: 35-44.
[16]
MONTAZEAUD G, LANGRUME C, MOINARD S, GOBY C, DUCANCHEZ A, TISSEYRE B, BRUNEL G. Development of a low cost open-source ultrasonic device for plant height measurements. Smart Agricultural Technology, 2021, 1: 100022.
[17]
BRONSON K F, FRENCH A N, CONLEY M M, BARNES E M. Use of an ultrasonic sensor for plant height estimation in irrigated cotton. Agronomy Journal, 2021, 113(2): 2175-2183.
[18]
WANG X, SINGH D, MARLA S, MORRIS G, POLAND J. Field-based high-throughput phenotyping of plant height in Sorghum using different sensing technologies. Plant Methods, 2018, 14(1): 53.
[19]
TAO H L, FENG H K, XU L J, MIAO M K, YANG G J, YANG X D, FAN L L. Estimation of the yield and plant height of winter wheat using UAV-based hyperspectral images. Sensors, 2020, 20(4): 1231.
[20]
ZHANG Z, LOU Y S, MOSES O A, LI R, MA L, LI J. Hyperspectral vegetation indexes to monitor wheat plant height under different sowing conditions. Spectroscopy Letters, 2020, 53(3): 194-206.
[21]
张凤, 张锦水, 段雅鸣, 杨志. 迁移深度卷积神经网络模型秋粮作物泛化识别. 遥感学报, 2024, 28(3): 661-676.
ZHANG F, ZHANG J S, DUAN Y M, YANG Z. Transferring deep convolutional neural network models for generalization mapping of autumn crops. National Remote Sensing Bulletin, 2024, 28(3): 661-676. (in Chinese)
[22]
赵威, 马睿, 王佳, 郭宏杰, 许金普. 基于果穗图像的玉米品种分类识别. 中国农业科技导报, 2023, 25(6): 97-106.
ZHAO W, MA R, WANG J, GUO H J, XU J P. Classification and identification of corn varieties based on ear image. Journal of Agricultural Science and Technology, 2023, 25(6): 97-106. (in Chinese)
[23]
杜甜甜, 南新元, 黄家興, 张文龙, 马志侠. 改进RegNet识别多种农作物病害受害程度. 农业工程学报, 2022, 38(15): 150-158.
DU T T, NAN X Y, HUANG J X, ZHANG W L, MA Z X. Identifying the damage degree of various crop diseases using an improved RegNet. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(15): 150-158. (in Chinese)
[24]
李阳德, 马晓慧, 王骥. 基于轻量级MobileNet V3-YOLOv4的生长期菠萝成熟度分析. 智慧农业(中英文), 2023, 5(2): 35-44.
LI Y D, MA X H, WANG J. Pineapple maturity analysis in natural environment based on MobileNet V3-YOLOv4. Smart Agriculture, 2023, 5(2): 35-44. (in Chinese)

doi: 10.12133/j.smartag.SA202211007
[25]
AGARWAL M, GUPTA S K, BISWAS K K. Development of efficient CNN model for Tomato crop disease identification. Sustainable Computing: Informatics and Systems, 2020, 28: 100407.
[26]
HOWARD A, SANDLER M, CHU G, CHEN L C, CHEN B, TAN M X, WANG W J, ZHU Y K, PANG R M, VASUDEVAN V, LE Q V, ADAM H. Searching for MobileNetV3.2019: 1905.02244. https://arxiv.org/abs/1905.02244v5.
[27]
ZOPH B, VASUDEVAN V, SHLENS J, LE Q V. Learning transferable architectures for scalable image recognition. 2017: 1707.07012. https://arxiv.org/abs/1707.07012v4.
[28]
TAN M X, LE Q V. EfficientNetV2: smaller models and faster training. Computer vision and pattern recognition, 2021. https://arxiv.org/abs/2104.00298.
[29]
RADOSAVOVIC I, KOSARAJU R P, GIRSHICK R, HE K M, DOLLÁR P. Designing network design spaces. 2020: 2003. 13678. https://arxiv.org/abs/2003.13678v1.
[30]
HEADY H F. The measurement and value of plant height in the study of herbaceous vegetation. Ecology, 1957, 38(2): 313.
[31]
PÉREZ-HARGUINDEGUY N, DÍAZ S, GARNIER E, LAVOREL S, POORTER H, JAUREGUIBERRY P, BRET-HARTE M S, CORNWELL W K, CRAINE J M, GURVICH D E, URCELAY C, etc. New handbook for standardised measurement of plant functional traits worldwide. Australian Journal of Botany, 2013, 61(3): 167-234.
[32]
KERAS. Adam, 2023. https://keras.io/api/optimizers/adam/.
[33]
KINGMA D P, BA J L. Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, 2015.
[1] WANG XiaoJun, WANG JinLan, JU ZeLiang, LIANG GuoLing, JIA ZhiFeng, LIU WenHui, MA Xiang, MA JinXiu, LI Wen. Comprehensive Evaluation on Production Performance and Nutritional Quality of Different Varieties of Forage Oat in the Qinghai Lake Area [J]. Scientia Agricultura Sinica, 2024, 57(19): 3730-3742.
[2] LIU DeLong, LI ShiRu, WANG ChuanXing, GUO ShuQing, MA ZhiXiu, WU YongJiang, HAN HuiBing, LI YuJie, ZHANG PanPan, YANG Pu. Phenotypical Variation and Dynamic QTL Mapping of Plant Height in Foxtail Millet at Different Developmental Stages [J]. Scientia Agricultura Sinica, 2024, 57(18): 3533-3550.
[3] WANG YunYun, LI YiNian, CHEN YuLun, DING QiShuo, HE RuiYin. A Method for Testing Phenotype Parameters of Wheat Grains on Spike Based on Improved Mask R-CNN [J]. Scientia Agricultura Sinica, 2024, 57(12): 2322-2335.
[4] CHEN HengGuang, PEI XiaoMeng, XIA YuTing, LIU Jing, MAO DaGan. Effects of Adding Curcumin to Perinatal Diets on Production Performance, Blood Indexes and Gene Expression of Inflammatory Factors in Goats [J]. Scientia Agricultura Sinica, 2024, 57(12): 2483-2496.
[5] MIAO Long, SHU Kuo, HU YanJiao, HUANG Ru, HE GenHua, ZHANG WenMing, WANG XiaoBo, QIU LiJuan. Identification and Gene Mapping of Hard Seededness Mutant Mzp661 in Soybean [J]. Scientia Agricultura Sinica, 2024, 57(11): 2065-2078.
[6] ZHOU WenQi, ZHANG HeTong, HE HaiJun, GONG DianMing, YANG YanZhong, LIU ZhongXiang, LI YongSheng, WANG XiaoJuan, LIAN XiaoRong, ZHOU YuQian, QIU FaZhan. Candidate Gene Localization of ZmDLE1 Gene Regulating Plant Height and Ear Height in Maize [J]. Scientia Agricultura Sinica, 2023, 56(5): 821-837.
[7] WANG Peng, LIU ZiYi, LIU YuFang, CHU MingXing. miR-535 Targets the GAB2 Gene to Promote Goat Granulosa Cell Proliferation Through Activation of the PI3K/AKT Signaling Pathway [J]. Scientia Agricultura Sinica, 2023, 56(23): 4757-4771.
[8] PENG Pai, WANG XiaoLong, MA Lan, ZOU XiaoYang, MA QianYing, ZHANG XinYu, LI XiaoPing, HU XinZhong. Effects of Oat Bran Flour and Tartary Buckwheat Bran Flour on Structure, Cooking Quality and Digestive Characteristics of Wheat Noodles [J]. Scientia Agricultura Sinica, 2023, 56(20): 4102-4114.
[9] LI MianYan, WANG LiXian, ZHAO FuPing. Research Progress on Machine Learning for Genomic Selection in Animals [J]. Scientia Agricultura Sinica, 2023, 56(18): 3682-3692.
[10] CAO Jie, GU YongZhe, HONG HuiLong, WU HaiTao, ZHANG Xia, SUN JianQiang, BAO LiGao, QIU LiJuan. Pigment Identification and Gene Mapping in Red Seed Coat of Soybean [J]. Scientia Agricultura Sinica, 2023, 56(14): 2643-2659.
[11] YAO QiFu, CHEN HuangXin, ZHOU JieGuang, MA RuiYing, DENG Liang, TAN ChenXinYu, SONG JingHan, LÜ JiJuan, MA Jian. QTL Identification and Genetic Analysis of Plant Height in Wheat Based on 16K SNP Array [J]. Scientia Agricultura Sinica, 2023, 56(12): 2237-2248.
[12] ZHOU MingXing, DAI ZiJun, FAN Jun, FU Wei, HAO MingDe. Effect of No-Tillage Combined with Mulching on the Structure and Organic Carbon Content of Aggregates in Heilu Soil of the Weibei Dry Plateau [J]. Scientia Agricultura Sinica, 2023, 56(12): 2329-2340.
[13] BIN Yu, ZHANG Qi, WANG ChunQing, ZHAO XiaoChun, SONG Zhen, ZHOU ChangYong. Screening of the Host Factors Interacting with CP of Citrus Yellow Vein Clearing Virus by Yeast Two-Hybrid System [J]. Scientia Agricultura Sinica, 2023, 56(10): 1881-1892.
[14] LI Qi, YANG ChangHeng, WANG Yong, LIN YaQiu, XIANG Hua, ZHU JiangJiang. Role of FATP1 in Promoting Lipid Deposition in Goat Intramuscular Adipocytes [J]. Scientia Agricultura Sinica, 2023, 56(10): 2007-2020.
[15] LIU Shuo,ZHANG Hui,GAO ZhiYuan,XU JiLi,TIAN Hui. Genetic Variations of Potassium Harvest Index in 437 Wheat Varieties [J]. Scientia Agricultura Sinica, 2022, 55(7): 1284-1300.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备09089781号-30
Copyright 2018 © Editorial office of Scientia Agricultura Sinica
Tel: 86-010-82106279    E-mail: zgnykx@mail.caas.net.cn
Supported by Beijing Magtech Co.Ltd