关键词: 立体图像, 深度学习, 株高, LabVIEW, 燕麦

Abstract:

【Objective】 Oat plant height affects the productivity per plant and the yield per unit area together with planting density. This study explores automatic, real-time, and precise methods for acquiring oat plant height in a field environment, aiming to provide technical references for the automated field management of oat. 【Method】 Firstly, an oat depth image acquisition system was built based on Intel RealSense D435 depth camera and LabVIEW software development platform. Taking Oat ‘Pinyan No. 4’ as the research object, 26 376 modeling data and 2 205 test data were obtained during the whole oat growth process. The average and highest plant height of oats in each depth image were measured with a scale. The quantity of modeling data and test data in each height range of oat plant was relatively balanced. The images were preprocessed by high restoration, grayscale and scaling. Each image was tagged with two labels, one for the average and one for the highest plant height of the oats in the image. Then, based on 8 classical convolutional neural network models, the last layer (classification layer) of each network model was removed, and two fully connected layers with single nodes and no activation function were added to construct the double output regression convolutional neural network estimation model. Mean square error (MSE) was used to evaluate the accuracy of each model in estimating oat plant height. Finally, based on the TensorFlow deep learning platform, Modified EfficientNet V2L was selected as the estimation model by 5-fold cross-validation using the modeling data. 【Result】 The generalization performance of Modified EfficientNet V2L model to estimate oat plant height was investigated using test data not involved in model training. The mean absolute error (MAE), root mean square error (RMSE) and mean relative error (MRE) to estimate oat average plant height were 2.30 cm, 2.90 cm and 4.4%, respectively. Meanwhile, the MAE, RMSE and MRE to estimate highest plant height was 2.24 cm, 2.82 cm and 4.1%, respectively. The average estimated time of the model was 52.14 ms. The accuracy of estimating crop plant height using this method was similar to that of existing methods. However, when estimating crop plant height used this method, once the estimation model was trained, the average and maximum crop plant height could be automatically estimated by inputting the pre-processed crop depth image, and the average estimation time could meet the real-time requirements of crop plant height acquisition. The relative errors in estimating average plant height and maximum plant height of oat showed a general decline trend with the increase of crop plant height. This might be because when crop plant height was low, the estimated results were more affected by soil fluctuation. The results of feature map visualization showed that the model could estimate plant height according to the height and contour of oat in depth image. Finally, an oat plant height estimation system was built based on 2023 Q1 version of LabVIEW software development platform. After depth camera acquiring oat depth images, the system could accurately estimate average and highest oat plant heights in real time without manual intervention, and the average estimation time was less than 0.1 seconds. The system could be used for crops irrigation and fertilization management. It could also be installed on the tractors to control the height of a sprinkler head during spraying, and to adjust the height of a cutting table during harvesting. 【Conclusion】 The depth image and double output regression convolutional neural network could be used to estimate oat plant height, and the accuracy could meet the production demand, so this method provided a basis for field management of oat crops.

Key words: depth image, deep learning, plant height, LabVIEW, oat