用于诊断水稻幼穗分化期营养水平的混合CNN-LSTM模型

doi:10.1016/j.jia.2023.05.032

摘要/Abstract

摘要：

氮（N）和钾（K）是水稻生长过程中两种关键的矿质营养元素。准确诊断氮、钾的状况，对水稻在特定生长阶段的合理施肥具有重要意义。因此，我们提出了一种用于在幼穗分化期（EPIS）诊断水稻营养水平的混合模型，它将嵌入注意力机制的卷积神经网络和长短期记忆网络（LSTM）相结合。在为期两年的实验中，该模型在无人机从不同生长阶段的水稻冠层收集的大量序列图像上得到了验证。与 VGG16、AlexNet、GoogleNet、DenseNet 和 inceptionV3 相比，ResNet101 结合 LSTM的模型在黄花占（HHZ，一种籼稻品种）数据集上获得了 83.81% 的最高平均准确率。当在 2021 年的 HHZ 和秀水 134（XS134，一种粳稻品种）数据集上进行测试时，使用 Squeeze-and-Excitation (SE) 增强的 ResNet101-LSTM 模型达到了 85.38% 和 88.38% 的最高准确率，并且通过跨数据集方法，该模型在2022年测试的HHZ和XS134数据集上的平均准确率分别为81.25%和82.50%，表现出良好的泛化能力。我们提出的模型涉及水稻不同生育阶段的动态信息，可以有效地诊断在EPIS 中水稻不同的营养状况，有助于在水稻穗萌发阶段做出合理施肥的实际决策。

Abstract:

Nitrogen (N) and potassium (K) are two key mineral nutrient elements involved in rice growth. Accurate diagnosis of N and K status is very important for the rational application of fertilizers at a specific rice growth stage. Therefore, we propose a hybrid model for diagnosing rice nutrient levels at the early panicle initiation stage (EPIS), which combines a convolutional neural network (CNN) with an attention mechanism and a long short-term memory network (LSTM). The model was validated on a large set of sequential images collected by an unmanned aerial vehicle (UAV) from rice canopies at different growth stages during a two-year experiment. Compared with VGG16, AlexNet, GoogleNet, DenseNet, and inceptionV3, ResNet101 combined with LSTM obtained the highest average accuracy of 83.81% on the dataset of Huanghuazhan (HHZ, an indica cultivar). When tested on the datasets of HHZ and Xiushui 134 (XS134, a japonica rice variety) in 2021, the ResNet101-LSTM model enhanced with the squeeze-and-excitation (SE) block achieved the highest accuracies of 85.38 and 88.38%, respectively. Through the cross-dataset method, the average accuracies on the HHZ and XS134 datasets tested in 2022 were 81.25 and 82.50%, respectively, showing a good generalization. Our proposed model works with the dynamic information of different rice growth stages and can efficiently diagnose different rice nutrient status levels at EPIS, which are helpful for making practical decisions regarding rational fertilization treatments at the panicle initiation stage.

Key words: dynamic model of deep learning , UAV , rice panicle initiation , nutrient level diagnosis , image classification

. 用于诊断水稻幼穗分化期营养水平的混合CNN-LSTM模型[J]. Journal of Integrative Agriculture, 2024, 23(2): 711-723.

Fubing Liao, Xiangqian Feng, Ziqiu Li, Danying Wang, Chunmei Xu, Guang Chu, Hengyu Ma, Qing Yao, Song Chen.

A hybrid CNN-LSTM model for diagnosing rice nutrient levels at the rice panicle initiation stage [J]. Journal of Integrative Agriculture, 2024, 23(2): 711-723.

参考文献

Ali A M, Thind H S, Sharma S, Singh Y. 2015. Site-specific nitrogen management in dry direct-seeded rice using chlorophyll meter and leaf colour chart. Pedosphere, 25, 72–81.

Ballard D H. 1981. Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition, 13, 111–122.

Baral B R, Pande K R, Gaihre Y K, Baral K R, Sah S K, Thapa Y B, Singh U. 2021. Real-time nitrogen management using decision support-tools increases nitrogen use efficiency of rice. Nutrient Cycling in Agroecosystems, 119, 355–368.

Che S G, Zhao B Q, Li Y T, Yuan L, Lin Z A, Hu S W, Shen B. 2016. Nutrient uptake requirements with increasing grain yield for rice in China. Journal of Integrative Agriculture, 15, 907–917.

Chen L S, Wang K. 2014. Diagnosing of rice nitrogen stress based on static scanning technology and image information extraction. Journal of Soil Science and Plant Nutrition, 14, 382–393.

Chlingaryan A, Sukkarieh S, Whelan B. 2018. Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: A review. Computers and Electronics in Agriculture, 151, 61–69.

Condori R H M, Romualdo L M, Bruno O M, de Cerqueira Luz P H. 2017. Comparison between traditional texture methods and deep learning descriptors for detection of nitrogen deficiency in maize crops. In: 2017 Workshop of Computer Vision (WVC). Curran Associates, New York, NY. pp. 7–12.

Coskun D, Britto D T, Kronzucker H J. 2017. The nitrogen-potassium intersection: membranes, metabolism, and mechanism. Plant Cell and Environment, 40, 2029–2041.

Dua M, Makhija D, Mishra P Y L, Manasa P. 2020. A CNN–RNN–LSTM based amalgamation for Alzheimer’s disease detection. Journal of Medical and Biological Engineering, 40, 688–706.

Ghosh M, Swain D K, Jha M K, Tewari V K, Bohra A. 2020. Optimizing chlorophyll meter (SPAD) reading to allow efficient nitrogen use in rice and wheat under rice–wheat cropping system in eastern India. Plant Production Science, 23, 270–285.

Gonzalez-Sanchez A, Frausto-Solis J, Ojeda-Bustamante W. 2014. Predictive ability of machine learning methods for massive crop yield prediction. Spanish Journal of Agricultural Research, 12, 313–328.

Goyal K, Singh N, Jindal S, Kaur R, Goyal A, Awasthi R. 2022. Kjeldahl method. Advanced Techniques of Analytical Chemistry, 1, 105.

Gu Q, Deng J S, Lu C, Shi Y Y, Wang K, Shen Z Q. 2012. Diagnosis of rice nitrogen nutrition based on spectral and shape characteristics of scanning leaves. Journal of Agricultural Machinery, 43, 170–174. (in Chinese)

He K M, Zhang X Y, Ren S Q, Sun J. 2016. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, New York, NY. pp. 770–778.

Hochreiter S, Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9, 1735–1780.

Hou W F, Tränkner M, Lu J W, Yan J Y, Huang S Y, Ren T, Cong R H, Li X K. 2019. Interactive effects of nitrogen and potassium on photosynthesis and photosynthetic nitrogen allocation of rice leaves. BMC Plant Biology, 19, 1–13.

Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. 2017. Densely connected convolutional network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, NJ. pp. 4700–4708.

Huang Y G, Luo D Q, Jiang X H, Li M, Li L J, Ji G M, Li S X, Xu Y. 2021. Effects of different proportions of nitrogen application on grain yield and quality of Yixiangyou 2115. Hybrid Rice, 36, 94–99. (in Chinese)

Kang Y, Nam J, Kim Y, Lee S, Seong D, Jang S, Ryu C. 2021. Assessment of regression models for predicting rice yield and protein content using unmanned aerial vehicle-based multispectral imagery. Remote Sensing, 13, 1508.

Khaki S, Wang L. 2019. Crop yield prediction using deep neural networks. Frontiers in Plant Science, 10, 621.

Krizhevsky A, Sutskever I, Hinton G E. 2012. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60, 84–90.

Kwok N M , Wang D, Jia X, Chen S Y, Fang G, Ha Q P. 2011. Gray world based color correction and intensity preservation for image enhancement. In: 2011 4th International Congress on Image and Signal Processing. IEEE, Piscataway, NJ. pp. 994–998.

Latwal A, Saxena S, Dubey S K, Choudhary K, Sehgal S, Ray S S. 2019. Evaluation of pre-harvest production forecasting of mustard crop in major producing states of India, under fasal project. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, XLII–3/W6, 18–20.

Li H, Hu B, Chu C C. 2017. Nitrogen use efficiency in crops: lessons from Arabidopsis and rice. Journal of Experimental Botany, 68, 2477–2488.

Li L T, Zhang M, Ren T, Li X K, Cong R H, Wu L S, Lu J W. 2015. Diagnosis of N nutrition of rice using digital image processing technique. Journal of Plant Nutrition and Fertilizer, 21, 259–268. (in Chinese)

Li Q, Jin S C, Zang J R, Wang X, Sun Z Z, Li Z Y, Xu S, Ma Q, Su Y J, Guo Q H, Jiang D. 2022. Deciphering the contributions of spectral and structural data to wheat yield estimation from proximal sensing. The Crop Journal, 10, 1334–1345.

Li S X, Feng Z L, Yang B J, Li H, Liao F B, Gao Y F, Liu S H, Tang J, Yao Q. 2022. An intelligent monitoring system of diseases and pests on rice canopy. Frontiers in Plant Science, 13, 972286.

Lin T, Zhong R H, Wang Y D, Xu J F, Jiang H, Xu J L, Ying Y B, Rodriguez L, Ting K C, Li H F. 2020. DeepCropNet: A deep spatial-temporal learning framework for county-level corn yield estimation. Environmental Research Letters, 15, 034016.

Ling Q H, Wang S H, Ding Y F, Li G H. 2017. Re-evaluation of using the color difference between the top 3rd leaf and the 4th leaf as a unified indicator for high-yielding rice. Scientia Agricultura Sinica, 50, 4705–4713. (in Chinese)

Liu H J, Zhang H, Sheng J, Zhang Y F, Guo Z, Zheng J C, Chen J C, Chen L G. 2021. Effects of panicle fertilizer reduction on nitrogen use efficiency of rice under side deep application of basal fertilizer. Chinese Journal of Ecology, 40, 1366–1374. (in Chinese)

Ma P, Yang Z Y, Li Y, Lin D, Sun Y J, Ma J. 2019. Effects of nitrogen reduction in wheat/rape season and nitrogen fertilizer management in rice season on crop yield and nitrogen uptake in crop rotation system. Acta Agriculturae Zhejiangensis, 31, 1769–1778. (in Chinese)

Metropolis N, Ulam S. 1949. The monte carlo method. Journal of the American Statistical Association 44, 335–341.

Panda S S, Ames D P, Panigrahi S. 2010. Application of vegetation indices for agricultural crop yield prediction using neural network techniques. Remote Sensing, 2, 673–696.

Peng S B, Buresh R J, Huang J L, Zhong X H, Zou Y B, Yang J C, Wang G H, Liu Y Y, Hu R F, Tang Q Y, Cui K H, Zhang F S, Dobermann A. 2010. Improving nitrogen fertilization in rice by sitespecific N management. A review. Agronomy for Sustainable Development, 30, 649–656.

Simonyan K, Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations(ICLR). Ithaca, NY. pp.1–14.

Son H H. 2017. Toward a proposed framework for mood recognition using LSTM Recurrent Neuron Network. Procedia Computer Science, 109, 1028–1034.

Sun G J, Liu S H, Luo H L, Feng Z L, Yang B J, Luo J, Tang J, Yao Q, Xu J J. 2022. Intelligent monitoring system of migratory pests based on searchlight trap and machine vision. Frontiers in Plant Science, 13, 897739.

Sun Y Y, Tong C, He S, Wang K, Chen L S. 2018. Identification of nitrogen, phosphorus, and potassium deficiencies based on temporal dynamics of leaf morphology and color. Sustainability, 10, 762.

Sun Z Z, Li Q, Jin S C, Song Y L, Xu S, Wang X, Cai J, Zhou Q, Ge Y, Zhang R N, Zhang J R, Jiang D. 2022. Simultaneous prediction of wheat yield and grain protein content using multitask deep learning from time-series proximal sensing. Plant Phenomics, 2022, 13.

Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. 2015. Going deeper with convolutions. In: 2015 IEEEConference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, NJ. pp. 1–9.

Taghavi Namin S, Esmaeilzadeh M, Najafi M, Brown T B, Borevitz J O. 2018. Deep phenotyping: deep learning for temporal phenotype/genotype classification. Plant Methods, 14, 1–14.

Tran T T, Choi J W, Le T T H, Kim J W. 2019. A comparative study of deep CNN in forecasting and classifying the macronutrient deficiencies on development of tomato plant. Applied Sciences, 9, 1601.

Vitousek P M, Naylor R, Crews T, David M B, Drinkwater L E, Holland E, Johnes P J, Katzenberger J, Martinelli L A, Matson P A, Nziguheba G, Ojima D, Palm C A, Robertson G P, Sanchez P A, Townsend A R, Zhang F S. 2009. Nutrient imbalances in agricultural development. Science, 324, 1519–1520.

Wang L, Chen S S, Li D, Wang C Y, Jiang H, Zheng Q, Peng Z P. 2021. Estimation of paddy rice nitrogen content and accumulation both at leaf and plant levels from UAV hyperspectral imagery. Remote Sensing, 13, 2956.

Wang S W, Cheng X, Zhao Q H, Li X W, Zhang Y, Mani A, Wang R T. 2018. Estimating canopy nitrogen content of rice using hyperspectral reflectance combined with SG-FD-CARS-ELM in cold region. Systems Engineering, 3, 25–34.

Wang Y, Fu L D, Li X, Sui X, Ren H, Lv X H. 2013. Effect of nitrogen application amount on growth development and yield of rice. North Rice, 43, 14–17. (in Chinese)

Weiss K, Khoshgoftaar T M, Wang D D. 2016. A survey of transfer learning. Journal of Big Data, 3, 1–40.

Wood E M, Pidgeon A M, Radeloff V C, Keuler N S. 2012. Image texture as a remotely sensed measure of vegetation structure. Remote Sensing of Environment, 121, 516–526.

Wu G, Peng Y Q , Zhou G Q, Li X L, Zheng Y J, Yan H J. 2020. Recognition method for corn nutrient based on multispectral image and convolutional neural network. Smart Agriculture, 2, 111. (in Chinese)

Xiong J T, Dai S X, Ou J H, Lin X Y, Huang Q H, Yang Z G. 2020. Leaf deficiency symptoms detection method of soybean based on deep learning. Journal of Agricultural Machinery, 51, 195–202. (in Chinese)

Yao Q, Feng J, Tang J, Xu W G, Zhu X H, Yang B J, Lv J, Xie Y Z, Yao B, Wu S Z, Kuai N Y, Wang L J. 2020. Development of an automatic monitoring system for rice light-trap pests based on machine vision. Journal of Integrative Agriculture, 19, 2500–2513.

Yao Q, Li C, Li B, Yi J, Jing T T, Lv B. 2021. Preliminary study on the application of deep learning method in nitrogen nutrition diagnosis of rice. South China Agriculture, 15, 125–129. (in Chinese)

Zhang Z S, Xia J Q, Alfatih A, Song Y, Huang Y J, Sun L Q,Wan G Y, Wang S M, Wang Y P, Hu B H, Zhang G H, Qin P, Li S G, Yu L H, Wu J, Xiang C B. 2022. Rice NIN-LIKE PROTEIN 3 modulates nitrogen use efficiency and grain yield under nitrate-sufficient conditions. Plant Cell and Environment, 45, 1520–1536.

Zheng H B, Zhou M, Zhu Y, Cheng T. 2019. Exploiting the textural information of UAV multispectral imagery to monitor nitrogen status in rice. In: 2019 IEEE International Geoscience and Remote Sensing Symposium. IEEE, Piscataway, NJ. pp. 7251–7253.