Scientia Agricultura Sinica ›› 2019, Vol. 52 ›› Issue (16): 2776-2786.doi: 10.3864/j.issn.0578-1752.2019.16.004

• TILLAGE & CULTIVATION·PHYSIOLOGY & BIOCHEMISTRY·AGRICULTURE INFORMATION TECHNOLOGY • Previous Articles     Next Articles

Multi-View Geometric Reconstruction of Plant Based on Improved Region-Growing Algorithm

XIAO ShunFu,LIU ShengPing(),LI ShiJuan,DU MingZhu,Lü ChunYang,LIU DaZhong,YANG FeiFei,LIU Hang   

  1. Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100081
  • Received:2019-03-30 Accepted:2019-05-15 Online:2019-08-16 Published:2019-08-21
  • Contact: ShengPing LIU E-mail:liushengping@caas.cn

RichHTML

12

PDF

246

Abstract:

【Objective】 Three-dimensional reconstruction of three kinds of plants canopy with different levels of complexity was studied to provide a new method for more accurate acquisition of plants canopy phenotypic characteristics. 【Method】 In this paper, a sequence of pictures of three kinds of plant canopy with different levels of complexity were gotten by using a SLR camera, and dense point clouds of plants were gotten by using three-dimensional reconstruction method. Then, the original scale of dense point clouds of plants was restored, and noisy point of dense point clouds and segmenting plants canopy were filtered by using improved region-growing algorithm with the help of laser scanner, then the accuracy of leaves reconstructed from multi-view geometry method was evaluated from two-dimensional and three-dimensional aspects by manual measurement and laser scanning. Two-dimensional accuracy evaluation was carried out by statistical analysis of actual measured values of leaves length and width and comparing with leaves length and width reconstructed by multi-view geometry method and leaves length and width obtained by laser scanner, respectively. Three-dimensional accuracy was evaluated by using the traditional mesh comparison method Hausdorff distance and Geomagic Qualify software, which was a better accurate 3D accuracy comparison software in industrial meshes comparing. 【Result】 The results showed that the judgment coefficients (R 2) between the phenotypic information of plants leaves and manual measurements were higher than 0.96. The judgment coefficients (R 2) between the phenotypic information of plant leaves obtained by laser scanning and manual measurements were higher than 0.99. The proportion of leaves obtained by multi-view geometric reconstruction and laser scanning was more than 97% in the deviation range of 0-±1 mm. Taking the leaves mesh scanned by laser as a reference, more than 90% of the Hausdorff distance of the leaves mesh reconstructed by multi-view geometry was between 0-2 mm. It was proved that the combination of the multi-view geometric reconstruction method with the improved region-growing algorithm could achieve ideal reconstruction results for plants with different complexity. 【Conclusion】The reconstruction method combining multi-view geometry method with the improved region-growing algorithm proposed in this paper could make up for the shortcomings of region-growing algorithm. It had better segmentation effect on the surface of uneven plants canopy, and was suitable for three-dimensional reconstruction of plants with different complexity. It could provide some reference for breeding research to obtain plant phenotypes.

Key words: multi-view geometry, three-dimensional reconstruction, plant, region-growing algorithm, digital camera

Fig. 1

Workflow of three-dimensional reconstruction and the accuracy evaluation"

Fig. 2

Camera position reference"

Fig. 3

Workflow of generating point clouds based on multi-view geometry"

Fig. 4

Segmentation result of pepper by using region-growing algorithms with different parameters"

Fig. 5

The comparison of results using the region-growing algorithm and its improved algorithm"

Fig. 6

Results of point clouds processing and 3D reconstruction of plants"

Fig. 7

The relationship between leaf position and leaf width and length of pepper, cucumber and tomato"

Table 1

Comparisons of leaf length and leaf width between multi-view geometry reconstruction, laser scanning and manual measurement"

植株
Plant		 叶片参数
Leaf parameter		 决定系数
R12		 均方根误差
RMSE1 (mm)		 决定系数
R22		 均方根误差
RMSE2 (mm)
辣椒
Pepper		 长度 Length 0.997 0.49 0.98 0.96
宽度 Width 0.998 0.23 0.995 0.63
黄瓜
Cucumber		 长度 Length 0.998 0.55 0.98 1.57
宽度 Width 0.999 0.42 0.99 1.56
番茄
Tomato		 长度 Length 0.996 0.38 0.96 1.21
宽度 Width 0.992 0.46 0.96 0.93

Table 2

Percentage of intervals of Hausdorff distance calculated from multi-view geometry method and laser scanning (%)"

植株
Plant		 豪斯多夫距离区间 Intervals of Hausdorff distance (mm)
0-2 0-4 4-6 6-8 8-10
辣椒-1 Pepper-1 96.60 99.46 0.42 0.08 0.04
辣椒-2 Pepper-2 95.22 98.30 1.43 0.21 0.06
黄瓜-1 Cucumber-1 91.68 98.21 1.34 0.43 0.02
黄瓜-2 Cucumber-2 90.24 98.53 0.81 0.61 0.05
番茄-1 Tomato-1 93.86 99.11 0.73 0.09 0.07
番茄-2 Tomato-2 91.53 98.89 1.07 0.03 0.01

Table 3

Percentage of deviation calculated from multi-view geometry method and laser scanning"

植株
Plant		 偏差 Deviation (mm) 标准偏差
Standard deviation (mm)
0-±0.5 0-±1 ±1-±1.5 ±1.5-±2 ±2-±2.5
辣椒-1 Pepper-1 94.90 99.46 0.54 0 0 0.24
辣椒-2 Pepper-2 90.22 97.55 2.3 0.15 0 0.30
黄瓜-1 Cucumber-1 63.34 86.58 8 3.57 1.85 0.72
黄瓜-2 Cucumber-2 62.64 83.91 8.44 4.70 2.95 0.70
番茄-1 Tomato-1 84.60 98.06 1.92 0.02 0 0.37
番茄-2 Tomato-2 82.15 97.00 2.79 0.21 0 0.38

Fig. 8

Visualization of reconstruction quality for plants based on deviation"

[1] 粮农组织, 农发基金, 儿基会, 粮食署, 世卫组织. 2018年世界粮食安全和营养状况:增强气候抵御能力,促进粮食安全和营养. 罗马: 粮农组织, 2018.
Food and Agriculture Organization, International Fund for Agricultural Development, United Nations Children's Fund, The World Food Programme, World Health Organization. World Food Security and Nutrition in 2018: Improving Climate Resistance and Promoting Food Security and Nutrition. Rome: FAO, 2018. (in Chinses)
[2] DAVID H, GOVINDARAJU D R, STIG O . Phenomics: The next challenge. Nature Reviews Genetics, 2010,11(12):855-866.
[3] KLODT M, HERZOG K, TöPFER R, CREMERS D . Field phenotyping of grapevine growth using dense stereo reconstruction. Bmc Bioinformatics, 2015,16(1):1-11.
[4] FURBANK R T, TESTER M . Phenomics-technologies to relieve the phenotyping bottleneck. Trends in Plant Science, 2011,16(12):635-644.
[5] CHEN W T, YEH Y H F, LIU T Y, LIN T T . An automatic plant growth measurement system for plant factory. Ifac Proceedings Volumes, 2013,46(4):323-327.
[6] NGUYEN T T, SLAUGHTER D C, TOWNSLEY B, CARRIEDO L, MALOOF J N, SINHA N . Comparison of structure-from-motion and stereo vision techniques for full in-field 3D reconstruction and phenotyping of plants: An investigation in sunflower//The Proceedings of the 2016 ASABE Annual International Meeting. American Society of Agricultural and Biological Engineers(ASABE), Orlando, Florida,USA, 2016.
[7] DORNBUSCH T, WERNECKE P, DIEPENBROCK W . A method to extract morphological traits of plant organs from 3D point clouds as a database for an architectural plant model. Ecological Modelling, 2007,200(1):119-129.
[8] ELI K, NAOHIKO H, YUKO T, NAOKI Y, NOBUHARU G, NAOKI O, AKIHIKO K, MINAMI M, TETSURO T . Automatic quantification of morphological traits via three-dimensional measurement of Arabidopsis. Plant Journal, 2010,38(2):358-365.
[9] PAULUS S, SCHUMANN H, KUHLMANN H, LéON J . High- precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants. Biosystems Engineering, 2014,121(18):1-11.
[10] AZZARI, GEORGE, GOULDEN, MICHAEL L, RUSU, RADU B . Rapid characterization of vegetation structure with a Microsoft Kinect Sensor. Sensors, 2013,13(2):2384-2398.
[11] M H V D G W A, SONG Y, HORGAN G, POLDER G, DIELEMAN J A, BRINK M C A M, PALLOIX A, EEUWIJK V F A, GLASBEY C . SPICY: Towards automated phenotyping of large pepper plants in the greenhouse. Functional Plant Biology, 2012,39(10/11):870-877.
[12] SONG Y, GLASBEY C A, POLDER G, van der HEIJDEN G W A M . Non-destructive automatic leaf area measurements by combining stereo and time-of-flight images. IEEE Multimed, 2014,8(5):391-403.
[13] YANG M, CUI J S, JEONG E S, CHO S I . Development of high-resolution 3D phenotyping system using depth camera and RGB camera//The Proceedings of the 2017 ASABE Annual International Meeting. American Society of Agricultural and Biological Engineers (ASABE), Spokane, Washington, USA, 2017.
[14] HUI F, ZHU J Y, HU P C, MENG L, ZHU B L, GUO Y, LI B G, MA Y T . Image-based dynamic quantification and high-accuracy 3D evaluation of canopy structure of plant populations. Annals of Botany, 2018,121(5):1079-1088.
[15] 胡鹏程, 郭焱, 李保国, 朱晋宇, 马韫韬 . 基于多视角立体视觉的植株三维重建与精度评估. 农业工程学报, 2015,31(11):209-214.
HU P C, GUO Y, LI B G, ZHU J Y, MA Y T . Three-dimensional reconstruction and its precision evaluation of plant architecture based on multiple view stereo method. Transactions of the Chinese Society of Agricultural Engineering, 2015,31(11):209-214. (in Chinese)
[16] RABBANI T, HEUVEL F A V D, VOSSELMAN G . Segmentation of point clouds using smoothness constraint. Remote Sensing & Spatial Information Sciences, 2006,36(5):248-253.
[17] WU C C. VisualSFM: A visual structure from motion system. , 2011. http://www.cs.washington.edu/homes/ccwu/vsfm
[18] FURUKAWA Y, PONCE J . Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2010,32(8):1362-1376.
[19] RUSU R B, COUSINS S . 3D is here: Point cloud library (PCL)//The Proceedings of the IEEE International Conference on Robotics & Automation. Institute of Electrical and Electronic Engineers(IEEE), Shanghai, China, 2011.
[20] RUSU R B, MARTON Z C, BLODOW N, DOLHA M, BEETZ M . Towards 3D Point cloud based object maps for household environments. Robotics & Autonomous Systems, 2008,56(11):927-941.
[21] DICKERSON M T, DRYSDALE R L S, MCELFRESH S A, WELZL E . Fast greedy triangulation algorithms. Computational Geometry Theory & Applications, 1997,8(2):67-86.
[22] CIGNONI P, ROCCHINI C, SCOPIGNO R . Metro: Measuring error on simplified surfaces. Computer Graphics Forum, 1998,17(2):167-174.
[23] 杰魔软件有限公司. 三维检测软件Geomagic Qualify. 航空制造技术, 2009(20):99-99.
Geomagic Software Co., Ltd. 3D detection software of Geomagic Qualify. Aeronautical Manufacturing Technology, 2009(20):99-99. (in Chinese)
[24] 陈细涛 . 航空叶片原位检测方法与实验研究[D]. 武汉: 华中科技大学, 2012.
CHEN X T . Aero-engine blade in-situ inspection method and its experimental study[D]. WuHan: Huazhong University of Science and Technology, 2012. (in Chinese)
[25] 王秀鹏 . 基于面结构光的航空零部件三维测量和精度检测[D]. 武汉: 华中科技大学, 2012.
WANG X P . The Measurement and accuracy of detection for the aviation parts and components based on structured light 3D[D]. WuHan: Huazhong University of Science and Technology, 2012. (in Chinese)
[26] LEWIN R . Proposal to sequence the human genome stirs debate. Science, 1986,232(4758):1598-1600.
[27] NGUYEN C V, FRIPP J, LOVELL D R, FURBANK R, KUFFNER P, DAILY H, SIRAULT X . 3D scanning system for automatic high- resolution plant phenotyping//The Proceedings of the International Conference on Digital Image Computing: Techniques and Applications. Institute of Electrical and Electronic Engineers(IEEE), Gold Coast,Australia, 2016.
[28] NGUYEN T T, SLAUGHTER D C, TOWNSLEY B T, CARRIEDO L, MALOOF J N, SINHA N . In-field plant phenotyping using multi- view reconstruction: An investigation in eggplant//The Proceedings of the International Conference on Precison Agriculture. The International Society of Precision Agriculture, St. Louis, Missouri, USA, 2016.
[29] GALLIANI S, LASINGER K, SCHINDLER K . Massively parallel multiview stereopsis by surface normal diffusion//The Proceedings of the International Conference on Computer Vision(ICCV). Institute of Electrical and Electronic Engineers(IEEE), Santiago,Chile, 2015.
[30] JI M Q, GALL J, ZHENG H T, LIU Y B, LU F . SurfaceNet: An end-to-end 3D neural network for multiview stereopsis//The Proceedings of the International Conference on Computer Vision (ICCV). Institute of Electrical and Electronic Engineers(IEEE), Venice,Italy, 2017.
[31] YAO Y, LUO Z, LI S, TIAN F, LONG Q . MVSNet: Depth inference for unstructured multi-view stereo//The Proceedings of the European Conference on Computer Vision(ECCV). Institute of Electrical and Electronic Engineers(IEEE), Munich,Germany, 2018.
[1] ZHANG XiaoLi, TAO Wei, GAO GuoQing, CHEN Lei, GUO Hui, ZHANG Hua, TANG MaoYan, LIANG TianFeng. Effects of Direct Seeding Cultivation Method on Growth Stage, Lodging Resistance and Yield Benefit of Double-Cropping Early Rice [J]. Scientia Agricultura Sinica, 2023, 56(2): 249-263.
[2] WANG XuanDong, SONG Zhen, LAN HeTing, JIANG YingZi, QI WenJie, LIU XiaoYang, JIANG DongHua. Isolation of Dominant Actinomycetes from Soil of Waxberry Orchards and Its Disease Prevention and Growth-Promotion Function [J]. Scientia Agricultura Sinica, 2023, 56(2): 275-286.
[3] WANG CaiXiang,YUAN WenMin,LIU JuanJuan,XIE XiaoYu,MA Qi,JU JiSheng,CHEN Da,WANG Ning,FENG KeYun,SU JunJi. Comprehensive Evaluation and Breeding Evolution of Early Maturing Upland Cotton Varieties in the Northwest Inland of China [J]. Scientia Agricultura Sinica, 2023, 56(1): 1-16.
[4] LIN XinYing,WANG PengJie,YANG RuXing,ZHENG YuCheng,CHEN XiaoMin,ZHANG Lei,SHAO ShuXian,YE NaiXing. The Albino Mechanism of a New High Theanine Tea Cultivar Fuhuang 1 [J]. Scientia Agricultura Sinica, 2022, 55(9): 1831-1845.
[5] LIU Shuo,ZHANG Hui,GAO ZhiYuan,XU JiLi,TIAN Hui. Genetic Variations of Potassium Harvest Index in 437 Wheat Varieties [J]. Scientia Agricultura Sinica, 2022, 55(7): 1284-1300.
[6] WANG YaLiang,ZHU DeFeng,CHEN RuoXia,FANG WenYing,WANG JingQing,XIANG Jing,CHEN HuiZhe,ZHANG YuPing,CHEN JiangHua. Beneficial Effects of Precision Drill Sowing with Low Seeding Rates in Machine Transplanting for Hybrid Rice to Improve Population Uniformity and Yield [J]. Scientia Agricultura Sinica, 2022, 55(4): 666-679.
[7] HU ChaoYue, WANG FengTao, LANG XiaoWei, FENG Jing, LI JunKai, LIN RuiMing, YAO XiaoBo. Resistance Analyses on Wheat Stripe Rust Resistance Genes to the Predominant Races of Puccinia striiformis f. sp. tritici in China [J]. Scientia Agricultura Sinica, 2022, 55(3): 491-502.
[8] XIANG YuTing, WANG XiaoLong, HU XinZhong, REN ChangZhong, GUO LaiChun, LI Lu. Lipase Activity Difference of Oat Varieties and Prediction of Low Lipase Activity Variety with High Quality [J]. Scientia Agricultura Sinica, 2022, 55(21): 4104-4117.
[9] GUO BaoWei,TANG Chuang,WANG Yan,CAI JiaXin,TANG Jian,ZHOU Miao,JING Xiu,ZHANG HongCheng,XU Ke,HU YaJie,XING ZhiPeng,LI GuoHui,CHEN Heng. Effects of Two Mechanical Planting Methods on the Yield and Quality of High-Quality Late Indica Rice [J]. Scientia Agricultura Sinica, 2022, 55(20): 3910-3925.
[10] MA Xiao,CHEN PengFei. Improvement of Row Detection Method Before Wheat Canopy Closure Using Multispectral Images of UAV Image [J]. Scientia Agricultura Sinica, 2022, 55(20): 3926-3938.
[11] SUN BaoJuan,WANG Rui,SUN GuangWen,WANG YiKui,LI Tao,GONG Chao,HENG Zhou,YOU Qian,LI ZhiLiang. Transcriptome and Metabolome Integrated Analysis of Epistatic Genetics Effects on Eggplant Peel Color [J]. Scientia Agricultura Sinica, 2022, 55(20): 3997-4010.
[12] LI YangMei,LIU Xin,JIA MengHan,TONG YuXin. Tipburn Injury and Nutritional Quality of Lettuce Plants as Affected by Humidity Control During the Light Period in A Plant Factory [J]. Scientia Agricultura Sinica, 2022, 55(20): 4011-4019.
[13] LIU Xin,ZHANG YaHong,YUAN Miao,DANG ShiZhuo,ZHOU Juan. Transcriptome Analysis During Flower Bud Differentiation of Red Globe Grape [J]. Scientia Agricultura Sinica, 2022, 55(20): 4020-4035.
[14] GENG WenJie,LI Bin,REN BaiZhao,ZHAO Bin,LIU Peng,ZHANG JiWang. Regulation Mechanism of Planting Density and Spraying Ethephon on Lignin Metabolism and Lodging Resistance of Summer Maize [J]. Scientia Agricultura Sinica, 2022, 55(2): 307-319.
[15] SHA YueXia, HUANG ZeYang, MA Rui. Control Efficacy of Pseudomonas alcaliphila Strain Ej2 Against Rice Blast and Its Effect on Endogenous Hormones in Rice [J]. Scientia Agricultura Sinica, 2022, 55(2): 320-328.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备09089781号-30
Copyright 2018 © Editorial office of Scientia Agricultura Sinica
Tel: 86-010-82106279    E-mail: zgnykx@mail.caas.net.cn
Supported by Beijing Magtech Co.Ltd