改进区域增长算法的植株多视图几何重建

doi:10.3864/j.issn.0578-1752.2019.16.004

摘要/Abstract

摘要：

【目的】通过研究3种不同复杂程度植株冠层的三维重建,为更加精准获取植株冠层表型参数提供新方法。【方法】本文首先用单反相机获取3种不同复杂程度植株冠层图片序列,通过三维重建得到各植株稠密点云;随后还原植株点云原始尺度,过滤稠密点云中的噪声,再使用改进区域增长算法分割植株点云冠层;之后借助激光扫描仪,利用手动测量和激光扫描方法分别从二维和三维两个方面对多视图几何重建的叶片进行精度评价,二维精度评价为叶片长宽的实际测量值分别与激光扫描仪获取的叶片的长宽值和多视图几何重建叶片的长宽值进行统计分析,三维精度评价使用传统的网格对比方法豪斯多夫距离与更加精准的工业级网格3D精度对比检测软件Geomagic Qualify。【结果】多视图几何重建的植株叶片表型信息与手动测量值间的判定系数（R ²）均高于0.96,激光扫描方法获取的植株叶片表型信息与手动测量值间的判定系数（R ²）均高0.99;多视图几何重建的叶片与激光扫描得到的叶片在0—±1mm偏差范围内的比例大部分达到97%以上;以激光扫描的叶片网格为参考,多视图几何重建的叶片网格的豪斯多夫距离90%以上分布在0—2 mm。本研究的多视图几何重建方法与改进区域增长算法相结合能对不同复杂程度的植株取得比较理想的重建结果。【结论】本文提出的多视图几何方法与改进区域增长算法相结合的重建方法可以弥补区域增长算法的不足,对表面不平滑的植株冠层具有更好的分割效果,适合不同复杂程度植株三维重建,为育种研究获取植株表型提供一定的参考。

关键词: 多视图几何, 三维重建, 植株, 区域增长算法, 数码相机

Abstract:

【Objective】 Three-dimensional reconstruction of three kinds of plants canopy with different levels of complexity was studied to provide a new method for more accurate acquisition of plants canopy phenotypic characteristics. 【Method】 In this paper, a sequence of pictures of three kinds of plant canopy with different levels of complexity were gotten by using a SLR camera, and dense point clouds of plants were gotten by using three-dimensional reconstruction method. Then, the original scale of dense point clouds of plants was restored, and noisy point of dense point clouds and segmenting plants canopy were filtered by using improved region-growing algorithm with the help of laser scanner, then the accuracy of leaves reconstructed from multi-view geometry method was evaluated from two-dimensional and three-dimensional aspects by manual measurement and laser scanning. Two-dimensional accuracy evaluation was carried out by statistical analysis of actual measured values of leaves length and width and comparing with leaves length and width reconstructed by multi-view geometry method and leaves length and width obtained by laser scanner, respectively. Three-dimensional accuracy was evaluated by using the traditional mesh comparison method Hausdorff distance and Geomagic Qualify software, which was a better accurate 3D accuracy comparison software in industrial meshes comparing. 【Result】 The results showed that the judgment coefficients (R ²) between the phenotypic information of plants leaves and manual measurements were higher than 0.96. The judgment coefficients (R ²) between the phenotypic information of plant leaves obtained by laser scanning and manual measurements were higher than 0.99. The proportion of leaves obtained by multi-view geometric reconstruction and laser scanning was more than 97% in the deviation range of 0-±1 mm. Taking the leaves mesh scanned by laser as a reference, more than 90% of the Hausdorff distance of the leaves mesh reconstructed by multi-view geometry was between 0-2 mm. It was proved that the combination of the multi-view geometric reconstruction method with the improved region-growing algorithm could achieve ideal reconstruction results for plants with different complexity. 【Conclusion】The reconstruction method combining multi-view geometry method with the improved region-growing algorithm proposed in this paper could make up for the shortcomings of region-growing algorithm. It had better segmentation effect on the surface of uneven plants canopy, and was suitable for three-dimensional reconstruction of plants with different complexity. It could provide some reference for breeding research to obtain plant phenotypes.

Key words: multi-view geometry, three-dimensional reconstruction, plant, region-growing algorithm, digital camera

肖顺夫, 刘升平, 李世娟, 杜鸣竹, 吕纯阳, 刘大众, 杨菲菲, 刘航. 改进区域增长算法的植株多视图几何重建[J]. 中国农业科学, 2019, 52(16): 2776-2786.

XIAO ShunFu, LIU ShengPing, LI ShiJuan, DU MingZhu, Lü ChunYang, LIU DaZhong, YANG FeiFei, LIU Hang. Multi-View Geometric Reconstruction of Plant Based on Improved Region-Growing Algorithm[J]. Scientia Agricultura Sinica, 2019, 52(16): 2776-2786.

0
/ / 推荐

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: https://www.chinaagrisci.com/CN/10.3864/j.issn.0578-1752.2019.16.004

https://www.chinaagrisci.com/CN/Y2019/V52/I16/2776

图/表 11

图1

图2

图3

图4

图5

图6

图7

表1

表2

表3

图8

参考文献 31

[1]	粮农组织, 农发基金, 儿基会, 粮食署, 世卫组织. 2018年世界粮食安全和营养状况:增强气候抵御能力,促进粮食安全和营养. 罗马: 粮农组织, 2018.
	Food and Agriculture Organization, International Fund for Agricultural Development, United Nations Children's Fund, The World Food Programme, World Health Organization. World Food Security and Nutrition in 2018: Improving Climate Resistance and Promoting Food Security and Nutrition. Rome: FAO, 2018. (in Chinses)
[2]	DAVID H, GOVINDARAJU D R, STIG O . Phenomics: The next challenge. Nature Reviews Genetics, 2010,11(12):855-866.
[3]	KLODT M, HERZOG K, TöPFER R, CREMERS D . Field phenotyping of grapevine growth using dense stereo reconstruction. Bmc Bioinformatics, 2015,16(1):1-11.
[4]	FURBANK R T, TESTER M . Phenomics-technologies to relieve the phenotyping bottleneck. Trends in Plant Science, 2011,16(12):635-644.
[5]	CHEN W T, YEH Y H F, LIU T Y, LIN T T . An automatic plant growth measurement system for plant factory. Ifac Proceedings Volumes, 2013,46(4):323-327.
[6]	NGUYEN T T, SLAUGHTER D C, TOWNSLEY B, CARRIEDO L, MALOOF J N, SINHA N . Comparison of structure-from-motion and stereo vision techniques for full in-field 3D reconstruction and phenotyping of plants: An investigation in sunflower//The Proceedings of the 2016 ASABE Annual International Meeting. American Society of Agricultural and Biological Engineers(ASABE), Orlando, Florida,USA, 2016.
[7]	DORNBUSCH T, WERNECKE P, DIEPENBROCK W . A method to extract morphological traits of plant organs from 3D point clouds as a database for an architectural plant model. Ecological Modelling, 2007,200(1):119-129.
[8]	ELI K, NAOHIKO H, YUKO T, NAOKI Y, NOBUHARU G, NAOKI O, AKIHIKO K, MINAMI M, TETSURO T . Automatic quantification of morphological traits via three-dimensional measurement of Arabidopsis. Plant Journal, 2010,38(2):358-365.
[9]	PAULUS S, SCHUMANN H, KUHLMANN H, LéON J . High- precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants. Biosystems Engineering, 2014,121(18):1-11.
[10]	AZZARI, GEORGE, GOULDEN, MICHAEL L, RUSU, RADU B . Rapid characterization of vegetation structure with a Microsoft Kinect Sensor. Sensors, 2013,13(2):2384-2398.
[11]	M H V D G W A, SONG Y, HORGAN G, POLDER G, DIELEMAN J A, BRINK M C A M, PALLOIX A, EEUWIJK V F A, GLASBEY C . SPICY: Towards automated phenotyping of large pepper plants in the greenhouse. Functional Plant Biology, 2012,39(10/11):870-877.
[12]	SONG Y, GLASBEY C A, POLDER G, van der HEIJDEN G W A M . Non-destructive automatic leaf area measurements by combining stereo and time-of-flight images. IEEE Multimed, 2014,8(5):391-403.
[13]	YANG M, CUI J S, JEONG E S, CHO S I . Development of high-resolution 3D phenotyping system using depth camera and RGB camera//The Proceedings of the 2017 ASABE Annual International Meeting. American Society of Agricultural and Biological Engineers (ASABE), Spokane, Washington, USA, 2017.
[14]	HUI F, ZHU J Y, HU P C, MENG L, ZHU B L, GUO Y, LI B G, MA Y T . Image-based dynamic quantification and high-accuracy 3D evaluation of canopy structure of plant populations. Annals of Botany, 2018,121(5):1079-1088.
[15]	胡鹏程, 郭焱, 李保国, 朱晋宇, 马韫韬 . 基于多视角立体视觉的植株三维重建与精度评估. 农业工程学报, 2015,31(11):209-214.
	HU P C, GUO Y, LI B G, ZHU J Y, MA Y T . Three-dimensional reconstruction and its precision evaluation of plant architecture based on multiple view stereo method. Transactions of the Chinese Society of Agricultural Engineering, 2015,31(11):209-214. (in Chinese)
[16]	RABBANI T, HEUVEL F A V D, VOSSELMAN G . Segmentation of point clouds using smoothness constraint. Remote Sensing & Spatial Information Sciences, 2006,36(5):248-253.
[17]	WU C C. VisualSFM: A visual structure from motion system. , 2011. http://www.cs.washington.edu/homes/ccwu/vsfm
[18]	FURUKAWA Y, PONCE J . Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2010,32(8):1362-1376.
[19]	RUSU R B, COUSINS S . 3D is here: Point cloud library (PCL)//The Proceedings of the IEEE International Conference on Robotics & Automation. Institute of Electrical and Electronic Engineers(IEEE), Shanghai, China, 2011.
[20]	RUSU R B, MARTON Z C, BLODOW N, DOLHA M, BEETZ M . Towards 3D Point cloud based object maps for household environments. Robotics & Autonomous Systems, 2008,56(11):927-941.
[21]	DICKERSON M T, DRYSDALE R L S, MCELFRESH S A, WELZL E . Fast greedy triangulation algorithms. Computational Geometry Theory & Applications, 1997,8(2):67-86.
[22]	CIGNONI P, ROCCHINI C, SCOPIGNO R . Metro: Measuring error on simplified surfaces. Computer Graphics Forum, 1998,17(2):167-174.
[23]	杰魔软件有限公司. 三维检测软件Geomagic Qualify. 航空制造技术, 2009(20):99-99.
	Geomagic Software Co., Ltd. 3D detection software of Geomagic Qualify. Aeronautical Manufacturing Technology, 2009(20):99-99. (in Chinese)
[24]	陈细涛 . 航空叶片原位检测方法与实验研究[D]. 武汉: 华中科技大学, 2012.
	CHEN X T . Aero-engine blade in-situ inspection method and its experimental study[D]. WuHan: Huazhong University of Science and Technology, 2012. (in Chinese)
[25]	王秀鹏 . 基于面结构光的航空零部件三维测量和精度检测[D]. 武汉: 华中科技大学, 2012.
	WANG X P . The Measurement and accuracy of detection for the aviation parts and components based on structured light 3D[D]. WuHan: Huazhong University of Science and Technology, 2012. (in Chinese)
[26]	LEWIN R . Proposal to sequence the human genome stirs debate. Science, 1986,232(4758):1598-1600.
[27]	NGUYEN C V, FRIPP J, LOVELL D R, FURBANK R, KUFFNER P, DAILY H, SIRAULT X . 3D scanning system for automatic high- resolution plant phenotyping//The Proceedings of the International Conference on Digital Image Computing: Techniques and Applications. Institute of Electrical and Electronic Engineers(IEEE), Gold Coast,Australia, 2016.
[28]	NGUYEN T T, SLAUGHTER D C, TOWNSLEY B T, CARRIEDO L, MALOOF J N, SINHA N . In-field plant phenotyping using multi- view reconstruction: An investigation in eggplant//The Proceedings of the International Conference on Precison Agriculture. The International Society of Precision Agriculture, St. Louis, Missouri, USA, 2016.
[29]	GALLIANI S, LASINGER K, SCHINDLER K . Massively parallel multiview stereopsis by surface normal diffusion//The Proceedings of the International Conference on Computer Vision(ICCV). Institute of Electrical and Electronic Engineers(IEEE), Santiago,Chile, 2015.
[30]	JI M Q, GALL J, ZHENG H T, LIU Y B, LU F . SurfaceNet: An end-to-end 3D neural network for multiview stereopsis//The Proceedings of the International Conference on Computer Vision (ICCV). Institute of Electrical and Electronic Engineers(IEEE), Venice,Italy, 2017.
[31]	YAO Y, LUO Z, LI S, TIAN F, LONG Q . MVSNet: Depth inference for unstructured multi-view stereo//The Proceedings of the European Conference on Computer Vision(ECCV). Institute of Electrical and Electronic Engineers(IEEE), Munich,Germany, 2018.

植株 Plant	叶片参数 Leaf parameter	决定系数 R₁²	均方根误差 RMSE₁(mm)	决定系数 R₂²	均方根误差 RMSE₂(mm)
辣椒 Pepper	长度 Length	0.997	0.49	0.98	0.96
辣椒 Pepper	宽度 Width	0.998	0.23	0.995	0.63
黄瓜 Cucumber	长度 Length	0.998	0.55	0.98	1.57
黄瓜 Cucumber	宽度 Width	0.999	0.42	0.99	1.56
番茄 Tomato	长度 Length	0.996	0.38	0.96	1.21
番茄 Tomato	宽度 Width	0.992	0.46	0.96	0.93

植株 Plant	豪斯多夫距离区间 Intervals of Hausdorff distance (mm)
植株 Plant	0-2	0-4	4-6	6-8	8-10
辣椒-1 Pepper-1	96.60	99.46	0.42	0.08	0.04
辣椒-2 Pepper-2	95.22	98.30	1.43	0.21	0.06
黄瓜-1 Cucumber-1	91.68	98.21	1.34	0.43	0.02
黄瓜-2 Cucumber-2	90.24	98.53	0.81	0.61	0.05
番茄-1 Tomato-1	93.86	99.11	0.73	0.09	0.07
番茄-2 Tomato-2	91.53	98.89	1.07	0.03	0.01

植株 Plant	偏差 Deviation (mm)					标准偏差 Standard deviation (mm)
植株 Plant	0-±0.5	0-±1	±1-±1.5	±1.5-±2	±2-±2.5	标准偏差 Standard deviation (mm)
辣椒-1 Pepper-1	94.90	99.46	0.54	0	0	0.24
辣椒-2 Pepper-2	90.22	97.55	2.3	0.15	0	0.30
黄瓜-1 Cucumber-1	63.34	86.58	8	3.57	1.85	0.72
黄瓜-2 Cucumber-2	62.64	83.91	8.44	4.70	2.95	0.70
番茄-1 Tomato-1	84.60	98.06	1.92	0.02	0	0.37
番茄-2 Tomato-2	82.15	97.00	2.79	0.21	0	0.38