Please wait a minute...
Journal of Integrative Agriculture
Advanced Online Publication | Current Issue | Archive | Adv Search
Promote computer vision applications in pig farming scenarios: high-quality dataset, fundamental models, and comparable performance
Jiangong Li1, 2*#, Xiaodan Hu3*, Ana Lucic4, Yiqi Wu1, Isabella C.F.S. Condotta5, Ryan N. Dilger5, Narendra Ahuja3, Angela R. Green-Miller6

1State Key Laboratory of Animal Nutrition and Feeding, College of Animal Science and Technology, China Agricultural University, Beijing, 100193, China

2Key Laboratory of Equipment and Informatization in Environment Controlled Agriculture/Ministry of Agriculture and Rural Affairs, College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China

3Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana 61801, United States

4Applied Research Institute, University of Illinois at Urbana-Champaign, Champaign 61820, United States

5Department of Animal Sciences, University of Illinois at Urbana-Champaign, Urbana 61801, United States 

6Department of Agricultural and Biological Engineering, University of Illinois at Urbana-Champaign, Urbana 61801, United States

Download:  PDF in ScienceDirect  
Export:  BibTeX | EndNote (RIS)      
摘要  

机器视觉被广泛认为是动物精准管理领域中的核心应用技术。最新的研究表明,通过视频监控系统和机器视觉算法有望改善猪的健康和福利。然而,缺乏基准数据集和稳健的识别算法限制了机器视觉在实际生产中的应用。本研究旨在通过引入一个通用数据集(PigLife)、比较常见的机器视觉算法的基准性能、讨论不同视觉模型开发的方法和效果,来弥合算法研发与实际应用之间的差距。PigLife数据集包含了现代猪生产工艺(繁殖和妊娠、产仔至断奶、断奶与保育、育肥至出栏)中常规生产场景的视频片段和图像(38个短视频片段、2K图像帧、22K猪实例)。本研究利用PiLife 数据集,训练并对比了三种常规猪只检测算法(Faster R-CNN、RetinaNet、TridentNet)和三种分常规猪只分割算法(Mask R-CNN、MViTv2、Point-Rend)。此外,本研究评估了通过预训练模型的微调(YOLO8-m、Faster-RCNN-r50)和零样本模型(CLIP-SAM、Grouddino-HQSAM)的无训练两种方式,来加快机器视觉开发的速度。通过本研究的分析与讨论,展示了基准数据集在评估算法稳健性和识别各种算法面临的困难和挑战中的必要性;表明从预训练算法或零样本模型开发计算机视觉模型表现出更好的性能和更快的过程,可以极大的降低针对猪养殖场景开发高性能机器视觉产品的成本和难度。



Abstract  

Computer vision is widely recognized as an influential technology in the field of precision management of animals. Emerging studies have demonstrated the potential to improve pig health and welfare through animal surveillance systems and computer vision (CV) algorithms. However, the lack of benchmark datasets and robust fundamental algorithms restrict CV applications for the commercial use. This study aims to bridge the gap between technology development and commercial applications in pig farming scenarios by introducing a general-purpose dataset (PigLife), comparing benchmark performances of foundational CV algorithms and model development workflows. The PigLife dataset contains video clips and images (38 short video clips, 2K image frames, 22K pig instances) across most pig production phases in a typical commercial pig farm: Breeding and Gestation, Farrow to Wean, Weaning & Nursery, and Growth to Finish. Three detection algorithms (Faster R-CNN, RetinaNet, TridentNet) and three segmentation algorithms (Mask R-CNN, MViTv2, Point-Rend) were trained on the PigLife dataset from scratch. Fine-tuning of pre-trained models (YOLO8-m, Faster-RCNN-r50) and no-training from zero-shot models (CLIP-SAM, Grouddino-HQSAM) were also evaluated to suggest faster CV development workflows for commercial applications in pig farming. This study emphasizes the necessity of a benchmark dataset for evaluating the robustness of algorithms and identifying the remaining difficulties and challenges across various algorithms. Furthermore, developing CV models from pre-trained algorithms or zero-shot models showed better performance and a faster process, which could reduce barriers when developing high-performance CV products in pig production industry

Keywords:  pig life       pig recognition              benchmark performance              transfer learning              zero-shot algorithm  
Online: 22 August 2024  
Fund: 

This research received support from the Key Laboratory of Equipment and Informatization in Environment Controlled Agriculture, Ministry of Agriculture and Rural Affairs, China (2011NYZD2204); Chinese Universities Scientific Fund (2024TC097); the Agriculture and Food Research Initiative of Unite States (AFRI, 2020-67021-32799); and the USDA National Institute of Food and Agriculture (project accession 1024178).

About author:  Xiaodan Hu, Tel: +1-2179798149, E-mail: xiaodan8@illinois.edu; #Correspondence Jiangong Li, Tel: +86-17610958030, E-mail: jli153@cau.edu.cn *Authors contributed equally

Cite this article: 

Jiangong Li, Xiaodan Hu, Ana Lucic, Yiqi Wu, Isabella C.F.S. Condotta, Ryan N. Dilger, Narendra Ahuja, Angela R. Green-Miller. 2024. Promote computer vision applications in pig farming scenarios: high-quality dataset, fundamental models, and comparable performance. Journal of Integrative Agriculture, Doi:10.1016/j.jia.2024.08.014

Benjamin M, Yik S. 2019. Precision livestock farming in swine welfare: a review for swine practitioners. Animals, 9, 133.

Berckmans D. 2017. General introduction to precision livestock farming. Animal Frontiers, 7, 6-11.

Bergamini L, Pini S, Simoni A, Vezzani R, Calderara S, Eath R B, Fisher R B.2021. Extracting accurate long-term behavior changes from a large pig dataset. 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2021, 524-533.

Chen C, Zhu W, Oczak M, Maschat K, Baumgartner J, Larsen M L V, Norton T. 2020. A computer vision approach for recognition of the engagement of pigs with different enrichment objects. Computers and electronics in agriculture, 175, 105580.

Fang Y, Wang W, Xie B, Sun Q, Wu L, Wang X, Huang T, Wang X, Cao Y.2023. Eva: Exploring the limits of masked visual representation learning at scale. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19358-19369.

Fernandes A F A, Dórea J R R, Rosa G J D M. 2020. Image analysis and computer vision applications in animal sciences: an overview. Frontiers in Veterinary Science, 7, 551269.

Gebru T, Morgenstern J, Vecchione B, Vaughan J W, Wallach H, Iii H D, Crawford K. 2021. Datasheets for datasets. Communications of the ACM, 64, 86-92.

He Y, Tiezzi F, Howard J, Maltecca C. 2021. Predicting body weight in growing pigs from feeding behavior data using machine learning algorithms. Computers and electronics in agriculture, 184, 106085.

Ke L, Ye M, Danelljan M, Tai Y-W, Tang C-K, Yu F. 2024. Segment anything in high quality. Advances in Neural Information Processing Systems, 36,

Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg A C, Lo W-Y.2023. Segment anything. Proceedings of the IEEE/CVF International Conference on Computer Vision, 4015-4026.

Kirillov A, Wu Y, He K, Girshick R.2020. Pointrend: Image segmentation as rendering. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9799-9808.

Koger B, Deshpande A, Kerby J T, Graving J M, Costelloe B R, Couzin I D. 2023. Quantifying the movement, behaviour and environmental context of group‐living animals using drones and computer vision. Journal of Animal Ecology, 92, 1357-1371.

Lauer J, Zhou M, Ye S, Menegas W, Schneider S, Nath T, Rahman M M, Di Santo V, Soberanes D, Feng G. 2022. Multi-animal pose estimation, identification and tracking with DeepLabCut. Nature Methods, 19, 496-504.

Leonard S M, Xin H, Ramirez B C, Stinn J P, Dutta S, Liu K, Brown-Brandl T M. 2021. Static and Dynamic Space Usage of Late-Gestation Sows. Transactions of the ASABE, 64, 151-159.

Li J, Green-Miller A R, Hu X, Lucic A, Mohan M M, Dilger R N, Condotta I C, Aldridge B, Hart J M, Ahuja N. 2022a. Barriers to computer vision applications in pig production facilities. Computers and electronics in agriculture, 200, 107227.

Li Y, Chen Y, Wang N, Zhang Z.2019. Scale-aware trident networks for object detection. Proceedings of the IEEE/CVF international conference on computer vision, 6054-6063.

Li Y, Wu C-Y, Fan H, Mangalam K, Xiong B, Malik J, Feichtenhofer C.2022b. Mvitv2: Improved multiscale vision transformers for classification and detection. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4804-4814.

Lin T-Y, Goyal P, Girshick R, He K, Dollár P.2017. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, 2980-2988.

Liu S, Zeng Z, Ren T, Li F, Zhang H, Yang J, Li C, Yang J, Su H, Zhu J. 2023. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499,

Massa F, Girshick R.2018. maskrcnn-benchmark: Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch.

Matthews S G, Miller A L, PlÖtz T, Kyriazakis I. 2017. Automated tracking to measure behavioural changes in pigs for health and welfare monitoring. Scientific reports, 7, 17582.

Melfsen A, Lepsien A, Bosselmann J, Koschmider A, Hartung E. 2023. Describing behavior sequences of fattening pigs using process mining on video data and automated pig behavior recognition. Agriculture, 13, 1639.

Nazarkevych M, Kostiak M, Oleksiv N, Vysotska V, Shvahuliak A-T.2024. A YOLO-based Method for Object Contour Detection and Recognition in Video Sequences. CPITS, 49-58.

Neethirajan S, Kemp B. 2021. Digital livestock farming. Sensing and Bio-Sensing Research, 32, 100408.

Norton T, Chen C, Larsen M L V, Berckmans D. 2019. Precision livestock farming: Building ‘digital representations’ to bring the animals closer to the farmer. Animal, 13, 3009-3017.

Pan Y, Zhang Y, Wang X, Gao X X, Hou Z. 2023. Low-cost livestock sorting information management system based on deep learning. Artificial Intelligence in Agriculture, 9, 110-126.

Qiao Y, Kong H, Clark C, Lomax S, Su D, Eiffert S, Sukkarieh S. 2021. Intelligent perception for cattle monitoring: A review for cattle identification, body condition score evaluation, and weight estimation. Computers and electronics in agriculture, 185, 106143.

Radford A, Kim J W, Hallacy C, Ramesh A, Goh G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J.2021. Learning transferable visual models from natural language supervision. International conference on machine learning, 8748-8763.

Riekert M, Klein A, Adrion F, Hoffmann C, Gallmann E. 2020. Automatically detecting pig position and posture by 2D camera imaging and deep learning. Computers and electronics in agriculture, 174, 105391.

Schmidt T B, Lancaster J M, Psota E, Mote B E, Hulbert L E, Holliday A, Woiwode R, Pérez L C. 2022. Evaluation of a novel computer vision-based livestock monitoring system to identify and track specific behaviors of individual nursery pigs within a group-housed environment. Translational Animal Science, 6, txac082.

Shirke A, Golden R, Gautam M, Green-Miller A, Caesar M, Dilger R N. 2021. Vision-based behavioral recognition of novelty preference in pigs. arXiv preprint arXiv:2106.12181,

Siegford J M, Steibel J P, Han J, Benjamin M, Brown-Brandl T, Dórea J R, Morris D, Norton T, Psota E, Rosa G J. 2023. The quest to develop automated systems for monitoring animal behavior. Applied Animal Behaviour Science, 265, 106000.

T. Psota E, Schmidt T, Mote B, C. Pérez L. 2020. Long-term tracking of group-housed livestock using keypoint detection and map estimation for individual animal identification. Sensors, 20, 3670.

Tian M, Guo H, Chen H, Wang Q, Long C, Ma Y. 2019. Automated pig counting using deep learning. Computers and electronics in agriculture, 163, 104840.

Wang C-Y, Bochkovskiy A, Liao H-Y M.2023a. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 7464-7475.

Wang M, Li X, Larsen M L, Liu D, Rault J-L, Norton T. 2023b. A computer vision-based approach for respiration rate monitoring of group housed pigs. Computers and electronics in agriculture, 210, 107899.

Wu D, Cui D, Zhou M, Ying Y. 2022. Information perception in modern poultry farming: A review. Computers and electronics in agriculture, 199, 107131.

Yang Q, Xiao D. 2020. A review of video-based pig behavior recognition. Applied Animal Behaviour Science, 233, 105146.

No related articles found!
No Suggested Reading articles found!