Enhanced progressive fusion method for the efficient detection of multi-scale lightweight citrus fruits

Yanlin Zeng, Yao Lin, Yiting He, Tong Li, Jing Li, Baijuan Wang, Yi Yang

Abstract


Human labor efficiency has become unable to keep the pace with gradually annual citrus increasing production. Highly efficient and intelligent citrus picking and accurate yield estimation is the key to solve the problem. Success heavily depends on detection accuracy, prediction speed, and easy model deployment. Traditional target detection methods often fail to achieve balanced results in all those aspects. An improved YOLOv8 network model with four significant features is proposed. First, a lightweight FasterNet network structure was introduced to the backbone network, which reduced the number of parameters and computations while maintaining high-precision detection. Second, a progressive feature pyramid network AFPN structure was added to the neck network. Third, a parallel multi-branch attention mechanism PMBA was added before the detection head to improve the sensing ability after the feature fusion network. Fourth, a Wise-IoU was introduced to replace the original CIoU loss function to make the whole training process converge faster. Based on this, this study proposes an improved version of the YOLOv8 model: the FAP-YOLOv8. This improved model achieved an average accuracy (mAP@0.5) of 97.2% on the citrus datasets, with an accuracy that was 4.7% higher than the original YOLOv8, which was 19.2%, 7.4%, 5.1%, 4.9%, and 5.2% higher than the other models: Faster R-CNN, CenterNet, YOLOv5s, YOLOx-s, and YOLOv7, respectively. The number of parameters was reduced by 55.45%, the computation was reduced by 20% compared to the YOLOv8 benchmark, and the frame rate reached 46.51 fps to meet the detection requirements of lightweight networks. The experiments showed that the FAP-YOLOv8 models all outperformed the comparison models. Consequently, the proposed FAP-YOLOv8 model can help solve the citrus detection problem in orchards, which can be better applied to edge devices and provides strong support for intelligent orchard management.
Keywords: citrus fruit detection, enhanced progressive fusion model, multi-scale lightweight, attention mechanism
DOI: 10.25165/j.ijabe.20241706.8802

Citation: Zeng Y L, Lin Y, He Y T, Li T, Li J, Wang B J, et al. Enhanced progressive fusion method for the efficient detection of multi-scale lightweight citrus fruits. Int J Agric & Biol Eng, 2024; 17(6): 218–229.

Keywords


citrus fruit detection, enhanced progressive fusion model, multi-scale lightweight, attention mechanism

Full Text:

PDF

References


Liu S Y. Artificial intelligence (AI) in agriculture. IT Professional, 2020; 22(3): 14–15.

Tang Y C, Chen M Y, Wang C L, Luo L F, Li J H, Lian G P, et al. Recognition and localization methods for vision-based fruit picking robots: A review. Frontiers in Plant Science, 2020; 11: 510.

Díaz I, Mazza S M, Combarro E F, Giménez L I, Gaiad J E. Machine learning applied to the prediction of citrus production. Spanish Journal of Agricultural Research, 2017; 15(2). doi: 10.5424/sjar/2017152-9090.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015; 521(7553): 436–444.

Sa I, Ge Z Y, Dayoub F, Upcroft B, Perez T, McCool C. DeepFruits: A fruit detection system using deep neural networks. Sensors, 2016; 16(8): 1222.

Bargoti S, Underwood J. Deep fruit detection in orchards. In: 2017 IEEE international conference on robotics and automation (ICRA), Singapore: IEEE, 2017; pp.3626–3633.

Bargoti S, Underwood J P. Image segmentation for fruit detection and yield estimation in apple orchards. Journal of Field Robotics, 2017; 34(6): 1039–1060.

Koirala A, Walsh K B, Wang Z, McCarthy C. Deep learning - Method overview and review of use for fruit detection and yield estimation. Computers and Electronics in Agriculture, 2019; 162: 219–234.

Gao F F, Fu L S, Zhang X, Majeed Y, Li R, Karkee M, et al. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Computers and Electronics in Agriculture, 2020; 176: 105634.

Kukreja V, Dhiman P. A Deep Neural Network based disease detection scheme for Citrus fruits. In: 2020 International conference on smart electronics and communication (ICOSEC), Trichy, India: IEEE, 2020; pp.97–101.

Horng G J, Liu M X, Chen C C. The smart image recognition mechanism for crop harvesting system in intelligent agriculture. IEEE Sensors Journal, 2019; 20(5): 2766–2781.

Liu Z S, Xiang X Y, Qin J H, Tan Y, Zhang Q, Xiong N N. Image recognition of citrus diseases based on deep learning. CMC-Computers Materials & Continua, 2021; 66(1): 457–466.

Qiu W J, Ye J, Hu L Q, Yang J, Li Q L, Mo J Y, et al. Distilled-MobileNet Model of convolutional neural network simplified structure for plant disease recognition. Smart Agriculture, 2021; 3(1): 109–117.

Liu Y P, Yang C H, Ling H, Mabu S, Kuremoto T. A visual system of citrus picking robot using convolutional neural networks. In: 2018 5th international conference on systems and informatics (ICSAI), Nanjing, China: IEEE, 2018; pp.344–349.

Lu J, Hu X W. Detecting green citrus fruit on trees in low light and complex background based on MSER and HCA. Transactions of the CSAE, 2017; 33(19): 196–201. (in Chinese)

Bi S, Gao F, Chen J W, Zhang L. Detection method of citrus based on deep convolution neural network. Transactions of the CSAM, 2019; 50(5): 181–186. (in Chinese)

Zhang W L, Wang J Q, Liu Y X, Chen K Z, Li H B, Duan Y L, et al. Deep-learning-based in-field citrus fruit detection and tracking. Horticulture Research, 2022; 9: uhac003.

Zhuang J J, Luo S M, Hou C J, Tang Y, He Y, Xue X Y. Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Computers and Electronics in Agriculture, 2018; 152: 64–73.

Lin G C, Tang Y C, Zou X J, Li J H, Xiong J T. In-field citrus detection and localisation based on RGB-D image analysis. Biosystems Engineering, 2019; 186: 34–44.

Chen J Y, Liu H, Zhang Y T, Zhang D K, Ouyang H K, Chen X Y. A multiscale lightweight and efficient model based on YOLOv7: Applied to citrus orchard. Plants, 2022; 11(23): 3260.

Lyu S L, Li R Y, Zhao Y W, Li Z, Fan R J, Liu S Y. Green citrus detection and counting in orchards based on YOLOv5-CS and AI edge system. Sensors, 2022; 22(2): 576.

Yang H W, Liu Y Z, Wang S W, Qu H X, Li N, Wu J, et al. Improved apple fruit target recognition method based on YOLOv7 model. Agriculture, 2023; 13(7): 1278.

Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016; pp.779–788.

Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv preprint arXiv: 1804.02767, 2018. doi: 10.48550/arxiv.1804.02767.

Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934, 2020. doi: 10.48550/arXiv.2004.10934.

Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada: IEEE, 2023; pp.7464–7475.

Chen J R, Kao S-H, He H, Zhuo W P, Wen S, Lee C-H, et al. Run, don’t walk: chasing higher FLOPS for faster neural networks. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada: IEEE, 2023; pp.12021–12031.

Yang G Y, Lei J, Zhu Z K, Cheng S Y, Feng Z L, Liang R H. AFPN: Asymptotic feature pyramid network for object detection. In: 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu, Oahu, HI, USA: IEEE, 2023; 2184–2189.

Liu Y C, Shao Z R, Teng Y Y, Hoffmann N. NAM: Normalization-based attention module. arXiv preprint arXiv: 2111.12419, 2021; doi: 10.48550/arXiv.2111.12419.

Woo S, Park J, Lee J-Y, Kweon I S. Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), Munich, Germany: Springer, 2018; doi: 10.1007/978-3-030-01234-2_1.

Wang Q L, Wu B G, Zhu P F, Li P H, Zuo W M, Hu Q H. ECA-Net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA: IEEE, 2020; pp.11534–11542.

Zheng Z H, Wang P, Liu W, Li J Z, Ye R G, Ren D W. Distance-IoU loss: Faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence, USA: AAAI Press, 2020; pp.12993–13000. doi: 10.1609/aaai.v34i07.6999.

Tong Z J, Chen Y H, Xu Z W, Yu R. Wise-IoU: Bounding box regression loss with dynamic focusing mechanism. arXiv preprint arxiv: 2301.10051, doi: 10.48550/arXiv.2301.10051.

Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017; 39(6): 1137–1149.

Duan K W, Bai S, Xie L X, Qi H G, Huang Q M, Tian Q. Centernet: Keypoint triplets for object detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South): IEEE, 2019; 6568–6577.

Ultralytics/yolov5. 2021; Available: https://github.com/ ultralytics/yolov5. Accessed on [2023-04-25].

Ge Z, Liu S T, Wang F, Li Z M, Sun J. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv: 2107.08430, 2021; doi: 10.48550/arXiv.2107.08430.

ultralytics, 2023. Available: https://github.com/ultralytics/ultralytics. Accessed on [2023-04-19].




Copyright (c) 2024 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office