Abstract
•The multi-vision system is powerful, but interferences always emerge in field.•A multi-geometry based framework is designed to ensure stable visual measurement.•The proposed adaptive stereo matching is much more efficient and reliable than the classic one.•The proposed fine stitching algorithm greatly compensates the unavoidable global calibration error.•This study provides reference for applying multi-vision system in complex agricultural picking tasks.
Automatic vision-based picking in orchards and fields is a highly challenging task. The orchard banana central stock, which is large in size, low in color contrast, and falls within a complex background, was taken as the subject in this research. A measurement framework based on multi-vision technology was established, and a set of general methods were utilized to improve the comprehensive performance of multi-view-geometry-based vision modules in orchard picking tasks. Multiple cameras at different angles were deployed to maximize the perception range. The global geometric parameters of the cameras were calibrated and a robust semantic segmentation network was trained to achieve effective image pre-processing. A novel adaptive stereo matching strategy was designed to ensure that the robot reliably completes 3D triangulation at various depths as it moves across the target area. Global calibration errors were corrected via a high-accuracy point cloud stitching algorithm. Experimental results indicated that the proposed adaptive stereo matching strategy was accurate to different sampling depths and showed stable performance, and the proposed point cloud stitching algorithm accurately stitched multi-view point clouds. This work provides theoretical and practical references for the 3D sensing of banana central stocks in complex environments. The proposed technique was designed for adaptability of the multi-vision system for field perception, so it can be easily transferred to similar applications such as the 3D reconstruction of agricultural targets, 3D positioning of fruit clusters, and 3D robotic arm obstacle avoidance.