This project presents an adaptive soft gripper system capable of detecting object shapes using computer vision and estimating object dimensions through 3D point cloud analysis. Based on the detected shape and size, the system automatically adjusts the gripper’s orientation.
The setup uses a RGBD camera to fuse color and depth information, allowing real-time classification of common shapes (e.g., circle, square, rectangle) and dimension estimation from aligned depth data.
- Vision-based shape detection using OpenCV
- Dimension estimation from aligned RGB-D data using point cloud processing via the RealSense SDK
- Real-time 3D visualization with Open3D
- Real-time feedback from a bending sensor integrated into the soft gripper for monitoring deformation
Watch the complete system in action:
Click the button above to view the comprehensive demonstration video showcasing the gripper’s adaptive capabilities across different object shapes and sizes.
This rotary-linear mechanism controls the gripper’s opening based on servo angles. Three configurations are used:
- Spherical: All servos at 90° for round objects.
- Cylindrical: Rear servos at 60° and 120°, front at 90°.
- Parallel: Rear servos at 145° and 30°, front at 90° for flat surfaces.
These configurations allow adaptive grasping based on object geometry.
The Intel RealSense D435i camera is positioned in front of the gripper, facing the object. It captures synchronized RGB and depth data used for shape detection and size estimation. The camera is fixed in place to maintain a consistent field of view during operation.
 Circle-Shaped Object |
 Rectangle-Shaped Object |
 Square-Shaped Object |
|
- Real-time Shape Recognition: Automatic detection of circular, rectangular, and square objects
- Adaptive Gripper Configuration: Dynamic adjustment of servo positions based on detected geometry
- 3D Dimension Analysis: Accurate size estimation using depth data for optimal grasping strategy
- Integrated Feedback: Continuous monitoring through embedded bending sensors
The system integrates multiple technologies to achieve autonomous adaptive grasping:
- Computer Vision Pipeline: RGB image processing for shape classification
- Depth Analysis: Point cloud processing for dimensional measurements
- Servo Control: Precise actuation based on object characteristics
- Sensor Feedback: Real-time deformation monitoring during grasping operations