A neural learning approach for simultaneous object detection and grasp detection in cluttered scenes

Object detection and grasp detection are essential for unmanned systems working in cluttered real-world environments. Detecting grasp configurations for each object in the scene would enable reasoning manipulations. However, finding the relationships between objects and grasp configurations is still a challenging problem. To achieve this, we propose a novel neural learning approach, namely SOGD, to predict a best grasp configuration for each detected objects from an RGB-D image. The cluttered background is first filtered out via a 3D-plane-based approach. Then two separate branches are designed to detect objects and grasp candidates, respectively. The relationship between object proposals and grasp candidates are learned by an additional alignment module. A series of experiments are conducted on two public datasets (Cornell Grasp Dataset and Jacquard Dataset) and the results demonstrate the superior performance of our SOGD against SOTA methods in predicting reasonable grasp configurations from a cluttered scene.

Automated summary

Object detection and grasp detection are crucial for unmanned systems operating in cluttered real-world environments. The proposed SOGD approach utilizes neural learning to predict the best grasp configuration for each detected object, overcoming the challenge of finding relationships between objects and grasp configurations. Experimental results demonstrate the superior performance of SOGD in predicting reasonable grasp configurations from cluttered scenes, outperforming state-of-the-art methods on public datasets.