Single Shot Multi Box Detector Approach to Autonomous Vision-Based Pick and Place Robotic Arm in The Presence of Uncertainties

Abstract

This research presents a problem of real-time accurate object detection in picking and placing objects using a robotic arm in conditions where conventional appearance-based approaches are largely ineffective. These conditions include partial occlusion, varying lighting, and change in camera pose. The methods presented in the literature have managed to achieve real time detection but at the expense of presence of the above mentioned scenarios which make them not usable in real world application. A single shot multibox detector Convolutional Neural Network is proposed to handle this problem. The network has been used for object detection in robotics but the performance of SSD with Resnet-50 as the backbone has not been explored. To evaluate the performance of the network, some challenges were formulated. The network was tasked to identify objects under uncertain conditions of varied lighting, partial occlusion, and changing camera pose. This was achieved by using bulbs of different lumen, occluding the objects in a manner that half of the object was visible to the camera and viewing the objects at an angle of 45 . This angle was different from the training viewing angle that was 0 . The network’s performance and speed of detection was tabulated for every experiment. The robot’s performance with the network was then evaluated by timing how long it took to identify, pick objects from one location, and place them in another. Successful attempts at grasping the objects were also evaluated. The proposed network helps to achieve real time detection in the range of 40 frames per second (fps) with accuracies of above 0.69 mAP (mean average precision) in varied lighting conditions, partial occlusion, and changing camera pose. This is an improvement to the SSD300 which was using VGG16 and produced 30 fps with accuracies of 0.65 mAP. Autonomous pick and place function was tested and was found to take between 15-30 seconds. The time was a factor of the shape of the object to be detected and how easy it was to pick and place. Experimental results validated the performance of the network and robot control method in a realistic scenario of picking and placing objects. xiv