Merging Outcomes of SAM Applied to RGB and Depth Images in Bin Picking Applications

Published

April 19, 2024

Author(s)

Marek Franaszek, Prem Rachakonda, Pavel Piliptchak, Kamel S. Saidi

Abstract

Segmenting images containing many objects stacked in unstructured piles is a challenging and vital task in robotic bin picking applications. Objects in such images are congested and occluded but, nevertheless, must be accurately segmented to calculate their 6DoF poses. For fast completion of automated tasks, many of these poses should be calculated and sent to the robot controller at once so that the path planning algorithm can prioritize which object to grasp. We show that the Segment Anything Model (SAM) can be used as the first step in processing such images to segment individual parts in a bin. However, applying SAM to red, green, blue (RGB) and depth images acquired from the same bin yields different results, with many segmentation masks present in only one type of image. Thus, merging two SAM outputs from both image types is suggested to maximize the number of segmented parts.

Proceedings Title

Sensors & Transducers

Conference Dates

April 17-19, 2024

Conference Location

Funchal, PT

Conference Title

6th International Conference on Advances in Signal Processing and Artificial Intelligence

Pub Type

Conferences

Download Paper

Local Download

Keywords

Foundation Models, Segment Anything Model (SAM), RGB and Depth Image, Robotic Bin Picking

Citation

Franaszek, M. , Rachakonda, P. , Piliptchak, P. and Saidi, K. (2024), Merging Outcomes of SAM Applied to RGB and Depth Images in Bin Picking Applications, Sensors & Transducers, Funchal, PT, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=957706 (Accessed July 17, 2024)