Object detection using background subtraction method for pick-and-place operation of robot manipulator

Размещено на http://www.allbest.ru/

UDC 004.932.72

Object detection using background subtraction method for pick-and-place operation of robot manipulator

T. Rain, V.M. Dovgal, Y. N. Soe

Annotation

sensor information robotics subtraction

In the industrial automation and robotics, robots need sensors to detect working environment and collect the required information to perform the specific tasks. Nowadays, vision systems and automatic object handling are widely used in industrial production and other robotic applications. This paper presents the algorithm for object detection using background subtraction methods for pick-and-place operation of robot manipulator.

Keywords object detection; background subtraction; basic background subtraction; Gaussian Mixture Model; robotics; pick and place manipulator.

Over the past ten years, vision systems have been widely applied to the applications of robot manipulators. One of the most common applications of robot manipulators is pick and place application. In pick-and-place operation vision sensors are used to detect, localize and identify objects in workspace. Object detection is the first stage to localize and identify objects. Several methods such as optical ?ow [1], frame difference [2], and background subtraction [3] can be used in object detection [4].

Optic flow is the pattern of apparent motion of objects in a visual scene caused by the relative motion between an observer (camera etc.) and a scene [5]. Optical flow method describes a two-dimensional vector, composed of the direction and time rate of pixels of two consequent frames in a time sequence [6, 7]. This method is used for particular fields such as tracking a moving object and estimating the motion of a moving object. However, this method is not suitable for real-time applications because it has a large number of calculation, sensitivity to noise, poor anti-noise performance.

The frame difference or temporal difference is the common method of detecting moving objects. In the frame difference method the presence of moving objects is determined by calculating the pixel-wise difference between two or three consecutive frames in an image sequence. Frame difference method is very adaptive to dynamic environments, but generally has a poor performance of extracting all the relevant pixels, as a result the detection of moving object is not accurate [4, 8, 9].

The background subtraction method detects the region of foreground by comparing the current image with background image. This method is suitable for motion segmentation, especially under those situations with a static, noise-free background. It is simple, but some videos with poor signal-to-noise ratio caused by a low quality camera, compression artifacts or a noisy environment, are likely to generate numerous false positives [10].

The main objective of the present research is to create an algorithm that can detect objects in a workspace of robot manipulator for pick-and-place application quickly and efficiently. In our case, the workspace of robot manipulator is indoor environment and assume that there will be no huge changes in background model. So the background subtraction method with simple algorithm was created to detect objects for our pick-and-place operation. To make the foreground segmentation more complete and accurate, morphological opening technique is applied in the proposed method. To trace boundaries of objects, we use Moore-Neighbor tracing algorithm. Experimental results show that our method can effectively eliminate ghosts and noise and fill the cavities of the detected object.

1. Background subtraction methods

The basic principle of background subtraction methods is the first frame stored as the background image B(x, y). Then the current frame image f_t(x, y) is compared against the background image to determine the variation between the two images. Most back subtraction methods can be summarized by the following formula:

,

where ф is a threshold and g_t(x, y) is the motion mask at time t.

Several background subtraction methods have been published in recent years. Among them Basic Background Subtraction (BBS), One Gaussian (1-G), Minimum Maximum and Maximum Inter-Frame Di?erence (Min-Max), Gaussian Mixture Model (GMM), Kernel Density Estimation (KDE), Codebook (CBRGB) and Eigen Backgrounds (Eigen) methods were commonly used in the most works [10]. In this paper we will use the BBS method to create an algorithm for object detection and the results will be compared with the GMM method's results.

A. BBS method

The background image is stored as a single grayscale/color image void of objects to detect. To overcome the illumination changes and background modifications, the background image can be iteratively updated as follows [10]:

where б is a constant whose value ranges between 0 and 1. To segment out the objects, the difference of each pixel by color spaces (R, G, B) of background image and current image is calculated using the following formula:

(1)

where d_t(x, y) is the difference of each pixel at the time (t) and R, G and B are the red, green and blue channels of the pixel. After that the values of d_t(x, y) are compared with the threshold value to determine whether the pixel (x, y) is the background or foreground pixel. The calculation formula is as follows.

Choosing a suitable threshold value is important to achieve the success of object detection. Setting the threshold value too small will produce false detections, on the other hand, setting the threshold value too large will reduce the ability to detect objects. In this work, we set the value of threshold (ф) to 20.

B. GMM method

In GMM method, the probability of observing a certain pixel value, P, at time t is described by means of a mixture of Gaussians [13]:

where K is the number of Gaussian distributions, щ_{i, t} is the weight of the i^th distribution at time t, µ_i,t is the mean of i^th distribution at time t, ?_{i, t} is the standard deviation of i^th distribution at time t respectively and з is a Gaussian probability density function:

where n is the dimension of the color space. In Stauffer and Grimson[13] the red, green and blue (RGB) values are assumed independent, the standard deviation for the RGB values is assumed the same and the covariance matrix ?_{i, t} is calculated as:

When the new frame incomes at times t, a match test is made of every new pixel value, X_t against the existing K Gaussian distributions. A match is defined as a pixel value which is within 2.5 standard deviations of its mean. A pixel is defined as background if it matches with the Gaussian distribution which is chosen as background, otherwise it is defined as foreground. If a pixel does not match with any of the distributions, then the least probable distribution is replaced with a distribution with the current value as its mean value, a high initial variance , and a small weight . If a pixel matches with any of the distributions, then the parameters of the Gaussian are updated for the next foreground detection.

where б is learning rate defined by user and M_k,t is 1 for the distribution which matched and 0 for the remaining distributions. The mean (µ) and variance (у) for the unmatched distributions remain the same and for the matched distribution, they are updated as follows:

where .

When the parameters initialization is done, all the distributions are ordered based on the ratio between their peak amplitude щ and standard deviation у. The first B Gaussian distribution which exceeds the threshold value T is chosen as a background distribution:

where T is a threshold.

2. Proposed algorithm

This section explains the proposed algorithm into three main stages. The first stage is object extraction using background subtraction technique. The next stage is applying morphological open operation and dilation operation to get the better boundaries of detecting object. The morphological operation of opening of an input image A by a structuring element B is defined as follows:

The dilation of an input image A by a structuring element B, can be defined as

.

After that, the Moore-Neighbor tracing algorithm was used to trace region boundaries in the image [12]. The detailed algorithm is described as follow:

1. Acquire video sequences.

2. Calculate Background Subtraction using Eq.(1).

3. Apply threshold value to detect foreground.

4. Apply morphological open operation to the image.

5. Apply morphological dilation to the image.

6. Use Moore-Neighbor tracing algorithm to detect boundaries of objects.

3. Experimental results

We tested the proposed algorithm with the video that was acquired using a fixed USB-camera (Fig.1). The experiment was performed using MATLAB on a Dual-Core 2.00 GHz CPU with 4 GB of RAM. The results of the proposed algorithm were compared with the results of the GMM method. Figure 2 shows the color input image of the workspace. Figure 3 shows the results of object detection using proposed algorithms and Figure 4 shows the results of GMM method. The processing time of proposed algorithm was 0.87s and the GMM method was 1.22s. The results clearly show that the proposed algorithm detects the objects more quickly and efficiently than the GMM method.

Figure 1. The setting of USB-camera in the workspace.

Figure 2. The input image of the workspace.

Figure 3. The results of proposed algorithm.

Figure 4. The results of GMM method.

Conclusion

In the present research, the objects in the workspace of a robot manipulator were detected by using the BBS method and GMM method. Morphological dilation and open operations are applied to the images to remove the noises caused by lighting and environment changes. Comparing with GMM method, the experimental results show that the proposed algorithm can detect the objects more quickly and efficiently. Due to the short processing time of the proposed algorithm, it is suitable to use in real-time application like pick-and-place operation.

References

[1] Xin Y, Hou J, Dong L, et al. A self-adaptive optical flow method for the moving object detection in the video sequences Optik - International Journal for Light and Electron Optics, 2014, 125(19):5690-5694.

[2] Srinivas C. R., Darwin P. Frame Difference and Kalman filter techniques for detection of moving vehicles in video surveillance. International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622. Vol. 2, Issue 6, November- December 2012, pp.1168-1170

[3] M.E. Rashid, Vinu Thomas. “A Background Foreground Competitive Model for Background Subtraction in Dynamic Background. Procedia Technology, 2016:25:536-543.

[4] Weiming Hu, Tieniu Tan, Liang Wang, S. Maybank: A Survey on Visual Surveillance of Object Motion and Behaviors. IEEE Transactions on Systems Man and Cybernetics 34, 334-352 (2004).

[5] David H. Warren & Edward R. Strelow (1985). Electronic Spatial Sensing for the Blind: Contributions from Perception. Springer. ISBN 90-247-2689-1.

[6] Milin P. Patel, Shankar K. Parmar, Moving object detection with moving background using optic flow. IEEE International Conference on Recent Advances and Innovations in Engineering(ICRAIE-2014), May 09-11, 2014, Jaipur, India.

[7] Sepehr Aslani, Homayoun Mahdavi-Nasab, Optical Flow Based Moving Object Detection and Tracking for Traffic Surveillance. World Academy of Science, Engineering and Technology International Journal of Electrical and Computer Engineering-Vol:7,No.9,2013.

[8] Himani S. Parekh, Darshak G. Thakore, Udesang K. Jaliya. A survey on object detection and tracking methods. International Journal of Innovative Research in Computer and Communication Engineering - Vol.2, Issue 2, February 2014, ISSN:2320-9798, p.2970-2978

[9] Nishu Singla, Motion detection based on frame difference method. International journal of information & computation technology. ISSN 0974-2239 Volume 4, Number 15 (2014), pp. 1559-1565

[10] Yannick Benezeth, Pierre-Marc Jodoin, Bruno Emile, Hйlиne Laurent, Christophe Rosenberger. Comparative study of background subtraction algorithms. Journal of Electronic Imaging, SPIE and IS 2010.

[11] R. Cucchiara, C. Grana, A. Prati, and R. Vezzani. Probabilistic posture classi?cation for human-behavior analysis. in Transactions on Systems, Man and Cybernetics, Part A, 35:42-54, 2005.[12] Gonzalez, R. C., R. E. Woods, and S. L. Eddins, Digital Image Processing Using MATLAB, New Jersey, Pearson Prentice Hall, 2004.

[13] C. Stau?er and W.E.L. Grimson. Adaptive background mixture models for real-time tracking. International conference on Computer Vision and Pattern Recognition, 2, 1999.

Thu Rain

Kursk State University

Postgraduate student

Phone +7(9510)85-76-12

Email: therein.48@gmail.com

Размещено на Allbest.ru