Video surveillance system of target contour

Размещено на http://www.allbest.ru/

Video surveillance system of target contour

M.P. Vasylenko, M.V. Haida

Aviation Computer-Integrated Complexes Department, National Aviation University, Kyiv, Ukraine

Abstract

Index Terms--Video surveillance; automatization; filter; contour; contour detection; image processing.

Система відеоспостереження за контуром цілі

М.П. Василенко. М.В. Гайда

У роботі розглянуто проблему виявлення та розпізнавання об'єктів, а саме, її вирішення без використання дорогих, ресурсоємних та складних систем збору та обробки даних, з можливістю її мобільності, простоти встановлення та початкового налаштування. Наведено доступні способи і методи вирішення проблеми, їх переваги і недоліки. Представлено роботу системи за алгоритмом розробленим на основі методу розпізнавання об'єктів, а саме виділення контурів фільтром на основі оператора Canny. У статті представлено проміжні та остаточні ілюстрації системного алгоритму на кожному кроці, що дає можливість ознайомитись з його перевагами перед класичними системами відеоспостереження та деякими недоліками. Для демонстрації роботи системи використано такі елементи, як веб-камера з частотою кадрів відео 25 кадрів в секунду, мобільний телефон та персональний комп'ютер із встановленим середовищем програмування Matlab2021 (обраний завдяки простоті використання та наявності вбудованих функцій обробки зображень).

Ключові слова: відеоспостереження; автоматизація; фільтр; контур; виявлення контуру; обробка зображень.

Система видеонаблюдения за контуром цели

М.П. Василенко. М.В. Гайда

В работе рассмотрена проблема выявления и распознавания объектов, а именно, ее решение без использования дорогих, ресурсоемких и сложных систем сбора и обработки данных, с возможностью ее мобильности, простоты установки и первоначальной настройки. Приведены доступные способы и методы решения проблемы, их преимущества и недостатки. Представлена работа системы по алгоритму разработанному на основе метода распознавания объектов, а именно выделения контуров фильтром на основе оператора Canny. В статье представлены промежуточные и окончательные иллюстрации системного алгоритма на каждом шаге, что дает возможность ознакомиться с его преимуществами перед традиционными системами видеонаблюдения и некоторыми недостатками. Для демонстрации работы системы использованы такие элементы, как веб-камера с частотой кадров видео 25 кадров в секунду, мобильный телефон и персональный компьютер с установленной средой программирования MatLab 2021 (избран благодаря простоте использования и наличию встроенных функций обработки изображений).

Ключевые слова: видеонаблюдение; автоматизация; фильтр; контур; выявление контура; обработка изображений.

Introduction

video surveillance system target

The rapid development of the functionality of video processing tools and their relatively low cost has led to the active use of digital technologies in various fields of human activity. In particular, computer video surveillance systems for surveillance in banks, offices, airports, supermarkets, and parking lots to search for objects by appearance have become widespread. In recent years, there has also been an active installation and use of video cameras at public transport stops, in parks, squares, schools, adjacent areas, etc. [3]. Such systems are increasingly used in forensics, access control systems, security systems (in our case, in the control system of cars entering the parking lot).

In such systems, with the help of applied intelligent information technology (IT), certain objects or situations are detected. That means this automation of the surveillance process can help maintain free space on the information storages and eliminates the need for online monitoring of the situation by the system operator, by notifying him visually or audibly.

When recognizing objects, the most informative part of the image is the contour [2]. An object contour is a part of an object that contains a lot of information about the shape of the object and almost does not depend on the color and texture of the image.

You can analyze the shape of the object along the contour. In many cases, information about the shape of the object is sufficient to organize automated or automatic systems. In addition, the transition to object recognition by their contours allows to reduce the amount of processed information by several times, as a plus, the contours are invariant to the brightness transformations.

Therefore, it is extremely important to select the contours of the processed images. This is what the article will be about.

Methods of Finding Objects in the Image

There are several popular methods, namely Template matching, Keypoint detection, Contour detection [4]. Each of the mentioned above methods has its own pros and cons. For example, the Template matching method assumes that objects are resized by the same number of times both horizontally and vertically. However, this is not always the case. And in the case of a disproportionate resizing of an object, the method will work incorrectly.

Keypoint detection offers a different approach: instead of looking for an entire object, let's find its typical parts, for example, corners, or characteristic elements. Usually “key points” are found automatically by finding pixels whose surroundings have certain properties. Many methods and criteria for finding them have been invented (Harris detector is the most understandable and accessible). After finding the key points in the picture and the template, you need to somehow compare them with each other. Some of the points of the pattern in the picture can be shifted by distortions, covered by other objects, so it seems unreliable to rely solely on the position of the points relative to each other. Therefore, for each key point, its neighborhood is built in order to make a certain description (descriptor), which will then allow us to take a couple of points (one point from the template, one from the picture), and compare their similarity (we have SIFT and BRIEF descriptors available). The method is good, and correctly handles situations when the sought object is rotated, its size is changed, or the object is partially hidden (which is good for finding complex objects). However, if there are few points on the object that you can "catch", or the shape of the object changes too much, then the key points on the template and the image may not coincide. Also, a background with a lot of small details can displace "key points" or change their descriptors.

The last method is Contour detection (and it's the most appropriate for this problem). If the internal content is not important to us, we can select the zones of contrast drops (edges), and among them find the edges, the outlines of which are similar to the contour of the object we need. The next step is to select the contours among the map with the found facets, and apply filtering to connect the contours selected with several lines. This solution takes into account only the shape of the object, and ignores its content and color (which can be either a plus or a minus).

Problem Statement

Video surveillance systems are one of the main components and occupy an important place in the overall structure of integrated security systems for facilities and individuals. In a world where crime, fighting, terrorist attacks and security breaches are on the rise, video surveillance systems are the right solution to prevent, detect and warn them.

Nowadays, video surveillance systems are increasingly integrated into various aspects of everyday life [2]. One of the options for classifying such systems is the scope of their use:

• video surveillance of roads and highways: measuring the speed of cars, detecting driving at a red traffic light, crossing the dividing lane and other traffic violations;

• public and commercial security: monitoring of public places to detect and prevent crime;

• environmental monitoring and research: observation of forest fires and pollution, habitats and migration of animals, mountain ranges, plant diseases, oceanographic research, preservation of historical and archaeological monuments, cultural heritage;

• military sphere: patrolling state borders, measuring refugee flows, monitoring the civilian population, ensuring the security of military bases, assistance and management during hostilities, etc.;

• quality control: monitoring of industrial and automated processes, production sites to identify faults and intrusions into their infrastructure;

• smart homes and personal safety: home surveillance to prevent theft and intrusion, health of patients, children, animals, etc.;

• analysis of video information: determination of patterns and anomalies in the movement of transport, pedestrians, sports indicators, traffic in shopping malls, amusement parks, etc.

The video surveillance system presented here will be used to identify the car in the image (namely the plate of its registration number), and identify it on the video stream by the frame and record information about the time of its movement and the beginning of video recording. That is why the Contour detection method, which I will use in my program, is the best solution to this problem, because it is the most resistant to various deformations of the object of observation.

For the simplest implementation of the system, the only components required are a webcam and a PC with Matlab software installed. This easy-to-use and inexpensive technology is suitable for installation at the entrances to the parking lots of supermarkets, yards and other private areas. Next, I will gradually talk about the sequence and implementation of algorithms for the system.

Preparation of the Image

For the convenience of work with the image and to reduce computing costs, it must first undergo special processing. This preparation is one-time (performed when installing the camera in the allocated position). This preparation includes the following steps.

1) Converting of RGB image or colormap to grayscale.

2) Smoothing the image with a Gaussian filter.

3) 2-D image flat-field correction.

4) Adjusting image intensity values.

So, at first I converted RGB image into grayscale. RGB2GRAY function mounted in Matlab converts RGB values to grayscale values by forming a weighted sum of the R, G, and B components: 0.2989 * R + 0.5870 * G + 0.1140 * B.

Now, when we have grayscale image (Fig. 1) it can be processed to more informative one. But at first it is needed to smooth unnecessary noises by 2-D Gaussian filtering.

Fig. 1. RGB image converted to grayscale

Pixels in the sliding window that are closer to the analyzed pixel should have a greater influence on the filtering result than the extreme ones [1]. Therefore, the coefficients of the mask weights can be described by a bell-shaped Gaussian function. When filtering images, a two-dimensional Gaussian filter is used:

Fig. 2. Image after Gaussian filter processing

The larger is the parameter a, the more the image is blurred. Typically, the filter radius is r = 3a. In this case, the size of the mask 2r +1X 2r +1 and the size of the matrix 6a +1X 6a +1. Outside this neighborhood, the values of the Gaussian function will be negligible. In MatLab, Gaussian filtering of an image can be performed using the imgaussfilt (I) function, shown in Fig. 2.

Now we have grayscale and unnoisy image, but it still has shading distortions. In order to reduce such distortions, I used imflatfield function (Fig. 3). It applies flat-field correction to the grayscale or RGB image.

Fig. 3. Result of flat-field correction

And the last point is to increase the contrast of the image. This can be done by the function imadjust. J = imadjust(I, [low in high in]) maps intensity values in I to new values in J such that values between low in and high in map to values between 0 and 1. Thus, a corrected image was obtained, which is ready for processing (Fig. 4).

Fig. 4. Image with increased contrast V. CONTOUR DETECTION

Typically, the registration number is a rectangular object (sometimes with rounded corners), the sides of which are parallel or nearly parallel to the coordinate axes. Then let's try to select the zones of contrast drops (edges), and among them we will find the edges, the outlines of which are similar to the contour of the object we need. This method is called contour detection.

Now we need to detect edges of object. Unlike keypoint detection, we are interested not only in keypoint-comers, but also in edges [5]. Let's calculate the derivatives of the intensity Ix and Iy. Since we do not need to distinguish angles from edges, we do not need to calculate the structure tensor - it is enough to calculate the strength of the gradient: It = I2 + I2y. After that, we will only leave pixels, which are local maxima in terms of It, but as the locale we will select not 8 neighboring pixels, but those pixels from these 8, towards which I is directed, and from the opposite side (Fig. 5):

Fig. 5. 3^3 operator of filter

The pixel in question is marked in blue, the arrow is in direction I. Green pixels are those that will be taken into account for non-maximum suppression.

This unusual selection of pixels for comparison is due to the fact that we do not want to make breaks in the border. In the left picture, the edge goes from top to bottom, and since the non-maximum suppression will not compare the intensity with pixels above and below blue, we will get a continuous edge. The result is shown in Fig. 6.

The next action Border tracking is to select contours among the map with found edges. Let's find the related components (islands of adjacent pixels that have passed all the checks), and check each of them to see how similar it looks to the number plate. After applying non-maximum suppression in the edge detector, we have guarantees that the edges will be one pixel thick, but let's not rely on it. For each pixel that has been assigned to a face, and next to which there is a non-face pixel, we refer to a “border”. Moving from one pixel of the border to another, we either come back to the same pixel (and then we found the outline), or to a dead end (then we can try to go back if there was a fork somewhere along the way), as shown in Fig. 7.

Fig.6. Selected contours (left) and initial processed image (right)

After applying this algorithm, we will be left with a set of contours that could potentially be number plates.

Fig. 7. The order of pixels' border tracking

Filtering the Contours

How do we know that our contour is a number frame? For rectangles and polygons, there is a great method based on path simplification. It is enough to gradually "collapse" the edges, if they are almost on the same straight line, and then count the number of remaining edges, and check the angles between them. Unfortunately, this method is not suitable for our case - our rectangle has rounded corners.

It is known that it is better to make a filter based on the properties of the form itself:

• the plate is large enough (area is more than 100 pixels);

• sides are parallel to the coordinate axes;

• the ratio of the area of the shape to the area of the bounding rectangle should be close enough to one. We set the threshold to 0.8, since the button is a rectangle with sides parallel to the coordinate axes, and the missing 20% are rounded corners and deviations from the axes.

Fig. 8. The use of the border search filter (a); blur applying

In addition, there may be problems with situations when a contrasting object lies under the plate of the license plate. Therefore, after applying the filter, we will blur the edges to close small holes that could arise from such objects.

We apply the obtained approach.

The use of the border search filter (Fig. 8a) found the necessary outlines, but due to the complex shape of the frame and the gradient, many outlines were found at once, and due to non-max suppression, some of them were not closed. Applying blur (Fig. 8b) fixed the problem.

Testing

Let's start searching for contours in the resulting frame. Let's paint the contour that passed the tests in red. If there are several of them, then we need to choose the most successful option among them. Select the largest contour and paint it green.

Fig. 9. Highlighted detected contours of registration number plate in 2 images

The resulting design found the number frames on the test images (Fig. 9). The run on all the images showed that occasionally (2 cases out of ~ 20) the contour is determined incorrectly, which is not critical in the online recording mode with a delay between frames of 0.2 s.

Conclusion

Proposed algorithm allows to perform video surveillance of target contour in real time. It has significant performance and allows to detect necessary types of objects automaticcally without need of human operator rpesence. It also allows to decrease the necessary storage capacity for video by starting recording only if the required type of object (in our example - car number plate) is detected.

Systems accuracy and reliability can be improved by combining the algorithms of Contour and Keypoint detection, but it will also decrease the reaction speed due to significantly greater number of image processing operations.

Proposed software can be deployed on a mobile controller (for example RaspberryPi) and become the universal pocket video surveillance system, which can be installed and configured anywhere.

REFERENCES

[1] Ya. E. Solem, Computer Vision Programming in Python, tr. from English A. A. Slinkin. Moscow: DMK Press, 2016. 312 p. [in Russian]

[2] K. Gzheshchyk, D. I. Zagorodnya, A. O. Sachenko, and B. P. Rusyn, "Video surveillance systems and methods of contour selection on images, " Project management and production development, no. 3, pp. 79-99, 2018. [in Ukrainian]

[3] R. Cipolla, S. Battiato, and G. M. Farinella, Advanced Topics in Computer Vision, Springer, 2013, 433 p.

[4] Angle detectors Available at: https://habr.com/ru/post/244541/ (accessed: 02.04.2021).

[5] M. P. Mukhina, "Algorithm of Visual Tracking of Contrast Moving Objects," of the Third World Congress "Aviation in TheXXI-st Century," 2008.

Размещено на Allbest.ru