Detection method for transparent glass cleaning device, image processing approach

When the camera is focused on the dust particles, the background is blurred as shown in Fig. 1a. At this point, the dust particles to be separated are in focus and the background, which should be ignored, is out of focus. To distinguish between the two, an edge detection method or a noise filter can be used. We experimented with different methods. For edge detection, the Sobel12 and Prewitt13 methods were evaluated. Gaussian blur16 and median filter14 techniques have been used as noise filtering methods. The edge detection method detects sharp dust pixels by viewing them as a type of boundary line. The noise filtering method considers dust pixels as noise and eliminates them, allowing dust pixels to be detected using image differentiation techniques.

Different methods create different detection images. The edge detection method performs thresholding on the image on which the method was applied. The noise filtering method performs thresholding on the difference between the original image and the noise filtered image. To determine which method to use, we defined two scores. The dust separation score (SSD) is a score indicating how well the method separates dust particles. The background ignores the score (BIS) is a score indicating the extent to which the method produces a good SSD with only a small degree of background blur. To calculate the score, we synthesized a randomly generated dust mask using various blurred backgrounds, after which four methods were compared using the corresponding images.

Dust mask and blurred background image set

For the test, we needed a randomly generated dust mask to overlay the 720p image. For an image with a resolution of 1280720 pixels, we generated dust particles with random size, blur and grayscale at a uniform random point count of 2304, 9216, 20736, 36864. The size range was [0   1]the sigma blur range (gaussian filter size) was [0   3] and [0  5]and the grayscale range was [50   255]. A set of images was created by combining this dust mask with background images of different background types and focus levels. MATLAB was used to generate and converge sets of test images.

The set of images to be used for the test was created by synthesizing an image taken by only adjusting the focus differently at the same position as the resulting dust mask. Once the true dust pixel information and background image sharpness information was known, the SSD and BIS could be calculated. Figure 3 shows the structure of the image set.

Figure 4

youHER the curve shows each method HER for the threshold range of [0   255]. The median filter got the highest SSD.

Dust separation score

First, it must be determined how well the method distinguishes between dust particles and background. The definition of the separation precision (HER) of an image, comparing the image detected by our method to the dust mask, with an image size of (n times m) is as follows:

$$begin{aligned} SA(I,M,T))=frac{sum _{n}^{i=1}sum _{m}^{j=1}T(I_{i, j},t) bigoplus T(M_{i,j},1)}{ntimes m} end{aligned}$$

(1)

you:

: target threshold, [0 255] full range.

(i_{i,j}):

: detection image using our methods. Pixel value of position (i,j).

(M_{i,j}):

: image of the dust mask, pixel value of the position (i,j).

J(p, you):

: Threshold function 1 (true) when (pge t)otherwise is 0 (false).

(bigoplus):

: XOR operation in Boolean value

SA has a range of [0   1], a value close to 1 indicates that the dust pixel was successfully separated. After calculating this index for all images in the image set, the highest average values ​​of you and HER at that time are called (t_s) (separation threshold) and SSDrespectively.

Therefore, the definition of DSS is as follows:

$$begin{aligned} DSS= & {} mean(sum SA(I,M,t_S)) end{aligned}$$

(2)

$$begin{aligned} t_s= & {} max(mean(sum SA(I,M,t)) end{aligned}$$

(3)

Figure 4 shows the (t-SA) conspiracy and SSD for the four methods. The median filter (filter size 5) showed the best SSD with a value of 0.898. Then the Sobel and Prewitt filters had a similar result SSD of 0.866 and 0.865. Finally, the Gaussian filter showed the lowest SSD0.86.

As the threshold value increases, more detected pixels are filtered out, so that the image converges to an image with all pixel values ​​of 0. Since the accuracy is evaluated by the XOR method, even if the part where the original value should be 0 is 0 , it is noted. Therefore, it converges to the score of the empty part of the dust mask. We can verify that for each of the four types of methods, when t tends to 255, it converges to a certain value. Sobel and Prewitt filters showed values ​​below the minimum SSD received even with a black screen. This means that no matter how precisely the threshold was set, it was not possible to reduce the misconception that there is dust where there is none. However, the Gaussian filter and the median filter showed SSD than the minimum SSD at a specific threshold value, which means that the noise reduction method has a better overall effect than the boundary detection method. Among them, the median filter showed the best SSD.

Figure 5
number 5

Resized FMHER the curve shows HER fall like FM increases. The Gaussian method obtained the highest BIS.

Score ignored in background

For each edge detection algorithm, a test was performed to determine how blurry the background needed to be before it could be detected. It depends on the camera, but the background must be in focus to use edge detection. At this point, the higher the minimum blur level that ignores the background, the greater the minimum distance between the transparent window and background objects. When detecting the background, the result is different because a part other than the desired dust particles is detected. Therefore, the best method to use is one that can separate background and dust pixels with the least amount of blurring. In this study, background sharpness was measured using a frequency modulation (FM) image quality measurement method.18.

We defined the background sharpness measured by FM on the x-axis. Using the previous introduction (t_s)an FM-SA plot was created by placing (SA(I,M,t_s)) as y axis. At this stage, the image used is the same background image with different levels of focus. Subsequently, the X and there the axes are scaled to a range of [0   1]. For scaling, MATLAB’s scaling function was used. Therefore, the definition of BIS is as follows:

$$begin{aligned} BIS=sum _{b_{type}}^{b=1}int _{f=f_{step}}^{f=1}rescaledSA(I_{b,f}, M,T_s)d FM(BI_{b,f})) end{aligned}$$

(4)

FM(I):

: measured value of image sharpness I by FM.

(b_{type}):

: Number of background types in the image set.

(f_{step}):

: number of focus level steps in the image set.

(I_{b,f}):

: picture to picture set with background type bfocus level F.

(BI_{b,f}):

: background image image set with background type bfocus level F.

Figure 5 shows the FM-SA plot and the BIS for each method. The Gaussian filter was highest at 0.822, followed by the Median filter at 0.741, and the Sobel and Prewitt filters were lower.

Dust separation step procedure

Through a series of comparisons, SSD was in order Median > Gaussian > Sobel ( simeq) Prewitt, and BIS was Gaussian > median > Sobel (simeq ) Prewitt. BIS can be overcome by adjusting camera focus and shooting environment, but DSS is difficult to change. Therefore, a median filter was used.

The first stage image processing procedure using a median filter is as follows. The filter takes the original image as input and produces a dust-separated binary image.

  1. 1.

    Import original image A.

  2. 2.

    To create (A_grey)i.e. a grayscale image of A.

  3. 3.

    To create (A m)i.e. a filtered median image of (A_grey) with filter size 5.

  4. 4.

    To create (A_{diff}) like (abs(A_{grey} A_m)).

  5. 5.

    To create (Burnt}) like (T(A_diff,t_{s,median}=6)).

In Fig. 1a, c represent (A_grey)(b) means (A m)and (d) represents (Burnt). The resulting binary image is used in the next step, i.e. the real density estimation step.

Figure 6
number 6

When mean shift clustering is in progress, the red circle represents the window of each centroid, the green dot represents the dead centroid, and the red dot represents the live centroid. (a) indicates the initial position of the centroid. (b) is the state in the middle of the clustering. You can see that the density shifts to higher spaces.

Comments are closed.