Section

A Study on Real-Time Vision-Based Detection of Skin Pigmentation

Liu Yang1,*, Suk-Hwan Lee2, Seong-Geun Kwon3, Ki-Ryong Kwon4
Author Information & Copyright
1Department of IT Convergence and Application Engineering, Pukyong National University, Busan, Korea, yangliu.0816@hotmail.com
2Department of Information Security, TongMyong University, Busan, Korea, skylee@tu.ac.kr
3Department of Electronics Engineering, Kyungil University, Daegu, Korea, seonggeunkwon@daum.net
4Department of IT Convergence and Application Engineering, Pukyong National University, Busan, Korea, krkwon@pknu.ac.kr
*Corresponding Author: Department of IT Convergence and Application Engineering, Pukyong National University, Busan, Korea, yangliu.0816@hotmail.com

© Copyright 2014 Korea Multimedia Society. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Aug 10, 2014 ; Revised: Sep 01, 2014 ; Accepted: Sep 20, 2014

Published Online: Sep 30, 2014

Abstract

Usually, the skin pigmentation detection and diagnosis are made by clinicians. In this process it is subjective and non-quantitative. We develop an approach to detect and measure the different pigmentation lesions base on computer vision technology. In the paper we study several usually used skin-detecting color space like HSV, YCbCr and normalized RGB. We compare their performance with illumination influence for detecting the pigmentation lesions better. Base on a relatively stable color space, we propose an approach which is RGB channels vector difference characteristic for the detection. After the object region detection, we also use the difference to measure the difference between the lesion and the surrounding normal skin. From the experiment results, our approach can effectively detect the pigmentation lesion, and perform robustness with different illumination.

Keywords: Skin pigmentation; Detection; Vision; Real-time; Normalized RGB

I. INTRODUCTION

Skin pigmentation lesions occur because in our skin the melanocytes or blood cells chromospheres become abnormal, and many reasons can cause that. For example, sun exposure, metabolism disorder, blood vessel congest and ectasis. Generally the abnormal can be divided into two kinds: benign and malignant. That means it not only affects people’s appearance, but also has an effect on people’s health. Whether from medical or beauty standpoint, the pigmentation lesions detection is very important.

In this paper, we focus on segmentation of the skin pigmentation region in a skin image; In other words, we can think it is an image classification problem. Then we need find an obvious stable characteristic between the skin pigmentation lesion and the normal skin for classification. Between color and texture characteristic, we selected the color. Because that the pigmentation lesion texture distinguish is difficult even in the high-resolution image. For giving a distinguishing characteristic we compare three color spaces which are often used in skin images; HSV, YCbCr and normalized RGB.

According to comparing color space, we find that the skin color has better stability in the normalized RGB color space, and synthesize intensity change in the shadow image sample. We propose a RGB channels vector difference characteristic to distinguish the normal skin and pigmentation lesion region. In the process, we use OTSU thresholding to segment out part of the normal skin as a standard channel values. In most of the cases, the skin images will contain normal skin, hemoglobin and melanin pigmentation lesion, under shadow region, hairs. Through the gray image we can find our normal skin has the highest intensity among them. Then the normal skin could be easily extracted. On the other hand the input image will be split into three single channel images. We transform them into the integral image for saving time. Then we can compute the pigmentation lesion characteristic block by block and label them on output image. We use the difference between normal skin channel difference and the pigmentation lesion to measure the lesion intensity change, and use the labeled block percentage as target area.

In the experiment part, we compare our approach with other paper. Because fewer image reading times and simple process, the program shows that the approach has relative fast speed with good illumination robustness. It can recognize the pigmentation lesion effectively.

II. Color space analysis and detection methods

In skin image processing area, several color spaces have been used widely, like HSV, YCbCr and normalized RGB. We compare them stability in different illumination environment to find which one could work better with different intensity. That will help us to distinguish the pigmentation lesion from the shadow part. Habili et al [1] present an automatic human skin segmentation algorithm based on YCbCr color space. They use that the CbCr is less influenced by illumination to model a skin-color distribution as a bivariate normal distribution in the CbCr plane and classify each pixel as skin or non-skin based on its Mahalanobis distance. The program distinguish the pixels is skin or not is based on an experiment statistical information. It needs many different intensity skin samples to do the statistical analysis, according to these statistics (Figure 1) develop a skin distribution model. Then program use a threshold value to make the decision. This approach has weakness that the threshold value is an experiment value; it does not have a relationship with the input image intensity. It a fix value for every different input image. And the model is developed by the sample images, but some normal skin may not contain in this clustering range.

jmis-1-1-77-g1
Fig. 1. (a) Pixel color distribution along Cb, (b) pixel color distribution along Cr
Download Original Figure

As Figure 2 we make a skin sample with shadow and pigmentation, we draw a green line on the sample. We can find that half part of the green line contains shadow and the other not, each of them cross a pigmentation spot. Then we can make each channel density distribution of YCbCr color space along this green line (Figure 3 a). From Y value we can locate the pigmentation spot location; on the image we can find the pigmentation lesion value is similar to the shadow part. It is difficult to distinguish the shadow and pigmentation lesion in YCbCr color space (Figure 4).

jmis-1-1-77-g2
Fig. 2. Pigmentation lesion skin sample.
Download Original Figure
jmis-1-1-77-g3
Fig. 3. (a) Pixel density in each channel of YCbCr (b) Pixel density in each channel of HSV.
Download Original Figure
jmis-1-1-77-g4
Fig. 4. YCbCr recognition result, (a)input image (b) the result.
Download Original Figure

Sheng and Liu [2] classify the skin in HSV color space. They propose a k (k=S/H) parameter as the segmentation feature parameter, make distinction by using the clustering characteristic. Flesh tones are often easier to pick out after conversion to an HSV color space. We analyze our sample along the green line (Fig.3 (b)), to shadow part the pigmentation lesion recognition is still difficult. Then we compare the HSV color space with normalize RGB. We capture two images (Figure 5) from different illumination environment. And we use RG and SH components to make histogram like Figure 6. Through comparing the difference of two histograms in different environment, we can find which color space has better robustness to the illumination. And according to the test result (table 1) we can find the normalized RGB color space works better than HSV in the context of lighting changes.

jmis-1-1-77-g5
Fig. 5. Captured image for color space comparison (a) With lighting environment (b) Without lighting environment
Download Original Figure
jmis-1-1-77-g6
Fig. 6. RG and HS histogram application in different illumination environment, (a) RG histogram, 1.Bright RG-result, 2. Dark RG-result (b) HS histogram, 1. Bright HS-result, 2.Dark HS-result.
Download Original Figure
Table 1. Histogram Comparison in RG and HS via four methods.
Comparing Histogram correlation Chi-square Intersection
HS-Histogram 0.37914 0.97145 0.368947
RG-Histogram 0.465301 0.834907 0.436768
Range −1/1(match) (match)0/2 0/1(match)
Download Excel Table

From the previous contents we can find the YCbCr color use the simple threshold to segment the skin area and non-skin area, It has fast processing speed but will make much mistakes when the skin image contain wide range of intensity. To HSV color space, it is easily to extract the flesh tones, but it maybe mismatch with intensity changes. According to the table 1 comparison and experiment, the normalized RGB works better than HSV with the intensity changes.

III. PROPOSED SKIN PIGMENTATION LESION DETECTION

The color of our skin is from two kinds of chromatophore, the hemoglobin and melanin. At the same time when the skin pigmentation lesion appears because that the two kinds of chromatophore is disorder. When the pigmentation lesion appears, comparing with the normal skin, the ratio between hemoglobin and melanin will change, but different intensity will cause of less change like this. We can see this in Figure 7 which expresses reflex pixel intensity via RGB color space in the skin sample image. Tsumura [3] built a relationship between the two chromatophore components and the RGB color model. We find that the pigmentation disorder causes the RGBscale change, but the shadow does not. We see that with the intensity change the R line and the G line are almost parallel and B line is part of, RGB channel difference value will obviously change where the pigmentation lesion appears. We will use this characteristic to distinguish the pigmentation lesion from the shadow.

jmis-1-1-77-g7
Fig. 7. Pixel density in each channel of RGB
Download Original Figure

We propose an approach for detecting the skin pigmentation lesion as Figure 8. In this approach, we have two branches; one is use the weighted OTSU thresholding [4] to segment the normal skin part and get the channel different value, the other split the input image initially into three single channels, and then transforms them into integral image for a better processing speed. At last we compare the vector difference value between the image and the normal skin part, and then label the target area.

jmis-1-1-77-g8
Fig. 8. Real-time skin pigmentation lesion detection flowchart
Download Original Figure
1. Normal skin values

We know that the pigmentation lesion area is the different color part to our normal skin. So when we get the parameters of normal skin and the relationship between the normal and the abnormal, this problem will become easier. For extracting the normal skin part we transform the input image into gray image, and we use OTSU thresholding algorithm to extract part of them. When we compare intensity among the normal skin, the pigmentation lesion area, the shadow part and hair, our normal skin has the highest intensity. So when we give a weight to the OTSU, part of the normal skin will be segment out as foreground. The OTSU will segment the image into foreground and background based on the image intensity, and the expressions are as following:

t = Max [ W 0 t * U 0 t - U 2 + W 1 t * U 1 t - U 2 ]
(1)
T = weight*t
(2)

where the w0 is the scale of background, u0 is the average of background, w1 is the scale of foreground, u1 is the average of foreground, u is the average of whole image. In practical process, we give a weight to make sure that only normal skin part is in the foreground even it is a relatively few parts (Figure 9). After we get the normal skin area, the normal skin intensity value and the difference between them in each channel can be calculated.

jmis-1-1-77-g9
Fig. 9. OTSU process result (a) input image, (b) output image
Download Original Figure
2. Fast image characteristic calculation

Multichannel image process always takes much time especially with the image size increase. After the input image is split into three single channels, we transform them into integral image. The integral image was proposed by Viola et al [6]. That through building the integral image, the sum of the pixel intensity in any rectangle region can be calculated by three times addition and subtraction. As a simple example, to sum over a simple rectangular region described by the corner points (x1, y1) and (x2, y2) where (x2 > x1)and(y2 > y1), then:

x 1 x x 2 y 1 y y 2 [ image ( x , y ) ] = [ sum ( x 2 , y 2 ) sum ( x 1 1 , y 2 ) sum ( x 2 , y 1 1 ) + sum ( x 1 1 , y 1 1 ) ]
(3)

In this way, it is possible to compute means, standard deviations and vectors faster. And perform fast block correlations even for variable windows sizes [7]. Then we can compare our image with the standard normal skin values in the last step block by block.

About the distinguishing parameters, we refer some normal skin intensity ranges and characteristics by Kovac [8].In RGB color space, the normal skin always has R>G>B. From the skin samples observation and statistics, we can find the normal skin keeps a relatively stable difference value between three channels. When the melanin pigmentation appears, the three channels’ values not only reduce at the same time, but also reduce the differences value too. In the hemoglobin case, the green and blue channels reduce obviously; the red channel will little change from the normal skin value than the other two channels. From the previous section we find the normalized RGB is a more suitable color space to deal with the illumination changes. It shows better clustering performance, and the normalized RGB can be transformed from RGB as following:

r = R R + G + B ,  g = G R + G + B ,  b = B R + G + B
(4)

In order to study the change laws of color in different brightness condition, we analyze a color checker chart (figure 10.a) under different brightness condition. In each brightness change, we take a RGB picture. We calculate the mean value the {R/G, B/G}, the 3D RGB image will be transformed into 2D color space. Then we take the logarithm, by using the log(R/G) as the horizontal axis, the log(B/G) as vertical axis, we can build the coordinate. The original image will be transformed into many points’ sets as Figure 10.b [9]. Though observing, the points of same color block will lay on an

jmis-1-1-77-g10
Fig. 10. a) Color checker chart. b) The point sets of color blocks in different brightness.
Download Original Figure

Approximate straight line in different brightness and these lines which are constituted by different color almost parallel each other.

We defined two components x and y to describe the skin pixels distribution by using the logarithm as following:

x i = log R i G i ,    y i = log B i G i
(5)

where is the index of block of the integral image, RGB is the mean value. From the normal skin ratio value can also get the x0 and y0 as original values. We build vectors the standard normal skin V0=(x0,y0), and the other skin area Vi=(xi,yi). According to the Figure 7 and functions derivation, we can find a characteristic of the vectors which is the normal skin vector Vi=(xi,yi)almost parallel to the standard vector V0=(x0,y0). But the pigmentation skin area is not, because the scale of RGB is changed. So the cosine similarity becomes a viable option for distinguishing the pigmentation lesion and the normal area. Cosine similarity is a measure of similarity between two vectors through measuring the cosine of the angle between them. Then we can split the pigmentation area by using following equation:

sim = cos ( θ ) = V 0 V 1 V 0 V 1
(6)

IV. EXPERIMENT RESULT

Figure 11 is a testing sample of our program. On the experiment phase, in order to basically describe our scheme property we estimate the program from two aspects: the illumination adaptability and processing time. We created a two hundreds pigmentation images database. In the database the image size is from 100*100 to 600*600 and the image contents contain five illumination levels, the brightness from low to high are in light shadow, indoors, outdoors in shade, in light and outdoors in the sun. Each of the level contains about forty different skin pigmentation images. Figure 12 intuitively reflects the skin pigmentation images database composition. From top to bottom of the gray block is the range of the whole pixels intensity distribution and the middle line is the most of pixels locate intensity level. From the figure we can find our testing samples contained most of the application environment and part of extremely cases. After we arrange the testing samples, for more intuitive illustration, we compared our approach with Lu’s [10].

jmis-1-1-77-g11
Fig. 11. Test experiment (a) input image, (b) output image
Download Original Figure
jmis-1-1-77-g12
Fig. 12. Skin pigmentation image pixel distribution
Download Original Figure

The paper contains two main algorithms: Independent Component Analysis (ICA) and Support Vector Machine (SVM). People skin color is composed by two parts: the hemoglobin and melanin, meanwhile the skin pigmentation abnormal is also because these two components scale disordered. The skin pigmentation ICA show us a mathematical model that we can map the skin pigmentation image from 3D RGB color space to 2D hemoglobin-melanin color space. It does not only reduce the dimension to save the process time, but also show us a way which we can more intuitively describe the skin pigmentation and the features of pigmentation disorder will be more obvious. This approach is capable of segmenting low contrast images with good detection performance. Lu et al extracted melanin and hemoglobin components based on and ICA algorithm and detected the skin pigmentation region with a histogram-based Bayesian classifier. Then, they identified erythema regions by using the SVM for two components. One of the reasons that make us to compare our approach with this paper is the skin pigmentation ICA, the other is the machine learning algorithms (Bayesian classifier and SVM). The machine learning algorithms analyze our collected features and adjust weights, thresholds and the other parameters then classifier the new input objects. As a new development area for image identification and classification it is closely tied to the computer vision algorithms that range from finding feature points via that trained classification to identify and segment the image. From the function point our approach and Lu’s are similar, and with the previous mentioned advantages, we decide to compare the performance with it.

In the illumination adaptability testing phase, we manually segment the skin pigmentation area via the Photoshop as the standard images. The melanin pigmentation disorder area has obvious boundary but part of hemoglobin pigmentation disorder area do not have this kind of boundary, so this part of Standard images contain a little subjective judgment. We respectively process the images in our skin pigmentation database by our approach and Lu’s. Then we calculate the segmented area difference between standard images and the processed images. We get the matching area ratio as table 2. From the table we can find most of the process results are acceptable, but both of the approaches perform not well in light shadow. In light case our approach is not good, we find in our skin pigmentation images some of them contain the highlight area, in the OTSU thresholding phase the highlight area pixels are segmented as the normal skin pixels that lead the output results failed. This case more usually appears in some cosmetic advertisement images, because they contain much this kind of highlight. But if the highlight is not too much, it will not affect the results

Table 2. Experiment results comparison
In light shadow Indoors Outdoors in the shade In light Outdoors in the sun
Lu’s approach 0.4780 0.9239 0.8411 0.9663 0.9731
Our approach 0.6399 0.9163 0.9307 0.6871 0.9346
Download Excel Table

In the process time test, the used computer configuration is as following: CPU-Core2 2.8 GHz, RAM-3GB, OS-WinXP, Graphics card-ATI HD3400. We find the image content barely affect the process time, so for each size level we only tested three images and compared the average value. In the small cases both of two approaches can quickly finish the process, but with the image size increase, our approach performance is obviously better than the other’s (Figure 13).

jmis-1-1-77-g13
Fig. 13. Program process time comparison.
Download Original Figure

V. CONCLUSION

In this paper we studied YCbCr, HSV and normalized RGB color space to find a robust one to illumination change. Though the comparison we find the normalized RGB has the best performance among them. Then we presented an approach to detect human skin pigmentation lesion from skin image. We used the weighted OTSU algorithm to segment part of the normal skin as a standard compared value. And we split the input image into three single channels; transform them into integral images for better process time. We also proposed a parameter RGB channels vector difference for distinguish the skin pigmentation lesion from the shadows. Then by using this parameter and other terms we effectively detect the skin pigmentation lesion in skin image. From the experiment results, we can see that this approach has good recognition ratio in different illumination environment and much improve the process time.

REFERENCE

[1].

N. Habili, C. C. Lim, and A. Moini, “Automatic human skin segmentation based color information in the YCbCr color space,” Information, Decision and Control 2002, pp. 377-382, Feb. 2002.

[2].

K. K. Sheng, Y. Liu, et al, “Method of Reducing Dimensions of Segmentation Feature Parameter applied to Skin Erythema Image Segmentation,” Engineering in Medicine and Biology 27th Annual Conference, pp. 3422-3424, Sep. 2005.

[3].

N. Tsumura, H. Haneishi, and Y. Miyake, “Independent component analysis of skin color model image,” journal of Optical Society of America, vol. 16,no. 9, pp. 2169-2176, 1999.

[4].

N. Otsu, “A Threshold Selection Method from Gray-Level Histograms?” IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, 1979

[5].

A. Cordiner, “Face detection using generalized integral image features,” Image Processing 16th IEEE International Conference, pp. 1229-1232, Nov. 2009.

[6].

P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” Computer Vision and Pattern recognition IEEE Computer Society Conference, vol.1, pp.511-518, 2001

[7].

H. Bay, A. Ess, et al , “Speeded-up robust features (SURF),” Computer Vision and Image Understanding, vol. 110, No. 3, pp. 346-359, Jun. 2008.

[8].

J. Kovac, P. Peer, and F. Solina, “Human skin color clustering for face detection,” EUROCON, vol. 2, pp.144-148, Sep. 2003.

[9].

D. F. Graham, S. D. Hordley, and M. S. Drew, “Removing Shadows from Images,” The 7th European Conference on Computer Vision-Part (ECCV), pp.823-836, 2002.

[10].

J. Lu, J. H. Manton, E. Kazmierczak, and R. Sinclair, “Erythema detection in digital skinimages,” Image Processing (ICIP), pp. 2545-2548, Sep. 2010.