facial images

THE localization of eyes in facial images has many applications in computer vision such as gaze estimation, pose estimation, face detection, face recognition, human computer interaction, eye blink detection etc. Eyes can be extracted from facial images using different eye features such as colour, illumination, shape, geometry, edges etc. Instead of improving existing techniques individually, this research proposes to combine existing techniques in a systematic way to provide a hybrid approach to developing a robust eye detection algorithm. In the proposed method, three existing methods based on illumination [1], colour [5] and Edges of eye regions [8], are used.

It is assumed in this paper that the input images satisfy the following conditions. * The input image is a colour head-shoulder image such as a passport photo.

* Both the eyes are visible i.e. the face is not tilted towards the right or the left more than 45 degrees.

* The eyes are not too close to the borders of the image.

The rest of the paper is organized as follow. Section 2 discusses the related work in the literature. Sections 3, 4 and 5 give an overview of the three selected eye detection techniques based on illumination, colour and edge density respectively. Section 6 gives details of the proposed method. Section 7 evaluates the proposed method by comparing it with existing methods. Section 8 concludes the paper.

II. RELATED WoRK

There has been extensive research in automatic eyes extraction for about two decades. Some face detection schemes also extract the eyes as the first step. Due to limited space it would not be possible to review all eye extraction methods, only a selected few are discussed here.

The red eye effect based eye detector is very robust and fast [3, 4]. However, there are three issues with this proposed scheme: 1) there is the need of specialized hardware, e.g. IBM PupilCam; 2) it is only applicable to videos; 3) the video needs to be taken at a short distance from the face in order to get the reflection from the pupils. Hsu, Mottaleb and Jain proposed an illumination-based eye detection method [1]. Although their method is reasonably robust, it is computationally expensive and it only works for colour images. Chiang, Tai, Yang, Huang and Huang proposed an eye detector based on the darkest part of the face [5]. In their method, the colour image is converted to equalized grey-level histogram. A threshold is applied to identify the eyes. Although this method is relatively fast, it fails to work with images of people with dark skin because their eyes are not always the darkest parts of their faces. Kumar, Raja and Ramakrishnan proposed a eye detection method [7] based on colour and projection functions, which is similar to [5]. Kawaguchi, Rizon and Hidaka proposed an eye detector that is based on Hough transform and separability filter [6]. The problem with this method is that it requires the rough size of the pupil to be known. Also, due to circle fitting in Hough transform, it is computationally expensive.

III. ILLUMINATIoN-BASED METHoD

Hsu, Mottaleb and Jain proposed an eye detector based on the illumination of the eye regions [1]. According to their proposed method, two separate eye maps, one from chrominance component of the image and the other from luminance, are built. The eye map from the chroma is based on the observation that high Cb and low Cr values are present around the eyes. It is calculated as follow

EyeMapC = [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] (1)

Where [C.sub.b], [C.sub.r] and [bar.C] are the normalized blue, red and negative of red chroma components respectively. The eye map from luma is based on the observation that eye regions contain both dark and bright pixels in the luma. So the morphological operators (e.g. erosion and dilation) can be designed to emphasize brighter and dark pixels in the luma component in the eye regions. The eye map from the luma component is calculated as follow

EyeMapL = Y (x,y) [direct sum] g[sigma] (x,y)/ Y (x, y) [??] g[sigma](x, y) (2)

Where Y(x,y) is the luma component of the image and g[sigma] (x, y) is structuring element. [direct sum] and [??] are morphological dilation and erosion. These two maps are then combined into a single eye map using the following formula

EyeMap = (EyeMapC)AND(EyeMapL) (3)

Figure 1 shows the step by step output of this method.

[FIGURE 1 OMITTED]

IV. CoLoUR-BASED METHoD

This method is based on the observation that the eyes are the darkest regions of the face [5]. According to this method, the colour image is converted to a histogram equalized grey level image. The eyes regions are then extracted from this histogram equalized image using a threshold operation with a threshold value of 20. Apart from the eyes, some other dark regions are also extracted as connected regions when this threshold operation. Unwanted regions can be removed via a component verification process. This method is very fast and simple but it fails for people with dark skin colour. Figure 2 shows the step by step output of this method.

V. EDGE DENSITY-BASED METHoD

This method is based on the observation that eye regions have more edge density as compared to other parts of the face. According to this method, the facial image is converted to grey level and edges are detected using Sobel edge detector. Morphological dilation is applied twice to enhance the connected regions. The small holes are filled inside the connected regions. Morphological erosion is applied three times to get rid of unwanted connected regions. Some shape and geometry based rules are applied to the remaining connected regions to extract eyes. This method works for all skin colours. However, it fails when the eyes are closed or the input image is badly illuminated. Figure 3 shows the step by step output of this method.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

VI. PROPOSED HYBRID METHOD

For the methods discussed above, when they fail to locate the eyes they are for different reasons. Therefore, the proposed method is to combine the three methods in a systematic way to overcome the weaknesses of the individual methods and result in a method that is more accurate than each of the individual methods. In the proposed method, blobs are detected using each of the methods. Each of the blobs obtained from the three methods are then tested using the following rules. Any of the blobs that does not satisfy any of the rules is removed.

* The solidity is of the region is greater than 0.5

* The aspect ratio is between 0.8 and 4.0

* The connected region is not touching the border

* The orientation of the connected component is between -45 and +45 degrees

Given [Image.sub.Illu], [Image.sub.Col] and [Image.sub.Edge] which are the binary images containing the blobs obtained through the illumination-based, colour-based and edge-density-based methods respectively, all the possible pairs of these images are made and images in each pair are combined through a bitwise (pixel by pixel) AND operators.

[Image.sub.IlluCol] = ([Image.sub.Illu])AND([Image.sub.col]) (4) [Image.sub.colEdge] = ([Image.sub.col])AND([Image.sub.Edge]) (5) [Image.sub.IlluEdge] = ([Image.sub.Illu])AND([Image.sub.Edge]) (6)

The unwanted connected regions which are detected by one method but not by others are automatically removed due to this bitwise AND operation.

The next step is to combine the three images, i.e. [Image.sub.IlluCol], [Image.sub.colEdge] and [Image.sub.IlluEdge], using a bitwise (pixel by pixel) oR operation:

[Image.sub.Hybrid] = ([Image.sub.IlluCol])oR([Image.sub.colEdge]) oR([Image.sub.IlluEdge]) (7)

Shape and geometry based rules are again applied to [Image.sub.Hybrid] to get rid of the false eye candidates. Figure 4 shows the schematic diagram of the proposed method.

VII. EXPERIMENTAL RESULTS

The proposed idea was implemented in Matlab using the image processing toolbox. The proposed algorithm was tested on PICS [2] facial images database. 80 photos of 50 individuals including both males and females of different ages and ethnicity were randomly selected. Most of them were of frontal faces. However, some faces were tilted to left or right. These are colour images with Widths vary from 360 to 480 pixels while their heights vary from 480 to 540 pixels. The accuracy of eye detection of the proposed method is 87 for PICS database. on Intel core duo 1.60 GHz Processor, the average processing time of our method was one second. In table 1, the proposed algorithm is compared with the existing methods. Since these existing methods [1, 5] dont mention their eyes verification steps explicitly thats why we are comparing only the initial blobs extraction and not the final eye detection. The qualitative comparison is summarized in table 2. Figure 5 shows some of the output images example of our method while figure 6 shows example images for which our method performed better than individual methods.

[FIGURE 5 OMITTED]

[FIGURE 6 OMITTED]

VIII. CONCLUSIONS AND FUTURE WORK

This paper presented a method for eyes detection by combining different eye features i.e. intensity, colour and edge. Each of the three methods those are based on the illumination, colour and edge properties are applied to the input image. The connected regions obtained from each of the methods are then combined in a systematic way to build a robust eye detector. The contribution of the proposed method is that it overcomes the weakness of each individual method by combining it with the other two methods. As the results show, the proposed method is very robust as compared to the other existing methods and it is more accurate than all of the three individual methods. our goal is to build a non-intrusive real time gaze estimation system which will use the method proposed in this paper as a first step to locate the eyes in first frame of the video.

REFERENCES

[1] Hsu, R.L., Abdel-Mottaleb, M. & Jain, A. 2002, "Face detection in color images", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 24, no. 5, pp. 696-706.

[2] PICS, 2003. Psychological Image Collection at Stirling (PICS image data base). Available from http://pics.psych.stir.ac.uk/, University of Stirling Psychology Department.

[3] Morimoto, C., Koons, D., Amir, A. & Flickner, M. 2000, "Pupil detection and tracking using multiple light sources", Image and Vision Computing , vol. 18, no. 4, pp. 331-335.

[4] Ebisawa, Y. & Satoh, S. 1993, "Effectiveness of pupil area detection technique using two light sources and image difference method", Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 1993. , pp. 1268-1269.

[5] Chiang, C.C., Tai, W.K., Yang, M.T., Huang, Y.T. & Huang, C.J. 2003, "A novel method for detecting lips, eyes and faces in real time", Real-Time Imaging , vol. 9, no. 4, pp. 277-287.

[6] Kawaguchi, T., Rizon, M. & Hidaka, D. 2005, "Detection of eyes from human faces by Hough transform and separability filter", Electronics and Communications in Japan(Part II Electronics) , vol. 88, no. 5, pp. 29-39.

[7] Kumar, R.T., Raja, S.K. & Ramakrishnan, A.G. 2002, "Eye detection using color cues and projection functions", Proceedings of International Conference on Image Processing , vol. 3.

[8] Shafi, M. & Chung, P.W.H. 2008 "Eyes Extraction from Facial Images Using Edge Density" Accepted in 7th IEEE International Conference on Cybernetic Intelligent Systems.

Muhammad Shafi is research student in the department of computer science, Loughborough University UK, email: m.shafi@lboro.ac.uk

Paul Chung is professor in the department of computer science, Loughborough University UK, email: p.w.h.chung@lboro.ac.uk

TABLE I
COMPARISON OF PROPOSED METHOD WITH EXISTING METHODS

Method Total Correct Incorrect Percentage
Images Detection Detection Accuracy

Illumination 160 144 16 90.00%
based

Colour- 160 134 26 83.75%
based

Edge- 160 128 32 80.00%
density-
based

Hybrid 160 150 5 93.75%

TABLE II
COMPARISON SUMMARY

Method Closed Eyes Dark Skin Bad
Colour Illumination

Illumination- [check] [check] x
based

Colour-based [check] x [check]

Edge- x [check] [check]
Density-
based

Hybrid [check] [check] [check]



Source Citation:Shafi, Muhammad, and Paul W.H. Chung. "A hybrid method for eyes detection in facial images.(Formula)(Technical report)." International Journal of Computer Systems Science and Engineering 5.4 (Fall 2009): 214. Computer Database. Gale. BROWARD COUNTY LIBRARY. 5 July 2009
.


(Web-Page) http://computer.tutor2008.googlepages.com/tutor2

(Album / Profile) http://www.facebook.com/album.php?aid=5745&l=970be&id=1661531726

Sincerely,
leonard.wilson2008@hotmail.com
Len Wilson