Robust Multi biometric Recognition Using Face and Ear Images

(IJCSIS) International Journal o f Comput er Science and Info rmation Security, Vol. 6, No. 2, 2009 Robust Multi-biometric Recognition Using Face and Ear Images Nazmeen Bibi Boodoo*, R K Subramanian Computer Scie nce Departm ent University of Mauritius Mauritius nazmeen182@y ahoo.com Abstract : This study investigates the use of ear as a biometri c for authentication and shows experi mental results obtained on a newly created dataset of 420 images. Images are passed to a quality module in order to reduce Fals e Rejection Rate. The Principal Comp onent Analysis (“eig en ear”) approach was u sed, obtaining 90.7 % recognition rate . Improvement in recognition results is obtain ed when ear b iometric is fu sed with face biometric. The fus ion is done at decision level, achiev ing a recognition rate of 96 % . Keywords: Biometric, Ear Recognition, Fa ce Recognition, PCA, Multi-biometric, Fusion. I. I NTRODUCTION Ear recognition has receive d considerably less attention t han many alternative biometrics, including face, fingerprint and iris recognition. Ear-based recogn ition is of particular interest because it is non-invasive, and because it is not affected by environm ental factors such as mood, healt h, and clothin g [11]. Also, the appearance of the auri cle (outer ear) is relatively unaffected by aging, m aking it bet ter sui ted for long -term identification. Ear images can be easily taken from a distance wit hout knowledge of the person concerned. Therefore ear biom etric is suitable of surveillance, security, access con trol and monitoring appli cations. Ear print s, found on the crim e scene, have been use d as a proof in over fe w hundreds cases i n the Netherlands a nd the United States [14]. The purpose of the proposed paper is to investigate whether th e integration of face and ear bi ometri cs can achi eve hi gher pe rformance t hat ma y not be possibl e using a sin gle biom etric indicat or alone. II. E AR B IOMETRIC Two studies performed by Iannarelli [2] p rovide enough evidence to show that ears are uniqu e biomet ric traits. The first study comp ared over 10,0 00 ears drawn from a randoml y selected sample in California, and the second st udy examined fraternal and identical twins, in which physiological features are known to be sim ilar. The evidence from these studies supports the hypot hesis that the ear c ontains unique physiological features, since in both st udies all examined ears were found to be unique though identical twin s were found to have similar, but not iden tical, ear structures especially in the Concha an d lobe areas. Fi g 1 shows the a natomy of t he ear [3]. Figure 1. 1 Helix Rim, 2 Lobule, 3 Antihelix, 4 Concha, 5 Tragus, 6 Antitragus, 7 Crus of Helix, 8 Triang ular Fossa, 9 Incisure Intert ragica The medical literature re ports [2] that ear gro wth after the first four mont hs of bi rth is pr oportional . It tur ns out t hat even though ear growth is propor tional, gravity can cause the ear to undergo st retching in the vertical direction. The effect of this stretching is most pronounced in the lobe of the ear, and measurements show that the chan ge is non-linear. The rate of stretching is approx imately five times greater than normal during the period f rom four months t o the age of eight, after which it is constant until aro und 70 when it again increases. The main dra wback of ear biom etrics is that they are not usable when the ear of the subject i s cove red [2]. In the case of active identification systems, th is is not a drawback as the subject can pull hi s hair back an d proceed with the authentication p rocess. The pr oblem arises during passive identification as in this case n o assistance on the p art of the subject can be assum ed. In the case of the ear being only partially occluded by hair, it is po ssible to recognize the hair and segment it out of the image. 164 http://sites.google.com/site/ijcsis/ ISSN 1947-5500 (IJCSIS) International Journal o f Comput er Science and Info rmation Security, Vol. 6, No. 2, 2009 III. R ELATED WORK Several Studies have been do ne in using ear a s a biom etric. The following sections give an overview of p revious works do ne. A. Ear Biometric One of the ea rliest ear det ection m ethods uses C anny edge maps to detect the ear contour [3]. Chang et al. [12] com pared ear recognition with face recognition using a standard principal compone nts analysis (PCA) tec hnique. Rec ognition rate obtained were 7 1.6 % and 70.5 % for ear a nd face recognition respectively. Hurley et al. [13] considered a “force field” feature extraction approac h that is based on simulated potential energy fields. They reported improv ed performance over PCA-based metho ds. Alvarez et al. [1] used a modi fied active contour algorithm and Ovoid m odel for detecti ng the ear. Yan and B owyer [ 8] proposed ta king a prede fined sector from the nose t ip to locat e the ear region. The non -ear portion fro m that sector is cropped out by skin detection and t he ear pit wa s detected usi ng Gaussian smoot hing and c urvature e stimation. Then, they applied an active contou r algorithm to extract the ear con tour. The system is automatic but fails if the ear pit is not visible. Li Yuan an d Mu [9] used a m odified CA MSHIFT alg orithm to roughly track the prof ile image as the region of inter est (ROI). Then, contour fitting is operated on ROI for further accurate localization usi ng the cont our inform ation of the ea r. Saleh et al. [18] tested a dataset of ear images using se veral image- based classifiers and feature-extraction m ethods. Classification accuracy ranged from 76.5% to 94.1% in the experim ents. Most recently, Islam et al. [5] pr oposed an ear detection approach base d on the AdaBoost al gorithm [7]. The system was trained with rectangular H aar-like features and usin g a dataset of varied races, sexe s, appearances, orientations and illuminations. Th e data was collected by cropping and synthesizing from several face im age databases. The approach is fully automatic, provid es 100% detection while tested with 203 non-occluded images and also works well with some occluded and degraded im ages. As summarized in the survey of Pun et al. [6] m ost of the proposed ear reco gnition approaches use either PCA (Principal Compone nt Analysis) or the ICP algorithm for m atching. Choras [4] proposed a di fferent aut omated geometrical method. Testi ng with 2 40 im ages (20 different views) of 12 subjects, 100% recogn ition rate is reported. The first ever ear rec ognition sy stem tested with a larger database of 415 sub jects is proposed by Yan and Bow yer [8]. Using a modified version of t he ICP, they achieved an accuracy of 95.7% with occl usion and 97.8 % without occlusion (with an Eq ual-error rate ( EER) of 1 .2%). The system does not work well if the ear pit is not visible. Islam et al . [10] prop osed a met hod for crop ping 3D profile face data for ear detection and applied the Iterative Closest Point (ICP) algorithm for recognition of the ear at different mesh resolutions of the ex tract ed 3D ear data. The system obtains a recognition rate of 9 3%. It is fully automatic and does not rely on the presen ce of a particular feature of the ear (e.g. ear pit). B. Face Biometric Research in automatic face r ecognition da tes back at 1960’s [19]. A survey of face rec ognition techniques has been given by Zhao et al., (2003). In ge neral, face recognition techniques can be divided int o two groups based on t he face representation they use: 1. Appearance-based: which uses holistic texture features and is applied to either whole-fa ce or specific regions in a face image; 2. Feature-based: whi ch uses ge om etric facial features (mouth, eyes, brows, cheeks etc.) and ge ometric rel ationships between them. Kirby and Si rovich wer e am ong the first to apply principal component a nalysis (PCA) to face im ages, and showed that PCA is an optimal compression scheme that minimizes the mean squared error between t h e original images and their reconstructio ns for any give n level of compression [2 0]. Turk and Pentland popularized the us e of PCA for face recognition [21]. They used PCA to c om pute a set of subspace basis vectors (which they called “eigenf aces”) for a database of face images, and projected the images in the database into the compressed subspace. New t est images we re then matched to images in the database by projecting t hem ont o the basi s vectors and finding the nearest com pressed image in the subspace (eigenspace). Researchers began to search for other subspaces t hat might improve performance. One altern ative is Fis her’s linear discrimi nant analysis (LDA, a.k.a. “fisher faces”) [22]. F or any N-class classification problem, th e goal of LDA is to find the N-1 basis vectors that maximize the interclass distances while minimizing the intra-class dist ances. At one level, PCA and LDA are ver y differe nt: LDA is a super vised lea rning technique that relies on class labels, whereas PC A is an unsupervise d techni que. One characteristic of both PCA and LDA is that they produce spatially glo bal feature vec tors. In ot her words, the basis vectors prod uced by PCA a nd LD A are non-zero for almost all dimensions, implying that a ch ange to a single input p ixel will alter every dimensi on of its subspace projecti on. There i s also a lot of interest in techniques that create spatially lo calized feature vectors, in the hopes t hat they m ight be l ess suscept ible to occlusio n and woul d impl ement reco gnition by parts . The most common method for generating spatially localized 165 http://sites.google.com/site/ijcsis/ ISSN 1947-5500 (IJCSIS) International Journal o f Comput er Science and Info rmation Security, Vol. 6, No. 2, 2009 features is t o apply in dependent component analysis (IC A) to produce basis vectors that are statist ically indepe ndent [23] . C. Ear Versus Face Biometric Though face recognitio n has been extensively studied in the past decades, im aging problem s (e .g., lighting, shadows, scale, and translation) make it difficult to bu ild an unconstrained face Identification. Also, it is di fficult t o collect consistent features from the face as it is argua bly the most changing part of the body due to facial e xpressions, co smetics, facial hair and hair styling [3]. The combinatio n of the typical imaging problems of feature extraction i n an unconstra ined environment, and t he changeability of the face, e xplains the difficulty of autom ating face bi ometrics. Colour distribution is more uniform in ear than in hu man face. Not much information is lost while working with grayscale or binarised images. Ear is also sm aller than face, which means that it is possible to work faster and more efficiently with images with the lower resoluti on. Ear i mages cannot be disturbed by glasses, bea rd or m ake-up. Howeve r, occlusi on by hair and earring is po ssible. D. Multi-Biometric Although m ost biom etric system s deployed i n real-worl d applications are unimoda l, so they rely on the evidence of a single source of information fo r authentication , these systems have to contend wit h a variety of problem s such as noise in sensed data, intra-class variations, inter-class similarities, non- universality, and spoof attack s. Some of the limitations imposed by un imodal biom etric system s can be overcom e by including multiple sources of information for establishing identity. These systems allow the integration of two or more types of biometric systems. Integrating multiple modalities in user verificati on and ident ificati on leads to hi gh perform ance [17]. IV. METHODOLOGY A. Dataset A multimodal dataset was created. It involves pe ople aged from 20 t o 50 years old. The Kodak di gital cam era of 7. 1 Mega pixels w as used. 30 persons we re involved, each one having 7 face im ages and 7 ear im ages, giving a total of 420images. To obtain ear images, the profile images were taken and cro pped. Face i mages are of 150 × 200 res olution while ear images are of 100 × 150 resolution. The setup for the image capture is shown in Fig 2. Exam ple of the dataset is given Fi g 3 and Fi g 4. Figure 2. Image Capture Setup Figure 3. Sample Face images Figure 4. Sample Ear Images The ear im ages have been m anually croppe d and resi zed from the original profile head images. B. Image Quality The quality of biometric sample has significant impact on performance of recognition. One of the main reasons for matching errors in bio metric systems is poor-quality images. Automatic biometric image quality assessment may help improve system performance. In this study, Normalised Cross-correlation is used as a measure to determine the quality of an input image. The basis of using correlatio n as a pattern matching method lies in determini ng the degree to whi ch the object under e xaminati on resembles that co ntained i n a given reference i mage. The degree of resemblance is a simple statistic o n which to base decisions about the object [25] . The so called normalised cross-correlati on method i s a widely used m atch measure in correlation based p attern recognition. For input image f and mean image in of trai ning se t, g, the normalised cross- correlation measure of ma tch is defined as Camera Light 1 Light2 Subject Position (1) 166 http://sites.google.com/site/ijcsis/ ISSN 1947-5500 (IJCSIS) International Journal o f Comput er Science and Info rmation Security, Vol. 6, No. 2, 2009 MM Fusion Fusion Score 1 Score 2 Score 3 Score 1 Score 2 Score 3 Final Score MM MM MM MM MM QM QM QM QM QM QM C. Face and Ear Verification The features extracted were based on the Karhunen-Loeve (KL) expansi on, also k nown as prin cipal com ponent analy sis (PCA). The main reasons to used KL expansion were that it has been ex haustively studied a nd have proved t o be qu ite invariant an d robust whe n prope r norma lization is ap plied ove r the faces [15] . On the other han d, the m ain disadva ntages of KL methods a re its com plexity and t hat the extracted base is data-dependent : if new images are ad ded to the database the KL base nee d to be re comput ed. The m ain idea i s to decompose a face picture as a weighted com bination of the orthonorm al base provided by the KL transf orm. The b ase corresponds to the eigenvectors of the covari ance matrix of the data, known as eigenfaces or eigenears. Thus, the decomposition of a face image into an eigenface space provides a set of features . The maxim um number of features is restricted t o the nu mber of im ages used to compute the KL transform, altho ugh usually only the more re levant features are selected, rem oving the ones associated with the smallest eigenvalues. In the classic eigenface method, proposed by Turk and Pent land [16] , the PCA is per formed o n a dataset of face im ages from all users to be recognized. D. Levels of Fusion Because of the use of m ultipl e modalities, fusion techni ques should be establis hed for combini ng the di fferent m odalities. Integration of information in a Multimod al biometric system can occur in three main levels, namely feature level, matching level or decision level [18]. At feature level, the feature sets of different modalities are combined. Fusion at this level provides the highest flexibility but classi fication problems may arise due to the large d ime nsion of t he combined feat ure vectors. Fusion at matchi ng level is the m ost comm on one, whereby the scores of t he cla ssifiers are usually norm alized and then they are com bined in a consiste nt manner. At fu sion on decision level each subsystem determines its own authentication decision and all individu al results are combined to a comm on decision of the fusion system . In this study, fusion at the decision level is applied for data fusion of the various modalities, based on the majority vote rule. For three samples, as is the case, a minimum of two accept votes is needed for acceptance. Also, for the final fusion, the AND rule is used . Fig 5 shows two-level fusion applied in this study. Figure 5. Multi-biometric fusion, QM: Qu ality Module, MM: Matching Module. V. EXPERIMENTAL RESULTS The test of the proposed biometric recognition system consists in the evaluati on of the qual ity modul es, matchi ng modules and the fusion bl ock repre sented in Fig 5. The ma tching algorithms generate a score for each template comparison based on the distance bet ween the te sted and sto red feature vectors. The Euclidean distance metric is used, as it achieves good results at a low c omputat ion cost [24] . The lowest distance score value indicates the best match. The perform ance of indi vidual biomet ric is show n in Fig 6 and Fig 7 below: -20 0 20 40 60 80 100 120 2. 15 2. 2 2.25 2. 3 2. 35 2. 4 2. 45 Th re s hol d (e + 004) Rate (% ) FAR Re co gn itio n r ate Figure 6. Face Recognition Performance Measures FRR 167 http://sites.google.com/site/ijcsis/ ISSN 1947-5500 (IJCSIS) International Journal o f Comput er Science and Info rmation Security, Vol. 6, No. 2, 2009 -20 0 20 40 60 80 100 120 2 .4 2 .5 2 .6 2 .7 2 .8 2 .9 3 3 .1 Thre s hold (e + 004 ) Rate (% ) FRR FAR Recognit i on Rate Figure 7. Ear Recognition Performance Measures Using threshold values that maximize the correct recognition rates for eac h individ ual biomet ric, after fu sion a F AR of 0 % was obtained, as illustrated in Table 1. TABLE I. R ESULTS FOR THRESHOLDS EQUI VALENT TO MAXIMUM CORRECT AUTHENTICATIONS Face Ear Multimodal Fusion Recognition Rate 94.7 % 90.7 % 96 % FAR 25 % 40% 0 % FRR 5 % 9.3 % 4% The unimodal face and ear bi omet ric gives recognition rate of 94 % and 90.7 % respectively. When fused, the multi- modal gives a recogni tion rate of 9 6 %, showing an improvem e nt in the accuracy. Also, both th e FAR and FRR have been considerably red uced, showing that the multi- modal system implem ented is more ro bust. VI. C ONCLUSION This paper proposes a mult imodal bio metric recogni tion system that exploits two m odalities, namely face and ear recognition. With multi-sampling and fusion at decision leve l, a recognition rate of 96 % was obtained. Currently, we are working to enhan ce the recognition rate und er uncontrolled environment so that it can be applied to surveillance applications. R EFERENCES [1] L . Alvarez, E. Gonzalez, and L. Mazorra. Fitting ear contour using an ovoid model. In Proc. of Int’l Carnahan Conf. on Security Technology, 2005., pages 145 – 148, Oct. 2005. [2] A. Iannarelli, Ear Identification . Forensic Identification Series. Paramont Publishing Company, Frem ont, California, 1989. [3] M . Burge and W. Burger. Ear biometrics in computer vision. In Proc. of the ICPR’00, pages 822 – 826, Sept 2000. [4] M. Choras. Ear biometrics based on geometrical feature extraction. Electronic Letters on Computer Vision and Im age Analysis, Vol. 5:84– 95, 2005. [5] S. Islam, M. Bennamoun, and R. Davies. Fast and Fully Automatic Ear Detection Using Cascaded AdaBoost. Proc. of IEEEWorkshop on Application of Computer VisionWACV 2008, Jan. 2008. [6] K. H. Pun and Y. S. Moon. Recent a dvances in ear biom etrics. In Proc. of the Sixth IEEE Int’l Conf. on Automatic Face and Gesture Recognition, pages 164 – 169, May 2004. [7] R. Schapire and Y. Singer. Improved boosti ng algorithms using confidence-rated predictions. Mach. Learn., 37(3):297–336, 1999. [8] P. Yan and K. W. Bowyer. Biometric recognition using 3d ear shape. IEEE Trans. on PAM I, 29(8):1297 – 1308, Aug. 2007. [9] L. Yuan and Z.-C. Mu. Ear detecti on based on skin-color and contour information. In Proc. of the Int’l Conf. on Machine Learning and Cybernetics, Vol. 4:2213 – 2217, Aug. 2007. [10] S. M. S. Isla m, M. Bennamoun, A. S. Mian and R. Davies, Proceedings of 3DPVT'08 - the Fourth Intern ational Sym posium on 3D Data Processing, Visualization and Transmission131 [11] Mohamed Saleh, Sherif Fadel, and Lynn Abbott, Using Ears as a Biometric for Human Recognition, I CCTA 2006, 5-7 September 2006, Alexandria, Egypt [12] K. Chang, K. W. Bowyer, S. Sarkar, and B. Victor, “Comparison and combination of ear and face images in appearance-based bio metrics,” IEEE Trans. Pattern Analys is and Machine Intelligence , vol. 25 no. 9, pp. 1160- 1165, Sept. 2003. [13] D. J. Hurley, M. S. Nixon, and J. N. Carter. Force field feature extraction for ear biometrics. Co mputer Vision and Image Understanding, 98(3):491–512, June 2005. [14] Hoogstrate A.J., Heuvel van den H., Hu yben E., Ear Identification Based on Surveillance Camera’s Images, Ne therlands Forensic Institute, 2000 [15] Chellappa R., Wilson C.L., Sirohey S., Human and Machine Recognition of Faces: A Survey. Pro ceedings of the IEEE. Volume 83. Number 5. May 1995. [16] Turk M., Pentland A., Eigenfaces fo r Recognition. Journal of Cognitive Neuroscience. Volume 3, Number 1. Massachusetts Institute of Technology, 1991. [17] A.K. Jain, A. Ro ss, S. Prabhakar, An introduction to biometric recognition , IEEE Trans. Circuits System s Video Technology, pp. 4–20, 2004. [18] A. Ross, A.K. Jain, “Multimodal Bio metrics: An Overview”, Proc. Of the 12th European Signal Processi ng Conference (EUSI PCO), Vienna, Austria, pp. 1221 - 1224, 2004. [19] W. W. Bledsoe, "The model method in facial recognition," Panoramic Research, Inc., Palo Alto, CA PRI:15, August 1966. [20] M. Kirby and L. Sirovich, "Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces," IEEE T ransactions on Pattern Analysis and Machine Intelligence, vol. 12, pp. 103-107, 1990. [21] M. Turk and A. Pentland, "E igenfaces for Recognition," Journal of Cognitive Neuroscience , vol. 3, pp. 71-86, 1991. [22] D. Swets and J. Weng, "Using Disc riminant Eigenfeatures for Image Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, pp. 831-836, 1996. [23] M. S. Bartlett, “Face Image Analys is by Unsupervised Learning”: Kluwer Academic, 2001. [24] Sanchez-Reillo, R., Sanchez-Avil a, C. and Gonzalez- Marcos, A., “Biometric Identification T hrough Hand Geometry Measurements”, IEEE Transactions on Pattern Analys is and Machine Intelligence, Vol. 22, No. 10, pp. 1168-117 1, Oct. 2000. [25] Zeynep Engin, Melvin Lim, Anil Anthony Bharath: Gradient Field Correlation for Keypoint Correspond ence. 481-484, Proceedings of the International Conference on Image Processing, ICIP 2007, September 16-19, 2007, San Antonio, T exas, USA. 168 http://sites.google.com/site/ijcsis/ ISSN 1947-5500 (IJCSIS) International Journal o f Comput er Science and Info rmation Security, Vol. 6, No. 2, 2009 Nazmeen Bibi Boodoo has done her degree in Computer Science and Engineering at the University of Maur itius. She is currently an MPhil/ PhD student at the University of Mauritius, Reduit, in the Department of Computer Science and Engineering. Her Research areas include Biometric Securit y and Computer Vision. R. K. Subramanian is a professor at the Univers ity of Mauritius, Reduit, in the Department of Computer Science and Engineering. 169 http://sites.google.com/site/ijcsis/ ISSN 1947-5500