Gender Recognition Based on Sift Features

GEDER RE COGITIO  BASED O  SIFT FE ATURES Sahar Yousefi, sahar_yo usefi@ymail.com Morteza Zahedi, zahedi@sh ahroodut.ac.ir School of Techn ology and Computer Engineering, Sh ahrood Univer sit y of Technology, Shahrood, Iran This paper proposes a robu st appro ach for f ace d etection and gender classificat ion in co lor images. Previo us researches about gender recognition suppose an expensive c omputational and time-consuming pre-processing step in order t o alignment in which face images are aligned so that facial landmarks like eyes, nose, lips, chin are placed in uniform locations in image. In this paper, a novel technique b ased on mathematical analy sis i s represented in three stages that eliminates alignment step. First, a new color based face detection method is represented with a better result and more robustness in complex backgr ounds. Next, the features which are invariant to affine transformations are extracted from each face using scale invariant feature transform (SIFT) method. To evaluate the performance of th e proposed algorithm, experiments have been conducted by emp loy ing a SVM class ifier on a database of face images which contains 500 images from dist inc t people with equal ratio of male and female. Keywords : Gender recognitio n; Color space; SVM cla ssifier; SIFT features; Keypoint. 1. Introductio n Automatic gender rec ognition p roblem has a high application pot ential in s ome places where people s hould be s erved depend on their gen der, suc h as c ostumer s tatistics collection in special places like park p laces, supermarkets, restaurants a nd also security surveillance in b uilding entrances . Hence, it is an important pr oblem in co mputer a ided s y stems whic h i nteract with human being. Several approaches have b een proposed to effectivel y segment a nd recognize human genders . But most efforts in t his field are not r obust to s ome changes li ke rotation and illumination. In this work, tha nks to using Sc ale Invariant Feature Transf orm (SIFT) a lgorithm a no vel method for gender rec ognition is prop osed whic h ha s a st rong adaption to misalignment and a ffine transformation. An affine t ransformation is any trans formation that preserves co-linearity r elation b etween points and ratios of distanc es al ong a l ine [ 1,2], like changes of light, rotation, etc. SIFT is a mathemati cal algorithm for extract ing interest point features from images that can be used to perform reliable matc hing between different views of objec ts. Necessa ry pre-processing step pri or to clas sification into male and female group i s r egions of interest (ROIs) segmentation. Gender clas sification is required to delineat e the boundaries of t he ROIs, ensuring that faces are outlined. In this paper, a new color bas ed segmentation m ethod is used whi ch experi ments on complica ted images with m any d etails ha ve shown high performance. The next st ep is extract ing the invariant features and classification of the p ersons' images into the two cat egories of male and female. T here a re s ome bas ic differences between male and female fac es i n ha irline, f orehead, eyebrows , nose, c heeks, chin, etc [ 3]. These characteristics a re s imilar in members of one category and differe nt from members of the other one which c an be extracted by SIFT algorithm finding the interes t points from images that can be used to perform reliable recognition. Finally, a bench mark databa se of face i mages for a gro up of peopl e was employed b y a SVM classifier. The data base contains 500 images f rom distinct p eople with equal ratio of male and fe male. The p roposed a lgorithm was p erformed ten times and in each trial t he database was divided in two p arts randomly, one part for t raining set and the other one for testing. 2. Related Wor ks The first attempt to s olve ge nder cl assification proble m was based o n neur al network and has been d one b y Cottrell et a l. They pr oposed a two layer neural netw ork, on e f or fa ce com press ion an d the other layer i n or der to fa ce c lassification. The result of hidden la yer of compres sion layer was reduced dimensionally si milar to Eigenface method. Such a network indicat es 63% accuracy for a da tabase containing 64 photos [ 4]. In a s imilar method a two-layer full y connected neural network named SEXNET with out di mensionally redu ction is us ed and results reported 91. 9% accuracy for a da tabase c ontaining 90 images [5]. This method was followed for a larger data base and reported 90% accuracy percentage [6] . After that a multi-layer neural network to identify gender from fa ce images of different solutions is used [7]. In another paper a mixture of ra dial basis function (RBF) network and decision tr ee for gender clas sification was us ed [ 8]. Also, Ra dial Bas is Function (RBF) and P erceptron network s for c om paring per formance of these n etworks on raw im age with PCA-bas ed rep resentations was used and the bes t performance of 91.8% has been r eported for a database contains 160 facial images [ 9]. Another articl e is employed support vector machine (SVM) that the result indicates 96.6% accuracy rate with 1755 images by using RB F k ernel [10]. Another literature used the wholist ic features which extracted by independent component anal ysis (ICA) and did classi fication b y using linear d iscr iminate a nalysis (LDA) [11] . In [ 12] Gaussian p rocess classifiers (GPCs) which are Bayesian k ernel classifiers over SVM a re introduced for gender classification which this method improved the resul ts of SVM method. On the ot her ha nd, [ 13] used pixel-pattern-based texture fea ture (PPB TF) for real time gender r ecognition. In this reference Adaboost m ethod used to select the most discrimin ative feature subset and support vector machine (SVM) for c lassification. Ultimately [14] considered the p roblem of gender classif ication from frontal facial image using genetic feature subs et s election and compared so m e methods l ike eigenvector, B a y es, neural netw ork, LDA a nd SVM for a database c ontains 400 fron tal images f rom 400 distinct people. The most literatures about gende r recognition have two constra ints. First, the database images merel y should be c ontaining face region wi thout a dditional de tails i.e. the face part in the picture has not to be located i n front of a complex ba ckground with various c olors and regions. The second, the proposed methods s o far have a n expensive comp utation step for aligning the face s on a pr ototy pe model. The novel method proposed here, not only c an detect the f aces locating in the pic tures with high complexity of ba ckgrounds, but also eliminates the a ligning step b y using the SI FT features whic h are invariant t o affine transformation. 3. Gender Classification There are some ba sic dif ferences between male and female fa cial la ndm arks whic h can be us ed for classification. Generally these c haracteristics could b e applied for a ll ethnic groups [3] . So gender c ould b e distinguished by people characteris tics. In order to find si milar male a nd femal e c haracteristics and matching the m for cl assification, the SIFT method is us ed. Figure 1 presents an overview for our gender c lassification system. This schema has three significant stages containing: • Color base d face detection • SIFT feature extract ion • SVM classif ication Fig. 1. The process of g ender classification 3.1. Color b ased face detection The face detection c ontains two s tages. Fi rst stage i s finding region of i nterest (ROI) that is possibly c ontains fac es. T he next stage is determining if the detected ROIs c ontain face or not. First, HSV color map for finding ski n regions f rom image was employed. This color sp ace i s closer than RGB system to human color perception system. The c onversion from RGB to HSV is p erformed using "Eq.(1)".    > ≤ − = G ifB G ifB , 360 , Hue θ θ (1) )] B , G , R [min( ) B G R ( 3 1 Saturation + + − = ) B G R ( 3 1 Value + + = In which, the θ is defined by "Eq.(2)".               − − + − − + − = − 2 / 1 2 1 )] B G )( B R ( ) G R [( )] B R ( ) G R [( 2 1 cos θ (2) As figure 2 shows there are s ome equations between HSV color map pa rameters (Hue, Saturation, Value) in skin regions. Fig. 2. Relation between HSV parameter i n skin regions Here, a new relation between HSV pa rame ters for finding ROIs was defined. For skin s egmentation pixel set that is s hown in "Eq.(3)" was found by experience. Set Set Set Face Value n Saturatio Hue = ROI − − (3) 200} < Hue < 20 | {Region Hue Region Set = 160} < Saturation < 30 | {Region aturation S Region Set = 255} < alue V < 150 | {Region alue V Region Set = Figure 3 shows t he res ult after finding ROIs for some i mages. Exp erimental re sults demonstrate that this technique is a good skin detection method in complicat ed images. Fig. 3. ROI segmentation using HSV colo r system Next, te mplate matching method, for face determining was u sed. In this t echnique a face template i s convolved wi th ROI regions that found from previous st ep. The face template is mean of some trai ning face images (figure 4). Fig. 4. A Template f or face detection 3.2. SIFT featu re extrac tion SIFT (Scal e invariant feature transform) is a mathematica l f eature ex tracti on method p roposed by David Lowe [ 15]. The extracted features a re invariant to a ny affine tra nsform ation. Under af fine tra nsformations, a ngles, lengths a nd shap es a re not preserved, but the following are so me of the p roperties of a plane figure that remain invaria nt [15, 16]: • The concurrency and colinearit y of corresponding lines and p oints; • The ratio in which a corresponding point divides a corresponding segment. Extracting features is p erforme d by a ca scade filtering approach using a four-stage a lgorithm. 3.2.1. Scale-sp ace extrema detection First a scale space is defined by "Eq. (4)". y) I(x, * ) y, G(x, = ) y, L(x, σ σ (4) Where * is the convolution opera tor, G( x, y, σ) is a v ariab le-scale Ga ussian a nd I(x, y) is the input image. For f inding stab le features difference-of-Gauss ian function convolved with i mage, which ca n be computed with difference of tw o nearby scales, After each octave, the Gaussian image is dow n-sampled by a factor of 2, and the proc ess repeated. y) I(x, * ) y, G(x, = ) y, L(x, σ σ (5) To detect the loca l maxi ma and minima of D(x, y, σ) each point is c ompared to its eight neighbors at t he same scale, plus the nine corresponding neighbors at neighboring s cales. If the pi xel is a local maximum or minimum, it is s elected as a ca ndidate keypoint. 3.2.2. Keypoint l ocalizati on and filte ring This step attempts to eliminate keypoints which are located on edges or the c ontrast between point its neighbors and is low. 3.2.3. Orient ation assig nment In t his stage, one or more ori entation is ass igned t o ea ch k eypo int in order to make them invariant to r otation. Suppose for a keypoint, L is t he image with the closest scale, gradient magnitude a nd orientation can be computed using "Eq. (6)".       − − + − − + = ) 1 y , x ( L ) 1 y , x ( L ) y , 1 x ( L ) y , 1 x ( L ctor Gradientve (6) ))) y , 1 x ( L ) y , 1 x ( L /( )) 1 y , x ( L ) 1 y , x ( L (( tan ) y , x ( 1 − − + − − + = − θ In ne xt stage a gradient histogram (36 b ins) is c reated. Any pea k within 8 0% of the highest pea k is used to crea te a keypoint with that orientation. 3.2.4. Keyp oint descript or The final s tep is to compute a d esc riptor t o make it in variant to re maining variations. For this purpos e a 16×16 Gr adient window is taken that partitioned into 4×4 sub wi ndows. Then, his togram of 4×4 sa mples in eight b ins is created. This result in a featur e v ector c ontaining 128 elements. When at lea st three keys a gree on the model par ameters with low residual, there is s trong e vidence for the p resence of the object. Since there m ay b e dozens of SIFT keys in the i mage of a typical object, it is possible to have substanti al level s of occlusion in the image and yet reta in high levels of reliabilit y [15]. Whereas t here are some b asic differences between mal e and female faces li ke hairlines, f oreheads, noses, eyes, etc, S IF T features are different in two c lasses and these feat ures could be used for gender classification. Figure5 s hows a nd two fem ale t wo mal e photos with keypoint v ectors which SIFT al gorithm finds. Most of the fea tures appear on the eyes, nose, mouth and cheeks. As before was mentioned, most similarities appear on the forehead, e y es, eyebrows and the top of nose. Fig. 5. Keypoint vectors are drawn on the two f emale and two male's face images, nu mbers of keypoints a re (left t o right): 123, 72, 56, 46 respectively. 3.3. S VM classifier Support vector machine (SVM) i s a s upervised learning tec hnique for pattern classification a nd regres sion. SVM is inspired from stat istical learning theory and b ased on concept of str uctural risk minimization. For a given traini ng set n 1 i i p i i i }} 1 , 1 { y , R x | ) y , x {( = − ∈ ∈ where i y indicates t he class to which sa mple i x is b elongs. Eac h i x is a p -dim ensional real vector. The problem is finding the opt imal hyper pl ane that cl assifies the sa mples in two class es. The hyper plane is defined in "Eq.( 7)". ∑ = + = l 1 i i i i b ) x , x ( k . y ) x ( f α (7) Where α a nd b are constants and ) ; (., k is a ker nel function a nd t he sign of ) x ( f determines t he clas s label of sample x. For linear SVM the kernel functi on is dot product of two N-di mensional vectors "Eq.(8)". j i j i x . x ) x , x ( k = (8) While for n on li near SVM, kernel function projects the sa mples to a n Eucli dean feature spac e of hi gher di mensional M via a nonlinear mapping funct ion  M , F R : M  >> → ψ a nd c onstruct a hyper plane in F . In t his case, kernel function defined as "Eq.(9)" , where ψ is the nonlinear pr ojection function. ) x ( ). x ( ) x , x ( k j j ψ ψ = (8) 4. Databa se Although, there are several da tabases available for face r ecognition, the researchers in this field face to the problem of lacking of availab le databases for the gender recognition purpose. Often, in those data bases that are a vailable, person's photo has been repea ted, e.g. AT&T face database containing 4 00 i mages for 40 peop le with 10 images per pers on [ 17]. In gender recognition p roblem for p urpose of avoiding repetition of pat terns in t raining, vali dation and tes t p arts should be selec ted one image per person. If we do this in AT&T database, images are redu ced to 40 individual images . Hence, we have been collect ed a d atabase contains 500 images f rom 5 00 d istinct people w ith different facial exp ression and under diff erent lighting conditions and equa l rate of males and females i n RGB form at. 5. Experi ment results In this pap er, a new app roach based o n SIFT features was p roposed f or gender recognition. The whole proces s of gender classificat ion can be explained in face detection, feature extrac tion and classification steps. For examining the proposed method, the collec ted database was used. T he da tabase contains mal e and fem ale's images, with same proportion. Images were pre processed and c ropped by a novel color face detection method. F or this goa l s kin regions were segmented using HSV color space parameters and then faces were detected using a face template which made by averaging training fa ces. Then, fe atures were e xtracted from all the p eople i mages in t he da tabas e using SIFT algorithm. For each image face 150 more frequent SIFT keypoints. Each k eypoint desc riptor contains 128 a ttributes that describe that region 2 2 )) 1 y , x ( L ) 1 y , x ( L ( )) y , 1 x ( L ) y , 1 x ( L ( ) y , x ( m − − + + − − + = in a scale a nd orientat ion in variant wa y. S IFT k eypoints can be seem as t he fingerprint of images, which each fingerprint identifies a un ique feat ure of an im age, hence e nables us t o discover similar fe atures across different images. Ultimately, the extracted SIFT features were rendered to a SVM class ifier for purpose of gender classification. There are distinctive differences between m ale and f emale f aces in forehead, cheek, l ips, ey es, etc . Therefore, recognition is bas ed on these differences. In other words, differences between male and female fac es and the keypoints whic h were extracted from faces for c lassifying wer e used. Th e novel a pproach was perform ed ten ti mes and in each r epetition the data base was d ivided randomly i n two parts, 9 0% for tr aining a nd 10 % for testing. Th e gen der c lassif ication was performed using some kernel functions (linear, Quadra tic, RBF). Table1 summarized the experiment results . Table 1. Accurate percentage of experiments Kernel classifier Accuracy percent Variance Linear kernel 81.15 6.2142 Quadratic kernel 79.09 6.3506 RBF kernel 86.842 5.1897 6. Conclusion This paper demonstrates a novel technique for fa ce detection based o n c olor space and gender recognition based on SIFT features. The proposed face detect ion technique with ab ility of detecti ng the faces locating in complex ba ckgrounds a nd the SIFT features with abili ty of finding the interest p oints of the faces describing the disc riminate characteristic s of male and female groups, l ead us to const ruct a system for gender rec ognition with high robustness in i nput images and acc uracy of recognition o utput. By employing a nonlinear SVM classifi er the propose d method yields a re cognition rat e of 87 % which is appreciable comparing to the other methods with constraints of having simple back grounds and being time consuming. References 1. De Villiers, M., 1 993, The affine invariance and lin e symmetries of the conics, Australian Senior Mathematics Jour nal, 7(2), 32–50. 2. Yaglom, Y., 1973, Geo metric Transformations III , 2nd print (Washington: MAA ), pp. 9–20. 3. Virtual Facial Femi nisation: http://www .virtualffs.co.uk/index.html 4. G . Cottrell and M. Fleming: Face recognition using unsupervised fea ture extraction. In: International Conference on Neural Network, pp. 322–325, 1990. 5. Golomb B., L awrence D., Sejnowski T.: Sexn et: a neural ne twork i dentifies sex from human faces. In: Advances in Neural Informatio n Processing Systems, pp. 572–577. M organ Kaufmann, Sa n Mateo, 1991. 6. S. Yen, P. Sajda and L. Finkel: Comparison of gender recog nition by pdp and radial basis function networks, In: The Neurobiology of Computation, pp. 433–438, 1 994. 7. Tamura S.H., Kawai, Mitsumoto H.: Male/female identification from 8×6 very low resolution face ima ges by neural net work. I n: Pattern Recognition, pp. 331–3 35, 1996. 8. G utta, S., Weschler, H., Phillips, P.J.: G ender a nd ethnic classificatio n of human faces using hybrid c lassifier s. In: Proceedings of IEEE International Conf erence on Automatic Face and G esture Recognition, pp. 1 94–199, 1998. 9. H. Abdi, D. Valentin, B. Edelm an and A. O’Toole: More about the difference between men and women: e vidence from linear neural networks and the principal compon ent approach. In: Perce ption, vol. 24, pp. 539–562, 1 995. 10. Moghaddam B. , Yang M.H.: Gender cla ssification with support vector mac hines. In: IE EE Trans. PAMI 24, pp. 707–711, 2002. 11. Jain, A., Huang, J., Fang, S.: G ender identification using frontal facial images. In: Proceedings of IEEE Conference on M ultimedia and Expo, pp. 1082–1085, 2005. 12. Kim H., Kim D., Ghahramani Z., Bang S.: Appea rance- based gender classi fication with Gaussian processes. In: pattern recog nition, pp. 618-626, 2006. 13. Haung Y., Chen Y., D. Y ang: Automatic gender recognition based on pixel-pattern-based texture f eature. In: Real-time image proc, pp. 109-116, 2008. 14. Zehang Sun, George Bebis, Xiaojing Yuan and Sushil J. L ouis: G enetic Feature Subset Selection for Gender C lassif ication: A Comparison Study. 15. Lowe David G., Object Recognition from Local Scale-Invariant Features. In: International Conference on Computer V ision, Canada , 1999. 16. Zhao, W., Xu, W. and Yu, Y.: Area Harmony Dominating Recti ficatio n Met hod for SIFT Image Mateh ing. In: The Eighth Internayiona l Conference on Electronic Mea surement and Instruments, ICE MI, vol.2, pp. 935-939, 2007. 17. AT&T f ace database: http://www.uk.research.att.com/f acedatabase.html 18. Ro bert van der Linden, J oel van Nee rbos, Tim Havinga: Di stinctive Image Features f rom Sca le-I nvariant Keypoints, University o f Groningen. 19. David G. Lowe: Distinctive Image Features fro m Scale-Invariant Key points, 2004. 20. Krzysztof Slot1 , Hyongsuk Kim2: Keypoints Derivation for Obj ect Class Detection with SIFT Algorithm, In: ICAISC, pp. 850–859, 2006. 21. Basri, Ronen, and David.W: J acobs, Recognition u sing region correspondences. I n: International Journal of Compute r Vision, pp. 141– 162, 1996.